|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/pyspark-foundation-for-data-engineering-begineers/
课程评论：没有评论
Coursera 课程总结：PySpark 数据工程入门 本课程旨在帮助初学者掌握 PySpark，为应对真实世界的数据工程师角色做好准备。课程重点在于 PySpark 的实际应用，弥合理论知识与实践技能之间的鸿沟。 **关于 PySpark：** 您将了解并运用 Spark，这项最新的大数据技术，并结合最流行的编程语言 Python。分析海量数据集是极具价值的技能，而 Apache Spark 是实现这一目标的最优技术之一。Google、Facebook、Netflix、Airbnb、Amazon、NASA 等顶级科技公司都在使用 Spark 解决大数据问题。Spark 的性能比 Hadoop MapReduce 快高达 100 倍，这极大地推动了对这项技能的需求。由于 Spark 2.0 DataFrame 框架的创新性，您有机会迅速成为就业市场上的专家。 **课程内容：** * **SparkSession 和导入**：学习如何启动和配置 Spark 环境。 * **Spark DataFrame**：深入理解 Spark DataFrame 的特性、语法和示例，并学会打印结果。 * **数据理解**：掌握如何查看数据记录数、列名，描述 DataFrame，以及理解 Schema。 * **DataFrame 操作**： * 创建新列，包括进行算术运算、更改列数据类型、创建常量列（整数和字符串）。 * 应用实际操作，如四舍五入、排序、删除列、重命名列。 * 使用条件语句和更改列的大小写。 * 进行过滤操作。 * 掌握分组和聚合（aggregations）。 **先修知识：** * 具备一些基础编程技能（非强制）。 * 具备将理论知识付诸实践的意愿。 **适合人群：** * 希望学习大数据技术的初学者。 * 希望转向大数据领域的经验人士。 * 希望学习实际编码技巧的大数据初学者。 * 有志于成为数据工程师的候选人。
This course will prepare you for a real world Data Engineer role (basics)!Learn to code PySpark like a real world developer. Here our major focus will be on Practical applications of PySpark and bridge the gap between academic knowledge and practical skill.In this course we will get to know and apply few of the most essential and basic functions in PySpark, that are used frequently in scripting for any project based on PySpark.About PySpark:Learn the latest Big Data Technology - Spark! And learn to use it with one of the most popular programming languages, Python!One of the most valuable technology skills is the ability to analyze huge data sets, and this course is specifically designed to bring you up to speed on one of the best technologies for this task, Apache Spark! The top technology companies like Google, Facebook, Netflix, Airbnb, Amazon, NASA, and more are all using Spark to solve their big data problems!Spark can perform up to 100x faster than Hadoop MapReduce, which has caused an explosion in demand for this skill! Because the Spark 2.0 DataFrame framework is so new, you now have the ability to quickly become one of the most knowledgeable people in the job market!What you will learn:SparkSession and importsSpark DataFrame and its characteristicsSyntax and examplePrint resultsUnderstanding the dataNumber of recordsColumns in dataFrameDescribe a DataFrameSchema of a DataFrameCreate a new columnArithmetic operations on DataChange column data typeCreate a column with integer as constantApply what we knowRounding of digitsSorting operationDrop columnsRename columnsCreate a column with string as constantConditional StatementsChanging case of a columnFilter operationsGrouping and aggregationsPrerequisites:Some basic programming skills (Not Mandatory)Will to implement theoretical knowledge in practical.Who this course is for:Beginners who want to learn Big Data or experienced people who want to transition to a Big Data roleBig data beginners who want to learn how to code in the real worldAspiring candidates for data engineering role
