|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/pyspark-python-spark-hadoop-coding-framework-testing/
课程评论：没有评论
课程名称：PySpark：Python、Spark和Hadoop编码框架与测试 课程概述：该课程旨在弥补学术学习与实际应用之间的差距，为您准备入门级的Big Data Python Spark开发者角色。您将获得实践经验，并学习开发Python Spark应用程序的行业标准最佳实践。课程涵盖Windows和Mac环境，确保您在任何操作系统上都能顺利学习。您将学习Python Spark编码的最佳实践，以编写干净、高效且易于维护的代码。课程中还将介绍日志技术，以帮助您跟踪应用程序行为并有效排除问题；错误处理策略将确保您的应用程序稳健且具容错性。此外，您将学习如何从属性文件读取配置，使您的代码更具适应性和可扩展性。 主要模块： - 使用PyCharm编写Python Spark编码的最佳实践，以确保代码的干净、高效和可维护性 - 实施日志来跟踪应用程序行为并排除问题 - 错误处理策略以构建稳健且具容错能力的应用程序 - 从属性文件中读取配置以实现灵活和可扩展的代码 - 在Windows和Mac环境中使用PyCharm开发应用程序 - 设置并使用本地环境作为Hadoop Hive环境 - 使用Spark读写Postgres数据库中的数据 - 使用Python单元测试框架来验证您的Spark应用程序 - 使用Hadoop、Spark和Postgres构建完整的数据管道 课程先决条件： - 基本编程技能 - 基本数据库知识 - 对Hadoop的入门级理解
This course will bridge the gap between academic learning and real-world applications, preparing you for an entry-level Big Data Python Spark developer role. You will gain hands-on experience and learn industry-standard best practices for developing Python Spark applications. Covering both Windows and Mac environments, this course ensures a smooth learning experience regardless of your operating system.You will learn Python Spark coding best practices to write clean, efficient, and maintainable code. Logging techniques will help you track application behavior and troubleshoot issues effectively, while error handling strategies will ensure your applications are robust and fault-tolerant. You will also learn how to read configurations from a properties file, making your code more adaptable and scalable. Key Modules: Python Spark coding best practices for clean, efficient, and maintainable code using PyCharmImplementing logging to track application behavior and troubleshoot issuesError handling strategies to build robust and fault-tolerant applicationsReading configurations from a properties file for flexible and scalable codeDeveloping applications using PyCharm in both Windows and Mac environmentsSetting up and using your local environment as a Hadoop Hive environmentReading and writing data to a Postgres database using SparkWorking with Python unit testing frameworks to validate your Spark applicationsBuilding a complete data pipeline using Hadoop, Spark, and PostgresPrerequisites:Basic programming skillsBasic database knowledgeEntry-level understanding of Hadoop
