|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/master-airflow-beginner-to-advance-with-project/
课程评论：没有评论
课程名称：掌握 Airflow：从入门到高级与项目实践 课程概述： Apache Airflow 是一个开源平台，用于工作流自动化、调度和复杂数据管道的 orchestration。随着数据量和复杂度的不断增长，效率和可扩展性在数据处理和管理中变得尤为重要。在这门综合课程中，您将从基础到高级概念，全面掌握 Apache Airflow 的使用。课程专为数据工程师、数据科学家、Python 开发者、软件工程师以及任何希望学习如何自动化和管理数据工作流的人士而设计。 您将学习如何使用 Apache Airflow 构建和管理数据管道、调度和触发任务、监控和排除工作流故障，以及与各种数据源和服务集成。课程内容包括： - Apache Airflow 和工作流管理基础 - Docker 介绍及命令 - Apache Airflow 的安装和配置 - 使用 Apache Airflow 构建和管理工作流 - 在 Apache Airflow 中调度和触发任务 - Apache Airflow 中的操作符 - 从 Web API 或 HTTP 获取数据 - Apache Airflow 中的文件传感器 - 与 Azure 或 AWS 的连接 - 使用 AWS S3 存储桶和 Azure Blob 存储来存取数据 - 创建自定义操作符和传感器 - 处理依赖关系和任务重试 - 监控和排除工作流故障 - 与数据源和服务的集成 - 使用 Celery Executors 扩展和优化 Apache Airflow，以进行大规模数据处理 - 使用 Fernet Keys 保护 DAG 连接 在课程中，您将通过实际练习和项目应用所学概念。课程结束时，您将对 Apache Airflow 有深入的理解，并具备构建和管理复杂数据工作流的技能。
Apache Airflow is an open-source platform used for workflow automation, scheduling, and orchestration of complex data pipelines. As data volumes and complexity continue to grow, the need for efficient and scalable data processing and management is critical. In this comprehensive course, you will learn how to master Apache Airflow, starting from the basics and progressing to advanced concepts.The course is designed for data engineers, data scientists, python developers, software engineers, and anyone interested in learning how to automate and manage data workflows. You will learn how to use Apache Airflow to build and manage data pipelines, schedule, and trigger tasks, monitor and troubleshoot workflows, and integrate with various data sources and services.The course will cover the following topics:Introduction to Apache Airflow and workflow managementIntroduction to Docker and Docker CommandsInstallation and configuration of Apache AirflowBuilding and managing workflows with Apache AirflowScheduling and triggering tasks in Apache AirflowOperators in Apache AirflowFetching data from Web APIs or HTTPFile Sensors in Apache AirflowConnecting with Azure or AWSUsing AWS S3 Bucket and Azure Blob Storage to store and retrieve dataCreating custom operators and sensorsHandling dependencies and task retriesMonitoring and troubleshooting workflowsIntegrating with data sources and servicesScaling and optimizing Apache Airflow for large-scale data processing using Celery ExecutorsSecuring Dags Connections using Fernet KeysThroughout the course, you will work on practical exercises and projects to apply the concepts you learn. By the end of the course, you will have a strong understanding of Apache Airflow and the skills to build and manage complex data workflows.
