|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/the-complete-hands-on-course-to-master-apache-airflow/
课程评论：没有评论
**课程名称：** Apache Airflow 3 全方位实战入门 **课程概述：** 本课程旨在帮助您全面掌握 Apache Airflow 3 的使用。Airflow 是一个强大的开源平台，用于以编程方式创建、调度和监控工作流，特别适合管理大量 ETL 任务。 **学习内容：** * **基础知识：** * 什么是 Apache Airflow？ * Airflow 的核心概念 * Airflow 的不同架构 * 工作流的运行机制 * **实际应用：** * 创建第一个数据管道，涵盖 Sensors（等待特定条件）、Hooks（与数据库交互）、Taskflow（编写高效的 DAG）、XCOMs（数据共享）等 Airflow 特性。 * 掌握 Airflow 3 中全新的 Asset 语法，包括 Asset 的定义、依赖关系创建和物化。 * **扩展与优化：** * 深入了解 AtScale 部署的各种执行器（Executors），特别是 CeleryExecutor。 * 学习如何配置 Airflow 以使用 CeleryExecutor，实现任务在不同 Worker 上的分布式执行，并通过 Queues 选择 Worker。 * **高级特性：** * 探索高级功能，提升 DAG 的复杂性和效率。 * 学习创建自定义 Airflow Provider 和执行 SQL 请求的新 Decorator。 **目标受众：** * 需要管理大量 ETL 任务的工程师。 * 希望深入了解并实际应用 Apache Airflow 3 的技术人员。 * 在公司中使用 Airflow 的开发者。 **结业成果：** 完成本课程后，您将具备独立创建和管理 Airflow 工作流的能力，并能熟练运用 Airflow 3 的新特性应对复杂的数据处理需求。
Apache Airflow is an open-source platform for programmatically authoring, scheduling, and monitoring workflows. If you have many ETLs to manage, Airflow is a must-have.In this course, you are going to learn everything you need to start using Apache Airflow 3 through theory and practical videos.You will start with the basics such as:What is Apache Airflow?The core concepts of AirflowDifferent architectures to run AirflowWhat happens when a workflow runsThen you will create your first data pipeline covering many Airflow features such as:Sensors, to wait for specific conditionsHooks, for interacting with a databaseTaskflow, for writing efficient, easy-to-read DAGsXCOMs, for sharing dataand much more.At the end of the project, you will be equipped for creating your own workflows!After the project, you will also discover the new Asset syntax that completely change your way of thinking about your tasks in Airflow 3.What is an AssetHow to create dependencies between AssetsHow to materialize an Assetand more.You will discover the different executors for running Airflow at scale. More specifically, the CeleryExecutor which is extremely popular.How to configure Airflow for using the CeleryExecutorHow to distribute your tasks on different WorkersHow to choose your Workers with Queuesand more.You will explore advanced features to elevate your DAGs to a new level, and conclude by creating your own Airflow provider and a new decorator for executing SQL requests.If you're working in a company with Airflow, you will love that part.Enjoy
