|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/apache-airflow-complete-distributed-configuration/
课程评论：没有评论
**课程名称：Apache Airflow 2.0：完整的分布式配置** **课程概述：** 本课程旨在教授学员如何构建一个完全分布式的Apache Airflow 2.0环境，并能够同时运行超过100个作业或DAG。Airflow是一个由社区开发的开源工作流编排平台，因其简洁性、可扩展性和可扩展性而被广泛应用。 课程将涵盖三种执行器（Executor）的配置：Sequential、Local和Celery Executor。学员将通过AWS EC2实例来实践这些配置。 课程重点介绍Airflow 2.0带来的显著改进，包括高可用调度器（HA Scheduler）、调度器和Celery Worker的性能大幅提升。学员将学习Airflow 2.0的新增强功能和高可用架构。 **具体内容包括：** * **安装与配置：** * 安装Webserver、Scheduler、Celery Worker和Flower组件。 * 配置Sequential、Local和Celery Executor。 * 在AWS EC2实例上进行实际操作。 * **Airflow 2.0 新特性：** * 高可用（HA）调度器架构。 * 新增强功能和性能提升。 * 配置多个调度器并观察其性能。 * **关键功能探索：** * 用户登录。 * 邮件告警（Email Alerting）。 * 日志管理（Logs Management）。 **课程目标：** 通过本课程的学习，学员将能够搭建一个强大的分布式Airflow环境，并能供组织内的多个团队共享使用，有效提升工作流的管理和执行效率。
Airflow is a platform created by the community to programmatically author, schedule and monitor workflows. Airflow is one of the best open source orchestrators and it is used widely because it is simplicity, scalability and extensibility.Main goal of this course is to achieve an Airflow distributed setup using Celery Executor and be able to run more than 100 jobs or DAGs in parallel at any instance in time. I cover Sequential, Local and Celery Executor in this course. We acquire few EC2 instances from AWS and configure these executors. Airflow community recently released Airflow 2.0. It contains many amazing features like HA Scheduler, Massive performance improvement on scheduler as well as celery workers etc. I myself fascinated by airflow 2.0 performance and migrated all airflow 1.x to 2.x in my organisation.I am adding a new module on apache airflow 2.0. Here, we begin the module by learning new enhancements and HA architecture of airflow 2.0. Next, we install Webserver, Scheduler, Celery workers and Flower components. At the end, we configure multiple schedulers and observe its performance.In addition to this, we explore salient features like Login, Email alerting and Logs management.By the end of this course, you own a great distributed airflow setup which could be shared by multi teams in your organisation.
