|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/the-ultimate-hands-on-course-to-master-apache-airflow/
课程评论：没有评论
课程名称：Apache Airflow：实用指南 课程概述：Apache Airflow是由社区创建的平台，用于程序化地编写、调度和监控工作流。其可扩展性、动态性和模块化使其成为数据工作者必备的技能之一。 课程学习内容： - 理解Airflow的基本概念，包括调度器和网页服务器的工作原理。 - 通过外汇数据管道项目深入探索多个Airflow操作符，涉及Slack、Spark和Hadoop等工具。 - 学习如何优化DAG（有向无环图），包括时间区处理、单元测试、DAG文件夹结构等。 - 详细了解如何通过不同的执行器（如本地执行器、Celery执行器和Kubernetes执行器）扩展Airflow。 - 构建一个包含3个节点的Kubernetes集群，使用Rancher、Airflow和本地Kubernetes执行器来运行数据管道。 - 通过实际案例学习高级概念，如DAG模板化、DAG间依赖性、子DAG及死锁问题等。 - 在AWS EKS和Rancher云端设置Kubernetes集群以使用Airflow和Kubernetes执行器。 - 学习如何通过Elasticsearch和Grafana监控Airflow。 - 课程还涵盖安全性问题，包括用户角色和权限管理、身份验证和密码保护、数据加密等。 此外，课程提供了丰富的实践练习和最佳实践指导，以帮助学员巩固所学知识。每个章节结束时会有测验，帮助评估理解程度。讲师将快速响应学员的问题，致力于提供优质的学习内容。完成课程后，您将对使用Airflow充满信心，预祝您取得成功！ 讲师：Marc Lamberti
Apache Airflow is a platform created by the community to programmatically author, schedule and monitor workflows.It is scalable, dynamic, extensible, and modulable. Without any doubt, mastering Airflow is becoming a must-have and an attractive skill for anyone working with data.What you will learn in the course:Fundamentals of Airflow are explained such as what Airflow is, how the scheduler and the web server workThe Forex Data Pipeline project is an incredible way to discover many operators in Airflow and deal with Slack, Spark, Hadoop, and moreMastering your DAGs is a top priority, and you can play with timezones, unit test your DAGs, structure your DAG folder, and much more.Scaling Airflow through different executors such as the Local Executor, the Celery Executor, and the Kubernetes Executor will be explained in detail. You will discover how to specialize your workers, add new workers, and what happens when a node crashes.A Kubernetes cluster of 3 nodes will be set up with Rancher, Airflow, and the Kubernetes Executor local to run your data pipelines.Advanced concepts will be shown through practical examples such as templating your DAGs, how to make your DAG dependent on another, what are Subdags and deadlocks, and more.You will set up a Kubernetes cluster in the cloud with AWS EKS and Rancher to use Airflow and the Kubernetes Executor.Monitoring Airflow is extremely important! That's why you will know how to do it with Elasticsearch and Grafana.Security will also be addressed to make your Airflow instance compliant with your company. Specifying roles and permissions for your users with RBAC, preventing them from accessing the Airflow UI with authentication and password, data encryption, and more.In addition:Many practical exercises are given along the course so that you will have occasions to apply what you learn.Best practices are stated when needed to give you the best ways of using Airflow.Quiz are available to assess your comprehension at the end of each section.Answering your questions fast is my top priority, and I will do my best for you.I put a lot of effort into giving you the best content, and I hope you will enjoy it as much as I wanted to do it. At the end of the course, you will be more confident than ever in using Airflow.I wish you a great success!Marc Lamberti
