|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/apache-airflow-using-docker-docker-swarm/
课程评论：没有评论
**课程名称：** 使用 Docker 和 Docker Swarm 进行 Apache Airflow 2.0 **课程摘要：** 本课程旨在帮助您掌握如何利用 Docker 和 Docker Swarm 构建高可用、弹性、分布式 and fault-tolerant 的 Airflow 2.0 环境。 **课程特色：** * **解决实际痛点：** 作者分享了在生产环境中配置 Airflow 时遇到的挑战，例如为每个 Airflow 组件编写 Docker Compose 文件，以及市场上缺乏支持 Docker Swarm 的 Airflow 镜像。 * **零基础入门 Docker：** 即使您没有 Docker 经验，课程也会涵盖所有必要的 Docker 概念，确保您能跟上进度。 * **深入 Airflow 2.0：** 学习 Airflow 2.0 相较于 1.x 的优势，并深入了解 Airflow 的高可用架构和系统需求。 * **实战操作：** 在 AWS 上获取机器，通过 Docker Compose 逐一容器化 Airflow 组件，并进行多调度器基准测试。 * **Docker Swarm 迁移：** 学习 Docker Swarm 的概念、架构、命令和网络，并将 Docker Compose 文件转换为 Docker 服务命令。最终，将 Airflow 服务部署到 Swarm 集群中。 **您将学到：** * 如何使用 Docker 容器化 Airflow 的各个组件。 * Airflow 2.0 的高可用架构设计及实现。 * Docker Swarm 的基础知识和高级用法。 * 如何使用 Docker Swarm 轻松部署弹性、分布式的 Airflow 集群。 * 为生产环境构建稳定可靠的 Airflow 系统。
Why Am I conducting this course ?When I started configuring airflow in my organisation, I spent many weeks on writing Docker Compose files for each airflow component. Airflow community provides a single docker compose file which installs all the components in a single machine. It works fine for practising purposes. But in production, we setup each component on different machines. Also, there is no docker image available on Docker registry to start airflow through Docker Swarm. Overall, I contributed many sleepless nights to achieve fault tolerant, resilient, distributed, Highly Available airflow using Docker Swarm. I consolidated all my learnings, knowledge into this course. So that others don't need to struggle like mine. What do you Learn From this Course ?The primary objective of this course is to achieve resilient airflow using the Docker and Docker Swarm. I am using the latest stable airflow (2.0.1) throughout this course.At first, we study all the required docker concepts. Don't worry if you have no prior experience on docker. I cover all docker necessary concepts which are used in this course. In the second module, we investigate Airflow 2.0 and understand the additional advantage over airflow 1.x. We discover the airflow HA architecture and discuss each system requirement.After this, we acquire these machines from AWS and start containerising one by one applications using docker compose.At the end, we run multiple airflow schedulers and benchmark it.Last module is on docker swarm and we witness how easy it is to setup the entire airflow just by running a few swarm commands.In the beginning, we learn all the swarm concepts, architecture, commands and networking. We translate the docker compose file into the docker service command. Subsequently, We create a Swarm Cluster and trigger one by one airflow services on it.
