|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/from-0-to-1-the-oozie-orchestration-framework/
课程评论：没有评论
课程名称：从0到1：Oozie编排框架 本课程将深入介绍Apache Oozie，一个强大的工作流调度系统，用于管理和协调Apache Hadoop生态系统中的数据处理任务。 **课程亮点：** * **实战经验丰富的团队授课：** 由两名斯坦福大学毕业、前谷歌员工以及两名前Flipkart首席分析师组成的团队，拥有数十年大规模数据处理实战经验。 * **Oozie的强大与挑战：** Oozie如同一个高效的“行政助理”。它虽然以XML为基础，上手有一定门槛，但一旦掌握，便能轻松处理复杂的依赖关系、多任务调度以及端到端的数据管道管理。 * **全面的内容覆盖：** * **工作流管理 (Workflow Management)：** 学习工作流定义、执行节点（Action nodes）、控制节点（Control nodes）、全局配置，并通过MapReduce和Shell等实际示例进行操作和优化。 * **时间/数据触发器 (Triggers)：** 掌握Coordinator的规范，模拟cron作业，设置时间或数据可用性触发器，处理积压任务，并运行时间触发和数据触发的Coordinator动作。 * **数据管道 (Data Pipelines)：** 学习Bundle的规范，理解Bundle的启动时间，以及如何利用Oozie运行Bundle。 **先修知识：** * 对Hadoop生态系统有基本了解。 * 熟悉运行MapReduce作业。 **学习目标：** 通过本课程，您将学会如何利用Oozie有效地管理和调度您的Hadoop作业、Java程序、脚本以及其他可执行文件，确保数据处理流程的顺畅和可靠。掌握Oozie的核心能力，如同学会“如何提出正确的要求”，从而轻松驾驭复杂的数据管道。
Prerequisites: Working with Oozie requires some basic knowledge of the Hadoop eco-system and running MapReduce jobs Taught by a team which includes 2 Stanford-educated, ex-Googlers and 2 ex-Flipkart Lead Analysts. This team has decades of practical experience in working with large-scale data processing jobs. Oozie is like the formidable, yet super-efficient admin assistant who can get things done for you, if you know how to ask Let's parse that formidable, yet super-efficient: Oozie is formidable because it is entirely written in XML, which is hard to debug when things go wrong. However, once you've figured out how to work with it, it's like magic. Complex dependencies, managing a multitude of jobs at different time schedules, managing entire data pipelines are all made easy with Oozie get things done for you: Oozie allows you to manage Hadoop jobs as well as Java programs, scripts and any other executable with the same basic set up. It manages your dependencies cleanly and logically. if you know how to ask: Knowing the right configurations parameters which gets the job done, that is the key to mastering Oozie What's Covered: Workflow Management: Workflow specifications, Action nodes, Control nodes, Global configuration, real examples with MapReduce and Shell actions which you can run and tweak Time-based and data-based triggers for Workflows: Coordinator specification, Mimicing simple cron jobs, specifying time and data availability triggers for Workflows, dealing with backlog, running time-triggered and data-triggered coordinator actions Data Pipelines using Bundles: Bundle specification, the kick-off time for bundles, running a bundle on Oozie
