|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/apache-druid-complete-guide/
课程评论：没有评论
**Apache Druid: Complete Guide 课程总结** 本课程是一份关于 Apache Druid 的全面指南，涵盖了从理论基础到实战集成。 **核心学习内容：** * **Druid 理论与核心特性：** 深入理解 Druid 的基本概念、架构和关键功能。 * **实时数据摄取：** * 构建一个 Twitter Producer 应用程序，实时抓取推文。 * 将推文推送到 Apache Kafka。 * 配置 Druid Kafka 流式摄取任务，从 Kafka 读取数据并存储到 Druid。 * 学习在摄取过程中进行数据转换、过滤、模式配置和性能调优。 * **批量数据摄取：** * 深入学习 Druid 的原生（Native）和 SQL 批量摄取方法。 * 自动化 ETL 流程中的数据加载到 Druid 环节。 * **与 Spark 集成：** * 学习如何使用 Spark 读取 Druid 表，并创建 Spark DataFrame。 * 探索 Spark 的谓词下推（predicate pushdown）和聚合下推（aggregate pushdown）功能。 * **与 Schema Registry 集成：** * 了解 Druid 如何与 Schema Registry 交互，实现模式验证。 * 学习 Druid 如何解析 Avro（或其他模式）记录。 * **扩展集成能力：** * **Hive 集成：** 学习如何在组织数据存储在 Hive 时，实现与 Druid 表的连接和联合查询。 * **Presto（Trino）集成：** 学习如何在组织数据存储在 Presto/Trino 时，实现与 Druid 表的连接和联合查询。 本课程旨在帮助学习者全面掌握 Apache Druid 的各项功能，并能在实际工作中将其与其他大数据组件（如 Kafka, Spark, Hive, Presto, Schema Registry）进行高效集成，满足复杂的数据分析和实时查询需求。
What do you learn from this course ?In this course, we learn end-to-end apache druid salient features and integration with Apache Hive, Prestodb (Trino), Spark and Schema Registry one by one.We start this course by gaining theoretical knowledge on Druid and its key features. We write our own Twitter Producer app which pulls the tweets from twitter in realtime and push the tweets to apache Kafka. We create a Kafka streaming task on Druid which pull tweets from Kafka and store it into Apache Druid. Also, we learn how to apply transformation, filter, schema configuration, tuning during kafka ingestion.In the 3rd module, we explore Native and SQL Batch ingestion methods in depth. In an ETL pipeline after extract and transform step, if you want to load the dataset to druid, then you must checkout this section. We will automate the entire loading to druid part.In the 4th module, we learn how to read druid tables using Spark and create Spark Dataframe from it. We also explore the predicate and aggregate pushdown spark features.Section 5, talk about the Schema Registry. We learn how druid talk to schema registry and achieve the schema validation. Also how druid parse the Avro records.Section 6 & 7 exposes the out of box druid capabilities. which are hive and presto integration. If your organisation data resides in hive or presto and you would like to join with druid table, then you should accomplish hive or presto integration.
