|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/apache-iceberg-fundamentals-to-mastery/
课程评论：没有评论
课程名称：Apache Iceberg + Snowflake：端到端数据湖指南 概述：本课程分为八个部分，旨在深入探讨Apache Iceberg与Snowflake的结合，帮助学员建立高效的数据湖。 1. **为什么选择Iceberg**：本部分将帮助您了解Iceberg的重要性及传统数据仓库架构所面临的挑战。 2. **Iceberg环境设置**：我们将在GitHub Codespaces中设置Spark环境，作为一个实验场所，您可以运行Iceberg命令并进行实践。 3. **Parquet文件格式**：深入讲解Parquet文件格式，以建立扎实的基础理解，因为Iceberg是基于Apache Parquet构建的，并利用其结构实现高效存储和查询。 4. **Iceberg功能特性**：将探讨Iceberg的关键特性，如隐藏分区、模式演化和时间旅行，以了解它如何解决传统数据湖的常见限制。 5. **Iceberg概念**：了解Copy-on-Write（COW）、Merge-on-Read（MOR）和快照隔离等概念，以深入理解Iceberg如何管理数据并确保数据的一致性。 6. **Iceberg与Snowflake结合**：配置Iceberg与Snowflake，并研究其集成方式，帮助您理解在Snowflake生态系统中使用Iceberg的基础概念。 7. **使用Snowflake Iceberg构建数据湖**：将使用Snowflake Iceberg构建示例数据湖，并演示如何从Spark查询Iceberg表，以实现跨平台互操作性。 通过本课程的学习，您将全面了解Iceberg表格式的优势、应用场景，以及如何利用Iceberg构建高效的数据湖。
This course is broadly divided into 8 sections,Why Iceberg:This will help you understand the significance of Iceberg and the challenges associated with traditional data warehouse architectures.Iceberg environment setup:We'll set up a Spark environment with Iceberg in GitHub Codespaces. This will serve as a playground where you can run Iceberg commands and experiment hands-on.Parquet file format:We'll dive deep into the Parquet file format to build a strong foundation. Understanding Parquet is essential because Iceberg is built on top of Apache Parquet and leverages its structure for efficient storage and querying.Iceberg features:We'll explore key Iceberg features such as hidden partitioning, schema evolution, and time travel to understand how it addresses common limitations in traditional data lakes.Iceberg concepts:We'll explore concepts like Copy-on-Write (COW), Merge-on-Read (MOR), and snapshot isolation to gain a deeper, more concrete understanding of how Iceberg manages data and ensures consistency.Iceber with snowflake:We'll configure Iceberg with Snowflake and explore how Iceberg integrates with it, helping us understand the foundational concepts of using Iceberg within the Snowflake ecosystem.Datalake with snowflake Iceberg:We'll build a sample data lake using Snowflake Iceberg and also demonstrate how to query Iceberg tables from Spark for cross-platform interoperability.By the end of this course, you'll have a solid understanding of the Iceberg table format-its advantages, use cases, and how to build an efficient data lake using Iceberg.
