|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/data-lake-in-aws/
课程评论：没有评论
课程名称：AWS中的数据湖 - 最简单的学习方式 [2025] 概述： 您好，我是Chandra Lingam，您在AWS数据湖课程中的讲师。在本课程中，我们将首先了解数据湖的基本概念以及何时选择数据湖而不是数据仓库。接着，我们将深入探讨构成数据湖解决方案的各种组件，包括直接使用SQL对文件进行查询，以便快速进行数据集的临时分析。 课程中将涉及处理数据湖中文件结构变化的主题。我们将讨论新字段、新分区、数据类型变化和缺失数据等不同场景，并有效处理这些变化的技术。此外，我们还会研究Glue目录管理和模式演变，重点是如何尽量减少对下游系统的干扰。 我们还将考察不同的数据格式，如CSV、Parquet、Avro和ORC，分析各自的优缺点。随后，我们将深入Glue ETL，这是一个基于Apache Spark的数据转换解决方案。课程中充满实践练习和项目经验。 您将分析一份内容易于理解、有实用价值且包含多种数据类型与数据质量问题的大学排名数据集。您将学习如何使用Athena查询数据，通过SQL解决数据质量问题，并使用Glue - Apache Spark ETL清洗数据。此外，课程还涵盖了使用视图简化查询和使用Amazon QuickSight进行数据可视化的技术。 为了展示Athena的可扩展性，我们将查询包含超过1.3亿条评论的大型Amazon客户评论数据集。最后，我们将构建一个无服务器应用程序，利用Kinesis Firehose、Lambda、Comprehend AI、Glue、Athena和S3，处理无限数量的客户评论，进行情感分析，并将结果存储在数据湖中以便进行查询。 期待与您见面！谢谢！Chandra Lingam Compute With Cloud Inc
Hello, my name is Chandra Lingam, and I will be your instructor for the Data Lake in AWS course.In this course, we will begin by gaining an understanding of the fundamental concepts of a data lake and when it is the appropriate solution as opposed to a data warehouseWe will then delve into the various components that make up a data lake solution, including the ability to query files directly using SQL for rapid ad hoc analysis of datasetsDuring the course, we will cover the topic of handling changes to the structure of the files in the data lake. We will delve into the various scenarios, such as new fields, new partitions, changes in data types, and missing data, and discuss the techniques on how to handle them effectively. We will also delve into Glue Catalog Management and the evolution of schemas, with a focus on minimizing disruption to downstream systemsWe will also look into different data formats, such as CSV, Parquet, Avro, and ORC, and examine their respective strengths and weaknesses. Following that, we will delve into Glue ETL, a robust Apache Spark-based solution for data transformation.This course is filled with hands-on exercises and projects.You will analyze a University Rankings dataset, which is easy to understand, useful, and has a mix of data types with many data quality issues.You will learn to utilize Athena for querying data, tackle data quality problems through SQL, and cleanse the data using Glue - Apache Spark ETL.Additionally, the course covers techniques for simplifying queries using views and visualizing data using Amazon QuickSight.To showcase the scalability of Athena, we will query the large Amazon Customer Reviews dataset containing over 130 million reviews. Finally, we will construct a serverless application using Kinesis Firehose, Lambda, Comprehend AI, Glue, Athena, and S3, which can process an unlimited number of customer reviews, perform sentiment analysis, and store the results in the data lake for querying.I am excited to meet you soon!Thank you!Chandra LingamCompute With Cloud Inc
