|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/mastering-amazon-redshift-and-serverless-for-data-engineers/
课程评论：没有评论
课程名称：精通Amazon Redshift和无服务器架构的数据工程师课程 课程概述：AWS（Amazon Web Services）中的Amazon Redshift是构建数据仓库或数据集市的重要服务，用于为业务用户提供报告和仪表板。在本课程中，您将学习Amazon Redshift的所有重要功能，以构建数据仓库或数据集市。课程涵盖了联邦查询、Redshift Spectrum、与Python的集成、AWS Lambda函数、Redshift与EMR的集成，以及使用AWS Step Functions的端到端数据管道等功能。 课程内容：首先，我们将学习如何通过AWS Web控制台开始使用Amazon Redshift。您将了解如何创建集群、连接集群，并使用基于Web的查询编辑器运行查询。接下来，我们将创建一个数据库及表格，并熟悉在Redshift集群中对数据库表执行CRUD（创建、读取、更新和删除）操作。 一旦在Redshift集群中设置了数据库和表格，我们将学习如何将数据导入表格；其中一种常用方法是通过从S3复制数据到Redshift表中，我们会逐步讲解如何使用COPY命令将数据从S3复制到Redshift表。 Python是构建数据工程或ETL（提取、转换和加载）应用程序的主要编程语言。在掌握了如何将数据从S3导入Redshift表后，我们将学习如何使用Python开发数据工程或ETL应用程序，包括执行CRUD操作和使用Python程序运行COPY命令。 接下来，我们将探讨在创建Redshift表时使用的分配键（Distkeys）和排序键（Sortkeys）等关键概念。此外，您还可以连接到远程数据库（如Postgres），并使用Redshift的联邦查询直接查询远程数据库表，或者通过Redshift Spectrum在Glue或Athena目录上运行查询。 课程还将介绍Amazon Redshift无服务器架构，您将学习如何部署一个数据管道，其中Spark应用程序将在AWS EMR集群上运行，并将处理后的数据加载到Redshift中。 通过本课程的学习，您将掌握使用Amazon Redshift和无服务器架构进行数据工程的全面技能。
AWS or Amazon Redshift is one of the key AWS Services used in building Data Warehouses or Data Marts to serve reports and dashboards for business users. As part of this course, you will end up learning AWS or Amazon Redshift by going through all the important features of AWS or Amazon Redshift to build Data Warehouses or Data Marts. We have covered features such as Federated Queries, Redshift Spectrum, Integration with Python, AWS Lambda Functions, Integration of Redshift with EMR, and End-to-End Pipeline using AWS Step Functions.Here is the detailed outline of the course.First, we will understand how to Get Started with Amazon Redshift using AWS Web Console. We will see how to create a cluster, how to connect to the cluster, and also how to run the queries using a Web-based query editor. We will also go ahead and create a Database and tables in the Redshift Cluster. Once we set up a Database and tables, we will also go through the details related to CRUD Operations against tables in Databases in Redshift Cluster.Once we have the databases and tables in Redshift Cluster, it is time for us to understand how to get data into the tables in Redshift Cluster. One of the common approaches we use to get data into the Redshift cluster is by Copying Data from s3 into Redshift Tables. We will go through the step-by-step process of copying the data into Redshift tables from s3 using the copy command.Python is one of the prominent programming languages to build Data Engineering or ETL Applications. It is extensively used to build ETL Jobs to get data into Database Tables in Redshift Cluster. Once we understand how to get data from s3 to Redshift tables using Copy Command, we will learn how to Develop Python-based Data Engineering or ETL Applications using Redshift Cluster. We will learn how to perform CRUD operations and also how to take run COPY Commands using Python-based programs.Once we understand how to build applications using Redshift Cluster, we will go through some of the key concepts used while creating Redshift Tables with Distkeys and Sortkeys.We can also connect to remote databases such as Postgres and run queries directly on the remote database tables using Redshift Federated Queries and also we can run queries on top of Glue or Athena Catalog using Redshift Spectrum. You will learn how to leverage Redshift Federated Queries and Spectrum to process data in remote Database tables or s3 without copying the data.You will also get an overview of Amazon Redshift Serverless as part of Getting Started with Amazon Redshift Serverless.Once you learn Amazon Redshift Serverless, you will end up deploying a Pipeline where a Spark Application is deployed on AWS EMR Cluster which will load the data processed by Spark into Redshift.
