|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/snowflake-cloud-database-with-airflow-python-talend/
课程评论：没有评论
课程名称：Snowflake云数据库与ELT（Airflow+Python+Talend） 课程概述：在我们之前的课程“Snowflake大师班（实时演示+最佳实践+实验室）”中，我们深入了解了Snowflake的基础知识，完成了大量作业，并掌握了数据加载和卸载的最佳实践。同时，我们仔细评估了大多数Snowflake功能，理解它们如何在后台工作。通过这些讨论，您也意识到如何有效利用Snowflake。然而，有一个缺失的部分，那就是如何在Snowflake上构建和调度ETL工作流。本课程正是针对这一点而设计。 在本课程中，我们将学习如何在Airflow中构建工作流。我们将利用Talend的能力，构建通用代码来将数据导入Snowflake并处理数据。我们还将建立审计表，记录在Snowflake上执行的每一个命令，记录每个任务所消耗的时间，并捕获Snowflake授信。完成框架后，我们将构建一个工作流来处理和转化250+GB的纽约市交通数据。最后，我们将使用Python与Snowflake连接，编写代码以捕捉我们加载到Snowflake的数据统计信息。您还将获得预配置的Jupyter Notebook，以便在Snowflake上运行Python代码。 如果您之前没有使用过Talend、Airflow和Python，也不用担心，这些工具都非常简单，我将提供必要的介绍。我相信您会在这个学习过程中收获颇丰。课程中见！
In our previous course Snowflake Masterclass [Real time demos+Best practices+Labs] we deep-dived and understood the fundamentals of snowflake, solved lot of assignments, and understood best practices to load data and unload data.Also, we closely evaluated most of the snowflake features understanding how they work under the hood. Through these discussions, you realized how to use Snowflake efficiently.There was one missing piece, How to build and orchestrate ETL workflows on Snowflake. This course is just about that.In this course, we are going to learn,Build workflows in Airflow.We will leverage Talend capabilities to build generic code to ingest data and process data in snowflake. We will build audit tables and record every command we fire on the snowflake. We will record the time consumed for each task and capture snowflake credits. Once we build the framework we will build a workflow to process and transform 250 + GB volume of NYC traffic data. At last, we will connect the Snowflake with python and write code to capture stats of data we loaded to the snowflake.you will also get access to preconfigured Jupyter notebook to run your python code on the Snowflake.If you have previously not worked with Talend, Airflow and Python don't worry they are very simple tools I will provide the necessary introduction.I am sure you will learn a lot from this journey. See you in the course!!
