|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/snowpark-data-engineering-with-snowflake/
课程评论：没有评论
课程名称：Snowpark：使用Snowflake进行数据工程 课程概述：Snowpark是什么？Snowpark是Snowflake为开发者提供的一种允许使用自己喜欢的编程语言编写代码的工具。除了Snowflake原有的SQL接口，现在Snowflake还支持以下编程语言：1. Python 2. Scala 3. Java。Snowpark的一些关键特性包括：您的代码将被推送到Snowflake，利用Snowflake仓库的计算能力。您无需将数据导出到不同的环境，而是将代码发送到数据。使用Snowpark，您可以构建复杂的数据管道或数据产品。此外，Snowpark还解决了传统数据处理管道中的一些不足之处，如节点集群的启动时间过长（如Hadoop和Spark系统通常需要5-10分钟来启动集群）、小文件问题、数据在节点间分配时的连接问题和垃圾回收的不确定性。 您将学习什么？您将学习SNOWPARK API的基础知识：基本的读写操作；从Amazon S3读取数据并将其加载到Snowflake表；深入分析SNOWPARK API的工作原理；SNOWPARK的使用规范和禁忌；构建处理数据的数据组件；建立数据管道以处理数据。
What is Snowpark ?With Snowpark, Snowflake allows developers to write code in their preferred language.Along with Snowflake's original SQL interface now snowflake allows you to write code in, 1. Python 2. Scala 3. JavaSome of the key features of Snowpark are,Your code will be pushed to snowflake leveraging the compute power of snowflake warehouses.You will not end up exporting data to different environment but rather your code is shipped to the data.You can build complex data pipelines or data products using SnowparkSnowpark also address below overheads in conventional data pipelines,Long startup time of node clusters: Systems like Hadoop and Spark requires cluster of nodes to process data. Most of the time it takes 5-10 minutes to just start the cluster. In case of Snowpark we will be using snowflake warehouse to process our data.Problem of small files , Problem of using right joins to shuffle data across nodes, Problem of garbage collection. uncertainty when the compute nodes goes down.All the above problems are well handled with Snowpark.What you will learn ?You will learn the basics of SNOWPARK API. > Basic read and write operations. > Read data from s3 and load that to snowflake table. > We will do deep analysis of how SNOWPARK API works. > Do's and Don'ts of SNOWPARK. > Build data components to process data. > Build data pipeline to process data.
