|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/data-pipelines-with-snowflake-and-streamlit/
课程评论：没有评论
课程名称：使用 Snowflake 和 Streamlit 构建数据管道 课程概述：本课程专注于构建一个数据工程管道，整合多个数据源，包括 Kaggle 数据集和通过 SerpAPI 获取的 Google Trends 数据，以分析 Netflix 节目发布与演员受欢迎程度之间的关系。您将学习如何收集和组合关于 Netflix 演员及其在 Google 上的趋势的数据，尤其是在节目发布后的几周内。课程将利用 Kaggle 作为 Netflix 节目和演员数据集的来源，通过 SerpAPI 获取实时搜索数据。这些数据将存储和处理在 Snowflake 数据库中，利用其云原生架构实现最佳的可扩展性和性能。 技术栈概述： - **Snowflake 数据库**：存储和查询数据的中央存储库。 - **Streamlit in Snowflake**：直接在 Snowflake 中可视化数据的网页应用框架。 - **AWS S3**：用于数据存储和检索，特别是中间数据集。 - **Snowflake Python 程序**：自动化数据操作和管道流程。 - **Snowflake 外部访问与存储集成**：管理对外部服务和存储的安全访问。 通过本课程，您将能够构建一个功能完整的数据管道，处理和结合流数据、云存储以及用于趋势分析的 API，最终通过 Snowflake 内部的互动 Streamlit 应用进行可视化。
This course focuses on building a data engineering pipeline that integrates multiple data sources, including Kaggle datasets and Google Trends data (fetched via SerpAPI), to analyze the relationship between Netflix show releases and the popularity of actors. You'll learn to gather and combine data on Netflix actors and their trends on Google, particularly in the weeks following a show's release.You will use Kaggle as a source for the Netflix shows and actors dataset and Google Trends (accessed via SerpAPI) to fetch real-time search data for the actors. This data will be stored and processed within the Snowflake database, leveraging its cloud-native architecture for optimal scalability and performance.Technical Stack Overview:Snowflake Database: The central repository for storing and querying data.Streamlit in Snowflake: A web app framework to visualize the data directly inside Snowflake.AWS S3: For data storage and retrieval, particularly for intermediate datasets.Snowflake Python Procedures: Automating data manipulation and pipeline processes.Snowflake External Access & Storage Integrations: Managing secure access to external services and storage.By the end of the course, you'll have a fully functional data pipeline that processes and combines streaming data, cloud storage, and APIs for trend analysis, visualized through an interactive Streamlit app within Snowflake.
