|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/transform-data-into-insights-with-dagster-and-deepnote/
课程评论：没有评论
课程名称：使用Dagster和Deepnote将数据转化为洞察 课程概述：如果你的企业因为数据分散、不一致和无法获取而难以做出数据驱动的决策，那么本课程就是你的解决方案！通过本课程，你将学习如何构建一个高效的ETL（提取、转换、加载）管道，帮助你将数据转化为可行动的洞察力。课程将教授你如何从多个来源收集数据、标准化数据并将其存储为一致且易于访问的格式。你将学习如何提取数据、探索和预处理数据，最终通过可视化来支持更好的决策和优化业务流程。 课程的第一个部分将介绍架构，并向你介绍将要使用的工具：Deepnote作为无设置的开发环境，Dagster作为管道调度器，Metabase作为低代码数据可视化平台。尽管课程会介绍Deepnote和Metabase的相关功能，但重点主要放在Dagster上。 接下来的部分将开始使用Deepnote生成假设公司的虚拟销售数据，提供用于此的代码。在我们获得数据后，课程将深入探讨使用Python和SQL在Deepnote中进行数据探索和预处理的技术，包括从关系数据库、JSON数据、Excel表格等各种来源中清洗和标准化数据。我们将在Deepnote中实施处理逻辑，然后提交到一个将与Dagster共享的Git仓库。 在随后的部分中，将使用Dagster操作和作业封装业务逻辑，并将其部署到Dagster Cloud（也提供自托管选项），这将允许你从一个统一视图管理一切。在这个部分，你还将学习一些加速和优化处理的技巧，如并行处理或流式处理。 在本课程的最后部分，你将通过Metabase为你的预处理数据赋予生命。即使是非技术人员，也能通过简单的点击创建惊人的、强大的可视化，充分挖掘数据的潜力。 通过本课程的学习，你将全面理解所用工具及其如何协同工作，能够在第一天就为你的客户或公司带来切实的收益，收益可以以数千或数万美元来衡量。机会就在眼前，你会抓住这个机会为你的公司或客户带来巨大的利益，并获得应有的回报吗？
Do you struggle with making data-driven decisions for your business due to scattered, inconsistent, and inaccessible data? This course is the solution! Learn to build a streamlined and efficient ETL pipeline that will allow you to turn data into actionable insights.This course teaches you how to build a system that collects data from multiple sources, normalizes it, and stores it in a consistent and accessible format. You will learn how to extract data, explore and preprocess it, and ultimately visualize it to support better decision-making and optimize business processes.Forget about big data and cluster management headaches, this course is designed to get you up and running quickly with a real-time ETL pipeline. With infrastructure costs under $50 a month, you can start seeing immediate results and return on investment for your clients or company.In the first part of the course, I will walk you through the architecture and introduce you to the tools we will be using:Deepnote, as a setup-free development environmentDagster, as the pipeline orchestratorMetabase, as a low-code data visualization platformWhile the course will introduce you to the relevant features of Deepnote and Metabase, it is mostly focused on Dagster.In the next part, we will get started by generating dummy sales data of a hypothetical company using Deepnote. The code will be provided for this. Once we have the data, the course will dive into data exploration and preprocessing techniques using Python and SQL in Deepnote, including cleaning and normalizing data from various sources such as relational and JSON data, Excel sheets, and more. We will implement the processing logic in Deepnote, then commit it to a Git repository that will be shared with Dagster.In the following section, we will wrap the business logic with Dagster operations and jobs, then deploy them to Dagster Cloud (self-hosted option also available), which will allow you to manage everything from a single, unified view. In this section, you will also learn a few tricks to speed up and optimize processing, such as parallelization or streamed processing.In the final section of this course, you'll bring your preprocessed data to life with Metabase. With a few simple clicks, even non-technical individuals will be able to create stunning, powerful visualizations that unlock the full potential of your data.By the end of this course, you'll have a comprehensive understanding of the tools used and how they work together, empowering you to provide tangible benefits to your clients or company from day one, measured in thousands or tens of thousands of dollars.The choice is yours - will you seize this opportunity to deliver massive benefits to your company or clients, and claim your fair share of the rewards?
