|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/data-engineering/
课程评论：没有评论
课程名称：Prophecy数据转换助手数据工程课程 概述：本课程旨在帮助数据工程师和分析师使用Prophecy的数据转换助手构建和部署云数据湖屋架构。该课程的目的是帮助您开启Spark和Prophecy的数据工程之旅。我们将从应用平台（如Salesforce）、具有CDC事务数据的操作数据库，以及机器生成的数据（如日志和指标）中对数据进行预处理。我们将清理和标准化采集的数据表，以准备一个完整、干净且高效的数据模型。基于该数据模型，我们将构建四个项目，为不同的现实应用场景创建消费应用。在每个项目中，您都将学到新的知识： 1. **财务部门的电子表格导出**：我们将探索数据建模和转换的概念，并学习如何设置单元测试和集成测试，以确保数据质量。 2. **运营支持团队的警报系统**：确保客户成功，我们将学习最佳的编排实践。 3. **销售数据的上传**：可再次导入Salesforce，探索高级可扩展性概念，帮助创建和遵循标准化的实践。 4. **为产品团队创建的Databricks实时使用监控仪表板**：我们将学习可观察性和数据质量的相关知识。 最重要的是，我们构建的所有代码均为开源且可访问。您将能够在实际项目中应用所学的所有内容。我们的团队由来自Salesforce、Databricks和Instagram等公司的顶尖数据工程师和架构师组成，将逐步引导您构建这些应用场景。
This course is designed to help data engineers and analysts to build and deploy a cloud data lakehouse architecture using Prophecy's Data Transformation Copilot. It is created with the intention of helping you embark on your data engineering journey with Spark and Prophecy. (This course is developed by Prophecy Data.)We will start by staging the ingested data from application platforms like Salesforce, operational databases with CDC transactional data, and machine generated data like logs and metrics. We're going to clean and normalize the ingested tables to prepare a complete, clean, and efficient data model. From that data model, we're going to build four projects creating consumption applications for different real-world use-cases. With each of the projects, you're going to learn something new:We will build a spreadsheet export for your finance department, where we will explore data modeling and transformation concepts. Since the finance department really cares about the quality of data, we're going to also learn about how to setup unit and integration tests to maintain high quality.We will create an alerting system for your operational support team to ensure customer success, where we're going to learn about orchestration best practices.Sales data upload that can be ingested back to Salesforce, where we will explore advanced extensibility concepts that will allows us to create and follow standardized practices.A dashboard directly on Databricks for your product team to monitor live usage. Here we we learn the a lot about observability and data quality.The best part? All of the code that will be building is completely open-source and accessible. You will be able to apply everything your learn here in your real projects.Our entire team of best in-class data engineers and architects with tons of experience from companies like Salesforce, Databricks, and Instagram are going to walk you through, step by step, building out these use-cases.
