|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/certification-course-in-azure-data-engineering/
课程评论：没有评论
课程名称：Azure数据工程认证课程 概述： 此课程旨在帮助您提升Azure数据工程技能，加深对可扩展数据解决方案设计的理解，推动您在云数据工程领域的职业发展。课程适合所有数据工程师、IT专业人员、云解决方案架构师和数据分析师，提供了设计和管理云端数据解决方案的机会。 课程内容： - 学习Azure数据工程所需的基本技能和概念，包括SQL、数据仓库、ETL/ELT流程以及云数据集成。 - 使用Azure Data Factory（ADF）、Databricks、Snowflake、PySpark和Delta Tables构建并优化数据管道，以确保高效的数据处理和转化。 - 探索实际应用Azure服务，包括数据湖存储、实时分析、数据监控及企业数据管理的安全最佳实践。 课程框架： 该课程通过视频讲座、案例研究、项目、可下载资源和互动练习等多种形式，深入探讨Azure数据工程的内容。课程包括多个案例研究、模板、作业、阅读材料、测验、自我评估以及动手实验，以加深对Azure数据工程概念和实际应用的理解。 课程分为多个模块： 1. SQL基础与高级概念：学习SQL的基础知识、查询结构、性能优化等。 2. 数据仓库概念：掌握OLTP与OLAP的区别、数据管道设计及创建样本架构。 3. Azure数据工程基础：介绍Azure平台及其关键服务，数据存储与集成。 4. Azure服务：了解Azure Functions、逻辑应用、Azure Event Hub等。 5. Azure Data Factory：学习ADF的架构、建立ETL管道及与其他服务的集成。 6. Databricks与PySpark：熟悉Distributed computing和PySpark应用。 7. Delta Tables与版本控制：探讨Delta Lake基础和实施变更数据捕获。 8. Snowflake核心概念：学习Snowflake的架构、数据加载与查询优化。 9. 生产管道与部署：设计可扩展管道，处理异常，使用Azure DevOps进行CI/CD。 10. 结课项目：设计并实施一个完整的Azure数据工程解决方案，使用Azure服务、Databricks、Snowflake和PySpark。 投资于学习Azure数据工程，掌握设计和管理可扩展、高性能数据解决方案的技能，帮助企业实现成功。
DescriptionTake the next step in your career! Whether you're an aspiring data engineer, an experienced IT professional, a cloud solutions architect, or a data analyst, this course is your opportunity to sharpen your Azure Data Engineering skills, enhance your ability to design scalable data solutions, and advance your professional growth in the field of cloud-based data engineering.With this course as your guide, you learn how to:Master the fundamental skills and concepts required for Azure Data Engineering, including SQL, Data Warehousing, ETL/ELT processes, and cloud-based data integration.Build and optimize data pipelines using Azure Data Factory (ADF), Databricks, Snowflake, PySpark, and Delta Tables, ensuring efficient data processing and transformation.Access industry-standard templates and best practices for data architecture, schema design, and performance optimization in cloud environments.Explore real-world applications of Azure services, including data lake storage, real-time analytics, data monitoring, and security best practices for enterprise-level data management.Invest in learning Azure Data Engineering today and gain the skills to design and manage scalable, high-performance data solutions that drive business success.The Frameworks of the CourseEngaging video lectures, case studies, projects, downloadable resources, and interactive exercises-this course is designed to explore Azure Data Engineering, covering SQL, Data Warehousing, ETL/ELT processes, and cloud-based data solutions using Azure services.The course includes multiple case studies, resources such as templates, worksheets, reading materials, quizzes, self-assessments, and hands-on labs to deepen your understanding of Azure Data Engineering concepts and real-world applications.In the first part of the course, you'll learn SQL basics and advanced techniques, data warehousing fundamentals, and data ingestion and transformation using Azure Data Factory (ADF) and Synapse Analytics.In the middle part of the course, you'll develop a deep understanding of Databricks and PySpark, Delta Tables, versioning, and real-time data streaming using Azure Event Hub and Stream Analytics.In the final part of the course, you'll gain expertise in Snowflake for Data Engineering, designing production pipelines, CI/CD implementation with Azure DevOps, and monitoring data workflows. Part 1Introduction and Study Plan· Introduction and know your instructor· Study Plan and Structure of the CourseModule 1. SQL Basics and Advanced Concepts1.1. Introduction to SQL1.1.1. Basics of relational databases and SQL.1.1.2. SQL syntax and query structure.1.1.3. SELECT, WHERE, GROUP BY, and ORDER BY clauses1.2. Advanced SQL techniques1.2.1. Joins (INNER, OUTER, LEFT, RIGHT).1.2.2. Subqueries, CTEs, and Window Functions.1.2.3. Aggregations and analytical functions.1.3. SQL for Data Engineering1.3.1. Data manipulation and transformation.1.3.2. Handling large datasets and performance tuning.1.3.3. Data ingestion and validation using SQL.Module 2. Data Warehousing Concepts2.1. Introduction to Data Warehousing2.1.1. OLTP vs. OLAP.2.1.2. Star and Snowflake schema designs.2.1.3. Dimensional modeling concepts.2.2. Data Pipeline Design2.2.1. ETL vs. ELT processes.2.2.2. Data staging, integration, and transformation layers.2.3. Hands-On Activity2.3.1. Creating sample schemas and loading sample data.Module 3. Azure Data Engineering Fundamentals3.1. Overview of Azure Data Engineering3.1.1. Introduction to Azure cloud platform.3.1.2. Key Azure services for Data Engineering.3.2. Azure Storage Solutions3.2.1. Azure Data Lake Storage.3.2.2. Blob storage and file management.3.2.3. Security and access control mechanisms.3.3. Azure Data Integration3.3.1. Introduction to Azure Synapse Analytics.3.3.2. Data movement and integration tools in Azure.Module 4. Azure Services for Data Engineering4.1. Azure Functions and Logic Apps4.1.1. Automating workflows using Logic Apps.4.1.2. Serverless computing with Azure Functions.4.2. Azure Event Hub and Stream Analytics4.2.1. Streaming data ingestion.4.2.2. Real-time analytics in Azure.4.3. Monitoring and Optimization4.3.1. Cost optimization techniques.4.3.2. Monitoring and debugging Azure workloadsModule 5. Azure Data Factory (ADF)5.1. Introduction to Azure Data Factory5.1.1. ADF architecture and components.5.1.2. Pipelines, triggers, and datasets.5.2. Building ETL Pipelines in ADF5.2.1. Creating and managing data pipelines.5.2.2. Data transformations using ADF.5.3. Integration with Other Services5.3.1. Integrating ADF with Databricks, SQL server, and Snowflake.5.4. Hands-On Activity5.4.1. Building a sample ETL pipeline in ADF.Module 6. Databricks and PySpark6.1. Introduction to Databricks6.1.1. Overview of Databricks and its architecture.6.1.2. Setting up Databricks workspaces.6.2. Introduction to PySpark6.2.1. Basics of distributed computing.6.2.2. Dataframes, RDDs, and Spark SQL.6.3. Advanced PySpark Techniques6.3.1. Writing and optimizing PySpark jobs.6.3.2. Working with large datasets.6.4. Hands-On Activities6.4.1. Building PySpark applications.6.4.2. Integrating Databricks with Azure services.Module 7. Delta Tables and Versioning7.1. Delta Lake Fundamentals7.1.1. Overview of Delta tables.7.1.2. ACID transactions and schema enforcement.7.2. Versioning and Time Travel7.2.1. Querying data at specific points in time.7.2.2. Implementing CDC (Change Data Capture) workflows.Module 8. Snowflake Core Concepts8.1. Introduction to Snowflake8.1.1. Architecture and key features of Snowflake.8.1.2. Warehouses, databases, and schema in Snowflake.8.2. Data Loading and Querying in Snowflake8.2.1. Copying data into Snowflake.8.2.2. Writing and optimizing queries.8.3. Snowflake for Data Engineering8.3.1. Integration with Azure services.8.3.2. Best practices for using Snowflake in production.Module 9. Production Pipelines and Deployment9.1. Designing Production Pipelines9.1.1. Best practices for scalable pipelines.9.1.2. Handling exceptions and retries.9.2. CI/CD for Azure Data Engineering9.2.1. Using Azure DevOps for pipeline deployment.9.2.2. Version control and automated testing.9.3. Monitoring and Maintenance9.3.1. Monitoring data pipelines in production.9.3.2. Troubleshooting and performance tuning.Part 2Module 10. Capstone Project10.1. Project Design and Implementation10.1.1. Design a complete Data Engineering solution.10.1.2. Use Azure services, Databricks, Snowflake, and PySpark.
