|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/azure-data-factory-essentials-training/
课程评论：没有评论
课程名称: Azure Data Factory 基础培训（动手实践） 课程概述: 本课程将介绍 Azure Data Factory 的基本概念及其在数据批处理中的应用。学生将通过动手实践、测验和项目学习如何利用 Data Factory 整合多种技术，从而构建完整的 ETL 解决方案，包括在 Azure DevOps 中的 CI/CD 管道。课程涵盖了与 DP-203 认证考试相关的 Azure 数据工程内容。 学习方式: 在此课程中，学生将通过动手实践活动、视频和测验学习 Microsoft Azure Data Factory 的相关知识和实际操作。课程结束时，学生将有机会提交一个项目，以加深对 ADF 工作原理、组件及其与 Databricks 整合的理解。 学生收获: - 理解 ADF 如何协调其他技术的特性以转换或分析数据； - 能够解释并使用组成 ADF 的组件； - 能够使用 ADF 整合两种或更多技术； - 能够自信地创建中等复杂的数据驱动管道； - 能够在 Azure DevOps 中开发 CI/CD 管道以部署 Data Factory 管道。 学习内容: - Azure Data Factory 介绍：理解其如何与其他技术整合及其连接器列表； - 从零开始设置 Data Factory：使用 Azure 门户和 PowerShell 进行操作； - ADF 组件与活动的理解，包括管道、数据集、触发器、链接服务等； - 使用 Mapping Data Flows 进行无代码数据转换、摄取和整合； - Azure Data Factory 与 Databricks 的整合，学习如何认证并在 ADF 中运行笔记本； - 使用 Azure DevOps 进行 Data Factory 部署，建立 CI/CD 流程。 课程大纲包括: - 课程模块介绍 - Azure Data Factory 组件深入理解 - 使用复制活动将数据摄取到数据湖存储 Gen2 - 利用 Mapping Data Flow 进行数据转换 - Azure Data Factory 与 Databricks 的集成 - Azure DevOps 中的 CI/CD 流程设置 本课程为希望提升数据工程技能的学习者提供了全面的实践训练及理论指导。
TL;DR.This course will introduce Azure Data Factory and how it can help in the batch processing of data. Students will learn with hands-on activities, quizzes, and a project, how Data Factory can be used to integrate many other technologies together to build a complete ETL solution, including a CI/CD pipeline in Azure DevOps. Some topics related to Data Factory required for the exam DP-203: Data Engineering on Microsoft Azure, are covered in this course. Learn by DoingTogether, you and I are going to learn everything you need to know about using Microsoft Azure Data Factory. This course will prepare you with hands-on learning activities, videos, and quizzes to help you gain knowledge and practical experience as we go along.At the end of this course, students will have the opportunity to submit a project that will help them to understand how ADF works, what are the components, and how to integrate ADF and Databricks.Student key takeaways:The student should understand how ADF orchestrates the features of other technologies to transform or analyze data.The student should be able to explain and use the components that make up ADF.The student should be able to integrate two or more technologies using ADF.The student should be able to confidently create medium complex data-driven pipelinesThe student should be able to develop a CI/CD pipeline in Azure DevOps to deploy Data Factory pipelinesWhat You'll Learn:Introduction to Azure Data Factory. You will understand how it can be used to integrate many other technologies with an ever-growing list of connectors.How to set up a Data Factory from scratch using the Azure Portal and PowerShell.Activities and Components that makeup Data Factory. It will include Pipelines, Datasets, Triggers, Linked Services, and more.How to transform, ingest, and integrate data code-free using Mapping Data Flows.How to integrate Azure Data Factory and Databricks. We'll cover how to authenticate and run a few notebooks from within ADF.Azure Data Factory Deployment using Azure DevOps for continuous integration and continuous deployment (CI/CD)Data Factory Essentials Training - OutlineIntroductionModules introductionGetting StartedUnderstand Azure Data Factory ComponentsIngesting and Transforming Data with Azure Data FactoryIntegrate Azure Data Factory with DatabricksContinuous Integration and Continuous Delivery (CI/CD) for Azure Data FactoryGetting startedSign up for your Azure free accountSetting up a BudgetHow to set up Azure Data FactoryAzure PortalPowerShellAzure Data Factory ComponentsLinked ServicesPipelinesDatasetsData Factory ActivitiesParametersPipeline ParametersActivity ParametersGlobal ParametersTriggersIntegration Runtimes (IR)Azure IRSelf-hosted IRLinked Self-Hosted IRAzure-SSIS IRQuizIngesting and Transforming DataIngesting Data using Copy Activity into Data Lake Store Gen2How to Copy Parquet Files from AWS S3 to Azure SQL DatabaseCreating ADF Linked Service for Azure SQL DatabaseHow to Grant Permissions on Azure SQL DB to Data Factory Managed IdentityIngesting Parquet File from S3 into Azure SQL DatabaseCopy Parquet Files from AWS S3 into Data Lake and Azure SQL Database (intro)Copy Parquet Files from AWS S3 into Data Lake and Azure SQL DatabaseMonitoring ADF Pipeline ExecutionTransforming data with Mapping Data FlowMapping Data Flow Walk-throughIdentify transformations in Mapping Data FlowMultiple Inputs/OutputsSchema ModifierFormattersRow ModifierDestinationAdding source to a Mapping Data FlowDefining Source Type; Dataset vs InlineDefining Source OptionsSpinning Up Data Flow Spark ClusterDefining Data Source Input TypeDefining Data SchemaOptimizing Loads with PartitionsData Preview from Source TransformationHow to add a Sink to a Mapping Data FlowHow to Execute a Mapping Data FlowQuizIntegrate Azure Data Factory with DatabricksProject Walk-throughHow to Create Azure Databricks and Import NotebooksHow to Transfer Data Using Databricks and Data FactoryValidating Data Transfer in Databricks and Data FactoryHow to Use ADF to Orchestrate Data Transformation Using a Databricks NotebookQuizContinuous Integration and Continuous Delivery (CI/CD) for Azure Data FactoryHow to Create an Azure DevOps Organization and ProjectHow to Create a Git Repository in Azure DevOpsHow to Link Data Factory to Azure DevOps RepositoryHow to version Azure Data Factory with BranchesData Factory Release WorkflowMerging Data Factory Code to Collaboration BranchHow to Create a CI/CD pipeline for Data Factory in Azure DevOpsHow to Create a CICD pipeline for Data Factory in Azure DevOpsHow to Execute a Release Pipeline in Azure DevOps for ADFQuiz
