|
所在平台: Coursera |
课程主页: https://www.coursera.org/learn/introduction-to-data-engineering
课程评论：没有评论
课程名称：数据工程入门 课程概述： 本课程介绍了数据工程的核心概念、流程和工具，为您奠定数据工程的基础知识。您将理解现代数据生态系统，以及数据工程师、数据科学家和数据分析师在该生态系统中的角色。数据工程生态系统包括各种不同的组成部分，如不同类型、格式和来源的数据。数据管道从多个源收集数据，将其转换为适合分析的数据，并使数据消费者可用于分析和决策。数据存储库（如关系型和非关系型数据库、数据仓库、数据市集、数据湖和大数据存储）处理和存储这些数据。数据集成平台将不同的数据整合为统一的视图，供数据消费者使用。在本课程中，您将学习到这些组成部分的知识，以及大数据和一些大数据处理工具的使用。 数据工程的生命周期通常包括设计数据平台、构造数据存储、收集、导入、处理、查询和分析数据，同时也包括性能监控和调优，以确保系统在最佳水平上运行。您将学习数据工程生命周期的各个方面，以及安全性、治理和合规性的重要性。 数据工程被公认为是当今发展最快的领域之一，课程中讨论了该领域的职业机会以及进入该领域的不同途径。 课程还包括动手实验，指导您创建IBM Cloud Lite账户、配置数据库实例、将数据加载到数据库实例中，并执行一些基本查询操作，以帮助您理解数据集。 课程大纲： 1. **什么是数据工程？** - 本模块将介绍现代数据生态系统的不同组成部分，以及数据工程师、数据科学家、数据分析师、业务分析师和商业智能分析师在生态系统中的角色。您将学习数据工程的定义和数据工程生命周期的关键任务，也将了解到数据工程师的职责、所需技能以及他们的日常工作情况。 2. **数据工程生态系统** - 本模块将深入探讨数据工程生态系统、不同类型的数据结构、文件格式、数据来源及数据专业人员日常使用的语言。您将了解不同类型的数据存储库，如关系型和非关系型数据库、数据仓库、数据市集和数据湖。学习ETL和ELT过程、数据管道以及数据集成平台的相关知识，并理解什么是大数据及处理和存储大数据的工具。最后，您将创建IBM Cloud账户，并配置IBM Db2实例。 3. **数据工程生命周期** - 本模块将引导您通过数据工程生命周期的各个环节，学习数据平台的架构、选择和设计数据存储的因素，以及数据平台和数据生命周期管理中的安全性各方面。您还将了解收集、导入、处理和查询数据的过程、步骤和工具。学习性能监控及如何排查性能问题，同时讨论治理法规及技术如何支持合规性。在本模块中，您将从CSV文件中加载数据到之前创建的IBM Db2实例，并将使用一些基本SQL查询来探索数据集。 4. **职业机会与数据工程实践** - 本模块将介绍数据工程领域的职业机会以及您可以采取的不同技能提升路径。课程末尾将有一个分数作业，包括两部分：第一部分是一些测验题，第二部分是一些开放式问题，由同伴进行评分。 欢迎参加本课程，开启数据工程的学习之旅！
Name:What is Data Engineering?
Description:In this module, you will learn about the different entities that come together to form a modern data ecosystem and the role Data Engineers, Data Scientists, Data Analysts, Business Analysts, and Business Intelligence Analysts play in this ecosystem. You will learn what data engineering is and the key tasks in a data engineering lifecycle. You will also gain an understanding of the responsibilities of a data engineer, the skillsets they need in order to be successful, and what a typical day in the life of a data engineer looks like.
Name:The Data Engineering Ecosystem
Description:In this module, you will learn about the data engineering ecosystem, the different types of data structures, file formats, sources of data, and the languages data professionals use in their day-to-day tasks. You will gain an understanding of several different types of data repositories such as relational and non-relational databases, data warehouses, data marts, and data lakes. You will learn about ETL and ELT processes, data pipelines, and data integration platforms. You will also gain an understanding of what big data is, and the tools used for processing and storing big data. At the end of this module, you will be guided to create an IBM Cloud account, and provision an instance of IBM Db2.
Name:Data Engineering Lifecycle
Description:In this module, we will walk you through the data engineering lifecycle. You will learn about the architecture of a data platform, factors for selecting and designing data stores, and the different facets of security as it applies to data platforms and data lifecycle management. You will also learn about the process, steps, and tools used for gathering, importing, wrangling, and querying data. You will gain an understanding of performance monitoring and the steps you can take to troubleshoot performance issues. We will also talk about governance regulations, why we need them, and how technology enables compliance to regulations. During the course of this module, you will be guided to load data from a CSV file into the IBM Db2 instance you created in the previous module. You will also be guided to explore your dataset using some basic SQL queries that will be provided to you.
Name:Career Opportunities and Data Engineering in Action
Description:In this module, you will learn about career opportunities in the field of Data Engineering and the different paths that you can take for getting skilled as a Data Engineer. At the end of the module, you will be presented with the final graded assignment which is divided into two parts. The first part of the final assignment includes a couple of quiz questions and the second part includes open-ended questions that will be reviewed and graded by a peer.
This course introduces you to the core concepts, processes, and tools you need to know in order to get a foundational knowledge of data engineering. You will gain an understanding of the modern data ecosystem and the role Data Engineers, Data Scientists, and Data Analysts play in this ecosystem. The Data Engineering Ecosystem includes several different components. It includes disparate data types, formats, and sources of data. Data Pipelines gather data from multiple sources, transform it into analytics-ready data, and make it available to data consumers for analytics and decision-making. Data repositories, such as relational and non-relational databases, data warehouses, data marts, data lakes, and big data stores process and store this data. Data Integration Platforms combine disparate data into a unified view for the data consumers. You will learn about each of these components in this course. You will also learn about Big Data and the use of some of the Big Data processing tools. A typical Data Engineering lifecycle includes architecting data platforms, designing data stores, and gathering, importing, wrangling, querying, and analyzing data. It also includes performance monitoring and finetuning to ensure systems are performing at optimal levels. In this course, you will learn about the data engineering lifecycle. You will also learn about security, governance, and compliance. Data Engineering is recognized as one of the fastest-growing fields today. The career opportunities available in the field and the different paths you can take to enter this field are discussed in the course. The course also includes hands-on labs that guide you to create your IBM Cloud Lite account, provision a database instance, load data into the database instance, and perform some basic querying operations that help you understand your dataset.
