|
所在平台: Coursera |
课程主页: https://www.coursera.org/learn/data-manipulation
课程评论：没有评论
课程名称：大规模数据处理：系统与算法 课程概述： 在证据基础的决策制定中，数据分析已经取代数据获取成为瓶颈。我们正被海量的异构和嘈杂的数据淹没。从大型数据集中提取知识，不仅需要强大的计算资源，更需要有效利用这些资源的编程抽象。过去十年中出现的抽象结合了并行数据库、分布式系统和编程语言的概念，创造出了一种新的可扩展数据分析平台，这些平台为实际规模下的数据科学奠定了基础。 在本课程中，您将学习相关系统的全景、它们所依赖的原则、权衡以及如何根据要求评估它们的实用性。您还将了解如何从计算机科学前沿研究中推导出实用系统，并了解未来将出现的系统。课程内容将涵盖云计算、SQL和NoSQL数据库、MapReduce及其衍生的生态系统、Spark及其 contemporaries，并且研究针对图和数组的专用系统。 您还将了解数据科学的历史和背景、所涉及的技能、挑战和方法论，以及如何构建一个数据科学项目。完成本课程后，您将能够： 学习目标： 1. 描述与数据科学项目相关的常见模式、挑战和方法，以及它们与相关领域项目的不同之处。 2. 确定并使用与可扩展数据处理相关的编程模型，包括关系代数、MapReduce和其他数据流模型。 3. 使用适用于大规模分析的数据库技术，包括驱动并行数据库、并行查询处理和库内分析的概念。 4. 评估键值存储和NoSQL系统，描述它们与可比较系统的权衡，以及这一领域中重要示例的细节和未来趋势。 5. 在MapReduce的思维框架下有效编写包括Hadoop和Spark在内的系统算法。您将理解它们的局限性、设计细节、与数据库的关系，以及相关算法、扩展和语言的生态系统。 6. 描述用于图、数组和流的专门大数据系统的全景。 课程大纲： 第一部分：数据科学背景与概念 说明：了解与数据科学相关的术语和反复出现的原则，以及数据科学项目的结构和新兴方法论。探讨这个新兴领域的存在原因、与其他领域的关系，以及本课程的独特之处。了解数据科学项目的样貌以及它们应如何被处理，并提供一些数据科学项目的示例。 第二部分：关系数据库与关系代数 说明：关系数据库是大规模数据管理的主力。尽管最初受到企业运营中问题的驱动，但它们在分析方面同样表现出色。更重要的是，关系数据库背后的原则在大规模数据的管理、操作和分析中是普遍适用的。即使在过去十年中，大规模数据系统的格局发生了巨大的变化，关系模型和语言仍然是一种统一的概念。在处理大规模数据时，没有比这更重要的编程模型了。 第三部分：MapReduce与并行数据流编程 说明：MapReduce编程模型（与其实现不同）被提出作为一种用于并行处理海量数据集的简化抽象，仍然是使用和评估现代大数据平台时必须了解的重要概念。 第四部分：图分析 说明：由于图结构数据在建模实体之间的沟通（如社交网络、互联网通信、城市与国家的交通网络，或财经交易中的公司）中越来越普遍，因此在数据科学环境中也越来越常见。学习从图数据中提取信息的常见算法及其扩展方法。
Part: 1
Title:Data Science Context and Concepts
Description:Understand the terminology and recurring principles associated with data science, and understand the structure of data science projects and emerging methodologies to approach them. Why does this emerging field exist? How does it relate to other fields? How does this course distinguish itself? What do data science projects look like, and how should they be approached? What are some examples of data science projects?
Part: 2
Title:Relational Databases and the Relational Algebra
Description:Relational Databases are the workhouse of large-scale data management. Although originally motivated by problems in enterprise operations, they have proven remarkably capable for analytics as well. But most importantly, the principles underlying relational databases are universal in managing, manipulating, and analyzing data at scale. Even as the landscape of large-scale data systems has expanded dramatically in the last decade, relational models and languages have remained a unifying concept. For working with large-scale data, there is no more important programming model to learn.
Part: 3
Title:MapReduce and Parallel Dataflow Programming
Description:The MapReduce programming model (as distinct from its implementations) was proposed as a simplifying abstraction for parallel manipulation of massive datasets, and remains an important concept to know when using and evaluating modern big data platforms.
Part: 4
Title:Graph Analytics
Description:Graph-structured data are increasingly common in data science contexts due to their ubiquity in modeling the communication between entities: people (social networks), computers (Internet communication), cities and countries (transportation networks), or corporations (financial transactions). Learn the common algorithms for extracting information from graph data and how to scale them up.
Data analysis has replaced data acquisition as the bottleneck to evidence-based decision making --- we are drowning in it. Extracting knowledge from large, heterogeneous, and noisy datasets requires not only powerful computing resources, but the programming abstractions to use them effectively. The abstractions that emerged in the last decade blend ideas from parallel databases, distributed systems, and programming languages to create a new class of scalable data analytics platforms that form the foundation for data science at realistic scales. In this course, you will learn the landscape of relevant systems, the principles on which they rely, their tradeoffs, and how to evaluate their utility against your requirements. You will learn how practical systems were derived from the frontier of research in computer science and what systems are coming on the horizon. Cloud computing, SQL and NoSQL databases, MapReduce and the ecosystem it spawned, Spark and its contemporaries, and specialized systems for graphs and arrays will be covered. You will also learn the history and context of data science, the skills, challenges, and methodologies the term implies, and how to structure a data science project. At the end of this course, you will be able to: Learning Goals: 1. Describe common patterns, challenges, and approaches associated with data science projects, and what makes them different from projects in related fields. 2. Identify and use the programming models associated with scalable data manipulation, including relational algebra, mapreduce, and other data flow models. 3. Use database technology adapted for large-scale analytics, including the concepts driving parallel databases, parallel query processing, and in-database analytics 4. Evaluate key-value stores and NoSQL systems, describe their tradeoffs with comparable systems, the details of important examples in the space, and future trends. 5. “Think” in MapReduce to effectively write algorithms for systems including Hadoop and Spark. You will understand their limitations, design details, their relationship to databases, and their associated ecosystem of algorithms, extensions, and languages. write programs in Spark 6. Describe the landscape of specialized Big Data systems for graphs, arrays, and streams
