|
所在平台: Coursera |
课程主页: https://www.coursera.org/learn/cloud-applications-part2
课程评论：没有评论
课程名称：云计算应用第2部分：大数据与云中的应用 概述：欢迎参加《云计算应用》课程，这是一个两部分课程系列的第二部分，旨在让您全面了解云计算和大数据的世界！在这门课程中，我们将继续探讨云计算如何促进对大规模数据的分析，这些数据可能是静态的或以高速度流动，并且具有巨大的信息多样性。云应用和数据分析在社会获取和利用信息的方式上带来了颠覆性的变化。 在第一周，我们将介绍一些主要的数据分析系统，包括Spark，以及分析应用程序的主要框架和分发版本，如Hortonworks、Cloudera和MapR。我们将介绍HDFS分布式和稳健的文件系统，这一系统在许多应用中得到了广泛使用，如Hadoop，并以强大的MapReduce编程模型收尾。同时，我们也讲述了像YARN和Mesos这样的分布式操作系统如何支持大数据分析的灵活性和可扩展性。 第二周，我们将介绍大规模数据存储及其面临的共识问题，包括使用大量处理器、内存和磁盘的存储。我们将讨论最终一致性、ACID和BASE，以及数据中心中使用的共识算法，如Paxos和Zookeeper。我们还将介绍分布式键值存储和用于性能优化的内存数据库，如Redis。接着，我们将探讨NOSQL数据库和支持Hadoop操作的可扩展低延迟数据库HBase。 第三周，我们将转向实时数据流，并介绍广泛应用于Yahoo等行业的Storm技术。同时，我们也将学习Spark Streaming、Lambda和Kappa架构，并展示流媒体生态系统。 第四周的重点是图处理、机器学习和深度学习。我们将介绍图处理的概念，并展示Pregel、Giraph和Spark GraphX。接着，我们将举例说明机器学习中的K均值、朴素贝叶斯和频繁模式挖掘等算法，并介绍Spark ML和Mllib。最后，我们将讨论深度学习技术，包括Theano、TensorFlow、CNTK、MXnet和Caffe。 课程大纲： - 课程导入：熟悉课程内容、同学以及学习环境，并掌握所需的技术技能。 - 模块1：Spark、Hortonworks、HDFS、CAP：介绍大数据应用、Apache Spark及HDFS文件系统。 - 模块2：大规模数据存储：讨论分布式存储的挑战、内存键值存储系统和NoSQL数据库。 - 模块3：流处理系统：深入探讨实时流处理技术，如Apache Storm和Spark Streaming。 - 模块4：图处理与机器学习：讨论大数据的应用，包括图处理和机器学习的基本概念，及深度学习的介绍。
Name:Course Orientation
Description:You will become familiar with the course, your classmates, and our learning environment. The orientation will also help you obtain the technical skills required for the course.
Name:Module 1: Spark, Hortonworks, HDFS, CAP
Description:In Module 1, we introduce you to the world of Big Data applications. We start by introducing you to Apache Spark, a common framework used for many different tasks throughout the course. We then introduce some Big Data distro packages, the HDFS file system, and finally the idea of batch-based Big Data processing using the MapReduce programming paradigm.
Name:Module 2: Large Scale Data Storage
Description:In this module, you will learn about large scale data storage technologies and frameworks. We start by exploring the challenges of storing large data in distributed systems. We then discuss in-memory key/value storage systems, NoSQL distributed databases, and distributed publish/subscribe queues.
Name:Module 3: Streaming Systems
Description:This module introduces you to real-time streaming systems, also known as Fast Data. We talk about Apache Storm in length, Apache Spark Streaming, and Lambda and Kappa architectures. Finally, we contrast all these technologies as a streaming ecosystem.
Name:Module 4: Graph Processing and Machine Learning
Description:In this module, we discuss the applications of Big Data. In particular, we focus on two topics: graph processing, where massive graphs (such as the web graph) are processed for information, and machine learning, where massive amounts of data are used to train models such as clustering algorithms and frequent pattern mining. We also introduce you to deep learning, where large data sets are used to train neural networks with effective results.
Welcome to the Cloud Computing Applications course, the second part of a two-course series designed to give you a comprehensive view on the world of Cloud Computing and Big Data! In this second course we continue Cloud Computing Applications by exploring how the Cloud opens up data analytics of huge volumes of data that are static or streamed at high velocity and represent an enormous variety of information. Cloud applications and data analytics represent a disruptive change in the ways that society is informed by, and uses information. We start the first week by introducing some major systems for data analysis including Spark and the major frameworks and distributions of analytics applications including Hortonworks, Cloudera, and MapR. By the middle of week one we introduce the HDFS distributed and robust file system that is used in many applications like Hadoop and finish week one by exploring the powerful MapReduce programming model and how distributed operating systems like YARN and Mesos support a flexible and scalable environment for Big Data analytics. In week two, our course introduces large scale data storage and the difficulties and problems of consensus in enormous stores that use quantities of processors, memories and disks. We discuss eventual consistency, ACID, and BASE and the consensus algorithms used in data centers including Paxos and Zookeeper. Our course presents Distributed Key-Value Stores and in memory databases like Redis used in data centers for performance. Next we present NOSQL Databases. We visit HBase, the scalable, low latency database that supports database operations in applications that use Hadoop. Then again we show how Spark SQL can program SQL queries on huge data. We finish up week two with a presentation on Distributed Publish/Subscribe systems using Kafka, a distributed log messaging system that is finding wide use in connecting Big Data and streaming applications together to form complex systems. Week three moves to fast data real-time streaming and introduces Storm technology that is used widely in industries such as Yahoo. We continue with Spark Streaming, Lambda and Kappa architectures, and a presentation of the Streaming Ecosystem. Week four focuses on Graph Processing, Machine Learning, and Deep Learning. We introduce the ideas of graph processing and present Pregel, Giraph, and Spark GraphX. Then we move to machine learning with examples from Mahout and Spark. Kmeans, Naive Bayes, and fpm are given as examples. Spark ML and Mllib continue the theme of programmability and application construction. The last topic we cover in week four introduces Deep Learning technologies including Theano, Tensor Flow, CNTK, MXnet, and Caffe on Spark.
