|
所在平台: Coursera |
课程主页: https://www.coursera.org/learn/machine-learning-big-data-apache-spark
课程评论：没有评论
课程名称：使用Apache Spark的可扩展机器学习与大数据 课程概述：本课程旨在帮助您掌握在大数据环境下使用Apache Spark进行数据科学和机器学习（ML）任务的技能。许多实际的机器学习工作涉及的数据集非常庞大，超出了单台计算机的CPU、内存和存储限制。Apache Spark是一个开源框架，利用集群计算和分布式存储实现高效且经济地处理极大数据集。因此，掌握Apache Spark的应用知识对于机器学习工程师来说是一项非常宝贵的资产。 完成本课程后，您将能够： - 实际理解Apache Spark，并应用于解决小型和大型数据的机器学习问题。 - 理解如何编写能够在数千个CPU上运行的并行代码。 - 使用大规模计算集群，通过Apache SparkML Pipelines对PB级数据应用机器学习算法。 - 消除传统机器学习框架在数据不适合计算机主内存时生成的内存不足错误。 - 并行测试数千种不同的机器学习模型，以找到表现最佳的模型，这一技术被许多成功的Kaggler广泛使用。 - （可选）使用Apache SparkSQL和Apache Spark DataFrame API在非常大的数据集上运行SQL语句。 加入我们，学习在阿里巴巴、苹果、亚马逊、百度、eBay、IBM、NASA、三星、SAP、TripAdvisor、Yahoo!、Zalando等公司成功应用的机器学习技术。 注：在课程期间，您将实践在IBM提供的Apache Spark集群上运行机器学习任务的操作，课程结束后也可继续使用该集群。 先决条件： - 基础的Python编程知识 - 基础的机器学习（课程中提供可选的介绍视频） - 基础的SQL技能（针对可选内容） 建议在参加本课程之前学习以下课程（除非您已经具备相应技能）： - https://www.coursera.org/learn/python-for-applied-data-science 或类似课程 - https://www.coursera.org/learn/machine-learning-with-python 或类似课程 - https://www.coursera.org/learn/sql-data-science 供可选讲座使用 课程大纲： 第1周：简介 - 了解Apache Spark的内部工作原理及其数据处理方法，介绍低级API RDD及并行编程/函数式编程，并对比不同的数据存储解决方案，讲解Apache Spark SQL及其优化器Tungsten和Catalyst。 第2周：在Apache Spark上扩展统计的数学 - 使用Apache Spark RDD API进行基本统计计算，以体验Apache Spark中的并行化工作原理。 第3周：Apache SparkML简介 - 理解机器学习管道的概念，以便理解Apache SparkML的编程原理。 第4周：使用SparkML进行有监督和无监督学习 - 应用有监督和无监督的机器学习任务，学习如何使用SparkML。
Part: 1
Title:Week 1: Introduction
Description:This is an introduction to Apache Spark. You'll learn how Apache Spark internally works and how to use it for data processing. RDD, the low level API is introduced in conjunction with parallel programming / functional programming. Then, different types of data storage solutions are contrasted. Finally, Apache Spark SQL and the optimizer Tungsten and Catalyst are explained.
Part: 2
Title:Week 2: Scaling Math for Statistics on Apache Spark
Description:Applying basic statistical calculations using the Apache Spark RDD API in order to experience how parallelization in Apache Spark works
Part: 3
Title:Week 3: Introduction to Apache SparkML
Description:Understand the concept of machine learning pipelines in order to understand how Apache SparkML works programmatically
Part: 4
Title:Week 4: Supervised and Unsupervised learning with SparkML
Description:Apply Supervised and Unsupervised Machine Learning tasks using SparkML
This course will empower you with the skills to scale data science and machine learning (ML) tasks on Big Data sets using Apache Spark. Most real world machine learning work involves very large data sets that go beyond the CPU, memory and storage limitations of a single computer. Apache Spark is an open source framework that leverages cluster computing and distributed storage to process extremely large data sets in an efficient and cost effective manner. Therefore an applied knowledge of working with Apache Spark is a great asset and potential differentiator for a Machine Learning engineer. After completing this course, you will be able to: - gain a practical understanding of Apache Spark, and apply it to solve machine learning problems involving both small and big data - understand how parallel code is written, capable of running on thousands of CPUs. - make use of large scale compute clusters to apply machine learning algorithms on Petabytes of data using Apache SparkML Pipelines. - eliminate out-of-memory errors generated by traditional machine learning frameworks when data doesn’t fit in a computer's main memory - test thousands of different ML models in parallel to find the best performing one – a technique used by many successful Kagglers - (Optional) run SQL statements on very large data sets using Apache SparkSQL and the Apache Spark DataFrame API. Enrol now to learn the machine learning techniques for working with Big Data that have been successfully applied by companies like Alibaba, Apple, Amazon, Baidu, eBay, IBM, NASA, Samsung, SAP, TripAdvisor, Yahoo!, Zalando and many others. NOTE: You will practice running machine learning tasks hands-on on an Apache Spark cluster provided by IBM at no charge during the course which you can continue to use afterwards. Prerequisites: - basic python programming - basic machine learning (optional introduction videos are provided in this course as well) - basic SQL skills for optional content The following courses are recommended before taking this class (unless you already have the skills) https://www.coursera.org/learn/python-for-applied-data-science or similar https://www.coursera.org/learn/machine-learning-with-python or similar https://www.coursera.org/learn/sql-data-science for optional lectures
