|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/scalable-programming-with-scala-and-spark/
课程评论：没有评论
课程名称：使用Scala和Spark进行可扩展编程 概述：本课程由一个四人团队教授，包括两位斯坦福大学教育背景的前谷歌工程师和两位前Flipkart的首席分析师。该团队在Java和处理数十亿条数据方面拥有数十年的实际经验。通过Spark和Scala，您将能够高效地进行分析、机器学习和数据科学。课程将介绍Spark的基本概念：如果您是一名分析师或数据科学家，您可能习惯使用多种系统来处理数据，如SQL、Python、R和Java。而Spark提供了一个单一的引擎，使您能够探索和处理大量数据，运行机器学习算法，并能使用同一系统将代码生产化。 Scala：Scala是一种通用编程语言，类似于Java或C++。它的函数式编程特性和REPL环境的可用性，使其特别适合分布式计算框架如Spark。 分析：通过Spark和Scala，您可以在交互式环境中分析和探索数据，快速获得反馈。课程将展示如何利用弹性分布式数据集（RDD）和数据框（DataFrame）轻松操作数据。 机器学习与数据科学：Spark的核心功能和内置库使复杂算法（如推荐系统）变得易于实现，代码行数极少。课程将涵盖多个数据集和算法，包括PageRank、MapReduce和图数据集。 课程内容包括： - Scala编程构造：类、特质、一等函数、闭包、柯里化和案例类 - 使用交替最小二乘法和Audioscrobbler数据集进行音乐推荐 - 使用数据框和Spark SQL处理Twitter数据 - 使用PageRank算法处理Google网络图数据集 - 使用Spark Streaming进行流处理 - 使用Marvel社交网络数据集处理图数据 - Spark的基础和高级特性：弹性分布式数据集、转换（map、filter、flatMap）、操作（reduce、aggregate）、配对RDD、reduceByKey、combineByKey、广播和累加变量 - Spark用于MapReduce - Spark的Java API - Spark SQL、Spark Streaming、MLlib和GraphX 通过本课程，您将能够掌握使用Scala和Spark进行大规模数据处理的技能，适用于分析、机器学习和数据科学。
Taught by a 4 person team including 2 Stanford-educated, ex-Googlers and 2 ex-Flipkart Lead Analysts. This team has decades of practical experience in working with Java and with billions of rows of data. Get your data to fly using Spark and Scala for analytics, machine learning and data science Let's parse that. What's Spark? If you are an analyst or a data scientist, you're used to having multiple systems for working with data. SQL, Python, R, Java, etc. With Spark, you have a single engine where you can explore and play with large amounts of data, run machine learning algorithms and then use the same system to productionize your code. Scala: Scala is a general purpose programming language - like Java or C++. It's functional programming nature and the availability of a REPL environment make it particularly suited for a distributed computing framework like Spark. Analytics: Using Spark and Scala you can analyze and explore your data in an interactive environment with fast feedback. The course will show how to leverage the power of RDDs and Dataframes to manipulate data with ease. Machine Learning and Data Science: Spark's core functionality and built-in libraries make it easy to implement complex algorithms like Recommendations with very few lines of code. We'll cover a variety of datasets and algorithms including PageRank, MapReduce and Graph datasets. What's Covered: Scala Programming Constructs: Classes, Traits, First Class Functions, Closures, Currying, Case Classes Lot's of cool stuff.. Music Recommendations using Alternating Least Squares and the Audioscrobbler datasetDataframes and Spark SQL to work with Twitter dataUsing the PageRank algorithm with Google web graph datasetUsing Spark Streaming for stream processing Working with graph data using the Marvel Social network dataset .. and of course all the Spark basic and advanced features: Resilient Distributed Datasets, Transformations (map, filter, flatMap), Actions (reduce, aggregate) Pair RDDs , reduceByKey, combineByKey Broadcast and Accumulator variables Spark for MapReduce The Java API for Spark Spark SQL, Spark Streaming, MLlib and GraphX
