|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/big-data-with-mapreduce-hands-on/
课程评论：没有评论
**Coursera 课程总结：MapReduce - 大数据实战** 本课程深入探讨了 Hadoop 的核心组件 MapReduce，重点介绍其在大规模数据处理中的应用和可扩展性。 **核心概念：** * **MapReduce 架构：** 课程详细解释了 MapReduce 的工作原理，即通过 mappers（映射器）和 reducers（归约器）并行处理数据。Mappers 将原始数据分割并进行初步处理，而 Reducers 则汇集 mappers 的结果，进行聚合和最终计算。 * **并行处理：** MapReduce 克服了传统串行数据分析的效率瓶颈，通过并行处理极大地缩短了大数据分析的时间。 * **应用场景：** 课程以 Flipkart 的年销售额统计为例，生动展示了 MapReduce 如何处理海量数据，从粗粒度的数据分割到精确的最终结果生成。 **学习重点：** * **处理海量和非结构化数据：** 学习如何有效地处理各种类型的数据，并将其整合以便分析。 * **数据科学家实践：** 提供数据科学家实际工作流程的实践视角。 * **Hadoop 生态系统工具：** 掌握使用 HIVE、PIG、MapReduce 和 Sqoop 等工具进行数据处理和程序开发。 * **定制化数据分析：** 理解如何利用 Hadoop 的子模块，如 PIG 和 HIVE，来简化复杂的数据分析程序，并根据不同组织的需求生成定制化输出。 * **框架选择：** 学习如何根据具体场景选择最合适的框架。 **课程目标：** 完成本课程后，学员将具备处理海量数据的能力，并能熟练运用 MapReduce 及其相关工具进行高效的数据分析和程序开发。
MapReduce can be defined as the sub-module of Hadoop that offer huge scalability of data spread across numerous of commodity clusters. MapReduce comprises of two things that work consecutively to process the analytics. The process in both the different parts is done in a parallel manner helping save a lot of time while working with significant data. In the traditional data analysis approach, the data was analyzed serially and MapReduce overcomes that problem.As it's named sound, it involves mapping and reducing process which is done by mappers and reducers. The dataset gets divided equally among different mappers and all of the processes or analyses the data in a parallel manner. Once the mapper produces the outcome, reducers come in to generate the outcome. The role of the reducer is to collect the data from all the mappers and then process their outcome to get the final result.For instance, if Flipkart needs to find out the total sell in 2018 in Mumbai. The entire process will flow below.The entire dataset will be divided into months which means the sell data of one year will be divided into 12 months like how much they made each month from which location.The dataset will be then assigned to 12 mappers.Each mapper will find out in which city and how of how much the goods were sold.After the mappers generate the report, now it comes to the turn of reducers.The reducers will grab the sell value from every month for Mumbai location.Eventually, they will all sell value to generate the outcome.In this MapReduce training course, you will learn something that is going to be the next big thing soon, generating lots of opportunities in the new future. You will learn how to work with mass data, unstructured data. Working with various kinds of data and try to get all of them on the same page anyway is what you will study here. In technical terms, you will be getting a practical insight into the working of data scientists. In addition to data processing, you will also learn to develop a program in HIVE, PIG, MapReduce, and Sqoop.Every organization has its requirement for data analysis so it is very important to develop a customized program that can generate the desired output. You will see and learn how the sub-modules of Hadoop like PIG or HIVE could be used to reduce the complexity of the program. In addition to all those vital things, you will learn which framework should you use and in which case. By the time you come to the end of the MapReduce certification, you will be enough cognizant to play with abundant data.
