|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/pig-for-wrangling-big-data/
课程评论：没有评论
课程名称：Pig用于大数据处理 课程概述： 本课程适合具备基本SQL查询语言知识，对Hadoop生态系统和MapReduce有简要了解的学员。由一支包括两位斯坦福大学毕业、前谷歌员工和两位前Flipkart首席分析师组成的团队授课，他们在大规模数据处理方面拥有数十年的实践经验。课程的重点是Pig，这是一种能处理任何数据类型的工具，可以将数据转换为干净、可存储于数据仓库中的结构化形式，便于报告和分析。 课程内容包括： - Pig基础知识：原子数据类型和复杂数据类型（如Bags、Maps、Tuples），以及基本转换操作，如Filter、Foreach、Load、Dump、Store、Distinct、Limit、Order by等内置函数。 - 高级数据转换与优化：学习嵌套Foreach的复杂用法，连接的优化（使用“parallel”、“merge”、“replicated”等关键字），Co-groups和Semi-joins，调试技巧使用Explain和Illustrate命令。 - 实际案例：使用Pig清理服务器日志。 此课程内容丰富，适合希望深入了解大数据处理和分析工具的学习者。
Prerequisites: Working with Pig requires some basic knowledge of the SQL query language, a brief understanding of the Hadoop eco-system and MapReduce Taught by a team which includes 2 Stanford-educated, ex-Googlers and 2 ex-Flipkart Lead Analysts. This team has decades of practical experience in working with large-scale data processing jobs. Pig is aptly named, it is omnivorous, will consume any data that you throw at it and bring home the bacon! Let's parse that omnivorous: Pig works with unstructured data. It has many operations which are very SQL-like but Pig can perform these operations on data sets which have no fixed schema. Pig is great at wrestling data into a form which is clean and can be stored in a data warehouse for reporting and analysis. bring home the bacon: Pig allows you to transform data in a way that makes is structured, predictable and useful, ready for consumption. What's Covered: Pig Basics: Scalar and Complex data types (Bags, Maps, Tuples), basic transformations such as Filter, Foreach, Load, Dump, Store, Distinct, Limit, Order by and other built-in functions. Advanced Data Transformations and Optimizations: The mind-bending Nested Foreach, Joins and their optimizations using "parallel", "merge", "replicated" and other keywords, Co-groups and Semi-joins, debugging using Explain and Illustrate commands Real-world example: Clean up server logs using Pig
