|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/flume-and-sqoop/
课程评论：没有评论
**课程名称：** Flume 和 Sqoop 用于大数据摄取 **课程概述：** 本课程由一支拥有数十年 Java 和处理海量数据实践经验的团队教授，其中包括两名斯坦福大学毕业、曾任职谷歌的专家。课程将重点介绍如何利用 Flume 和 Sqoop 将各种来源（如 Twitter 和 MySQL）的数据导入到 Hadoop 生态系统中的数据存储（HDFS、HBase 和 Hive）。 **核心内容解析：** * **导入数据：** Flume 和 Sqoop 在 Hadoop 生态系统中扮演着重要的数据传输角色。它们能够将来自本地文件系统、HTTP、MySQL 和 Twitter 等源头的数据，高效地传输到 HDFS、HBase 和 Hive 等数据存储中。这两个工具都内置了丰富的功能，能够屏蔽用户在系统间传输数据的复杂性。 * **Flume：** Flume 代理 (Agent) 可以将流式应用程序生成的数据传输到 HDFS 和 HBase 等数据存储中。 * **Sqoop：** Sqoop 用于将数据从传统的关系型数据库 (RDBMS) 大批量导入到 Hadoop 的存储架构（如 HDFS 或 Hive）中。 **涵盖主题：** * **多种数据源和数据存储的实际应用：** * **数据源：** Twitter, MySQL, Spooling Directory, HTTP * **目的地：** HDFS, HBase, Hive * **Flume 特性：** Flume Agents, Flume Events, Event bucketing, Channel selectors, Interceptors * **Sqoop 特性：** Sqoop 从 MySQL 导入数据, 使用 Sqoop Jobs 进行增量导入 **教学团队背景：** 教学团队由两名斯坦福大学毕业、曾任职谷歌的专家组成，在处理海量数据和 Java 开发方面拥有丰富的实践经验。
Taught by a team which includes 2 Stanford-educated, ex-Googlers. This team has decades of practical experience in working with Java and with billions of rows of data. Use Flume and Sqoop to import data to HDFS, HBase and Hive from a variety of sources, including Twitter and MySQL Let's parse that. Import data: Flume and Sqoop play a special role in the Hadoop ecosystem. They transport data from sources like local file systems, HTTP, MySQL and Twitter which hold/produce data to data stores like HDFS, HBase and Hive. Both tools come with built-in functionality and abstract away users from the complexity of transporting data between these systems. Flume: Flume Agents can transport data produced by a streaming application to data stores like HDFS and HBase. Sqoop: Use Sqoop to bulk import data from traditional RDBMS to Hadoop storage architectures like HDFS or Hive. What's Covered: Practical implementations for a variety of sources and data stores.. Sources: Twitter, MySQL, Spooling Directory, HTTPSinks: HDFS, HBase, Hive .. Flume features: Flume Agents, Flume Events, Event bucketing, Channel selectors, Interceptors .. Sqoop features: Sqoop import from MySQL, Incremental imports using Sqoop Jobs
