|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/apache-sqoop-with-hadoop-hive-and-mysql/
课程评论：没有评论
课程名称：掌握Apache SQOOP与Hadoop、Hive、MySQL（适用于Mac和Windows） 课程概述： Apache SQOOP旨在将数据从关系型数据库（如Oracle、MySQL等）导入到Hadoop系统中。Hadoop非常适合批处理大量数据，现已成为业内标准。在实际场景中，使用SQOOP可以将关系表中的数据转移到Hadoop中，借助Hadoop的并行处理能力处理大量数据并产生有意义的数据洞察。通过SQOOP导出功能，Hadoop处理的结果也可以重新存储到关系表中。大数据分析的第一步始于数据的进口，Apache SQOOP在此过程中发挥了关键作用。 课程内容： 在本课程中，您将逐步学习有关Apache Sqoop的所有知识以及如何将其集成到Hadoop生态系统中。每一个概念都通过真实世界的示例进行解释，您将学习如何创建数据管道，以便在Hadoop中导入/导出数据。课程中将深入讲解以下主要概念： 1. **Apache Sqoop - 导入主题**： - 默认Hadoop存储与特定目标在Hadoop存储 - 控制并行性、覆盖现有数据、追加数据 - 从MySQL表中加载特定列，控制数据拆分逻辑 - 调试Sqoop操作，处理各种文件格式（TEXT、SEQUENCE、AVRO、PARQUET、ORC） - 增量加载数据，直接写入Hive表，导入整个MySQL数据库到Hive数据库 2. **Apache Sqoop - 导出主题**： - 将数据从Hadoop移动到MySQL表，避免部分导出问题 - 在导出时执行更新操作 3. **Apache Sqoop - 作业主题**： - 创建、列出和检查现有Sqoop作业 - 执行和删除Sqoop作业，启用密码存储以便于生产中的执行 学习成果： 完成该课程后，学员将掌握一个在多项认证考试中经常被提及的主题，特别是： 1. CCA Spark和Hadoop开发者考试（CCA175） 2. Hortonworks数据平台（HDP）认证开发者考试（HDPCD） 授课团队： 本课程由在世界财富100强公司中积累了丰富经验的专业人士教授，他们为客户管理了每天超过百TB数据的提取、转换与处理，提供数据分析服务。继成功推出“完全的ElasticSearch课程”后，同一团队为您带来了完整的Apache Sqoop与Hadoop、Hive、MySQL的学习课程。 课程还将提供逐步安装所需工具和组件的说明，以便您在自己的机器上运行示例。您将获得可操作的代码，以便于探索和扩展，所有代码示例均在视频课程中展示。Windows用户需在设备上安装虚拟机以设置单节点Hadoop集群，而MacBook或Linux用户则可直接在其机器上安装Hadoop和Sqoop组件，课程中将详细说明该过程。
WHY APACHE SQOOPApache SQOOP is designed to import data from relational databases such as Oracle, MySQL, etc to Hadoop systems. Hadoop is ideal for batch processing of huge amounts of data. It is industry standard nowadays. In real world scenarios, using SQOOP you can transfer the data from relational tables into Hadoop and then leverage the parallel processing capabilities of Hadoop to process huge amounts of data and generate meaningful data insights. The results of Hadoop processing can again be stored back to relational tables using SQOOP export functionality. Big data analytics start with data ingestion and thats where apache sqoop comes in picture. It is the first step in getting the data ready.ABOUT THIS COURSEIn this course, you will learn step by step everything that you need to know about Apache Sqoop and how to integrate it within Hadoop ecosystem. With every concept explained with real world like examples, you will learn how to create Data Pipelines to move in/out the data from Hadoop. In this course, you will learn following major concepts in great details:APACHE SQOOP - IMPORT TOPICS >default hadoop storagespecific target on hadoop storagecontrolling parallelismoverwriting existing dataappend dataload specific columns from MySQL tablecontrol data splitting logicdefault to single mapper when neededSqoop Option filesdebugging Sqoop OperationsImporting data in various file formats - TEXT, SEQUENCE, AVRO, PARQUET & ORCdata compression while importingcustom query executionhandling null strings and non string valuessetting delimiters for imported data filessetting escaped charactersincremental loading of datawrite directly to hive tableusing HCATALOG parametersimporting all tables from MySQL database importing entire MySQL database into Hive databaseAPACHE SQOOP - EXPORT TOPICS > Move data from Hadoop to MySQL tableMove specific columns from Hadoop to MySQL tableAvoid partial export issuesUpdate Operation while exportingAPACHE SQOOP - JOBS TOPICS >create sqoop joblist existing sqoop jobscheck metadata about sqoop jobsexecute sqoop jobdelete sqoop jobenable password storage for easy execution in productionWHAT YOU WILL ACHIEVE AFTER COMPLETING THIS COURSEAfter completing this course, you will cover one of the topic that is heavily asked in below certifications. You will need to take other lessons as well to fully prepare for the test. We will be launching other courses soon.1. CCA Spark and Hadoop Developer Exam (CCA175)2. Hortonworks Data Platform (HDP) Certified Developer Exam (HDPCD)WHO ARE YOUR INSTRUCTORSThis course is taught by professionals with extensive experience in handling big data applications for Fortune 100 companies of the world. They have managed to create data pipelines for extracting, transforming & processing over 100's of Terabytes of data in a day for their clients providing data analytics for user services. After successful launch of their course - Complete ElasticSearch with LogStash, Hive, Pig, MR & Kibana, same team has brought to you a complete course on learning Apache Sqoop with Hadoop, Hive, MySQL.You will also get step by step instructions for installing all required tools and components on your machine in order to run all examples provided in this course. Each video will explain entire process in detail and easy to understand manner.You will get access to working code for you to play with it and expand on it. All code examples are working and will be demonstrated in video lessons.Windows users will need to install virtual machine on their device to setup single node hadoop cluster while MacBook or Linux users can directly install hadoop and sqoop components on their machines. The step by step process is illustrated within course.
