|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/real-time-data-ingestion-in-elasticsearch-through-kafka/
课程评论：没有评论
**课程名称：** 通过 Kafka 实现 Elasticsearch 实时数据摄取 **课程概述：** 本课程深入探讨了 Elastic Stack 与 Apache Kafka 在日志/事件处理领域的紧密集成。课程将重点介绍 Kafka 作为数据缓冲和传输层，在处理海量数据中的关键作用。许多公司使用 Kafka 作为数据进入 Elasticsearch 进行快速搜索和分析的预处理阶段。 我们将分享在实际部署中运营 Kafka 和 Logstash 处理高流量数据的经验。通过构建一个完整的数据处理管道，结合 Kafka、Logstash、Apache Monitor 和 Kibana，您将学会如何高效地处理海量数据。一旦管道建立，您只需将输入文件放置在指定目录，数据将自动摄取到 Elasticsearch，您便可在 Kibana 网页界面中进行搜索和可视化。 **核心内容：** * Elastic Stack 和 Apache Kafka 的集成优势 * Kafka 在海量数据处理中的作用 * 运营高流量 Kafka 和 Logstash 的实践经验 * 构建从 Kafka 到 Elasticsearch 的端到端数据管道 * 使用 Logstash、Apache Monitor 和 Kibana 实现自动化数据摄取、搜索和可视化
The Elastic stack and Apache Kafka share a tight-knit relationship in the log/event processing realm. A number of companies use Kafka as a transport layer for storing and processing large volumes of data. In many deployments we've seen in the field, Kafka plays an important role of staging data before making its way into Elasticsearch for fast search and analytical capabilities. I'd like to shine more light on how to set up and manage Kafka when integrating with the Elastic Stack. Specifically, we'll discuss our experiences operating Kafka and Logstash under high volume.We will create a single data pipeline to process huge data with the help of kafka,logstash,apache monitor and kibana. Once we implement this we just need to put input files into source folder/directory and data will automatically get ingested into Elasticsearch and we can search/visualize data on ES web interface that is called Kibana.
