|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/building-smart-city-real-time-streaming-systems/
课程评论：没有评论
课程名称：为智能城市构建实时流媒体系统 课程概述：本课程将带您全面了解如何构建一个复杂的智能城市端到端实时数据流媒体管道，从数据采集、处理到存储和可视化的每一个环节都有涉及。在这个动手实践的课程中，您将使用一系列行业领先的工具和技术，包括 Apache Kafka（用于高吞吐量、容错消息传递）、Apache Spark（用于实时数据处理）、Docker（用于容器化）以及 AWS 的一系列服务，如 S3、Glue、Athena、IAM 和 Redshift（用于基于云的数据存储、管理和分析）。 课程内容包括： - 设计智能城市端到端实时数据流媒体管道 - 设置和配置 Docker 容器以便开发和部署 - 编写 IoT 服务生产者以生成多样化的数据流，包括车辆信息、GPS 坐标、交通更新、天气状况和紧急事件 - 将数据流传输到 Apache Kafka 进行实时处理和分发 - 使用 AWS 服务（如 S3、Glue、Athena、IAM 和 Redshift）进行基于云的数据存储、管理和分析 - 配置 S3 存储桶和策略，管理 IAM 角色和凭证以确保安全访问 - 使用 AWS Glue 进行数据目录和转换，实现与下游分析工具的无缝集成 - 使用 Athena 查询存储在 AWS S3 中的数据，并对数据湖执行强大的 SQL 查询 - 设置和管理 AWS Redshift 的可扩展数据仓库，以进行高级分析和可视化 - 故障排除、调试以及优化您的流媒体解决方案，以确保最大性能和可靠性 - 构建一个展示您在实时数据工程和基于云的分析能力的项目组合 此外，您还将深入了解 AWS 设置，包括如何配置 S3 存储桶、管理 IAM 角色和凭证、以及利用 AWS Glue 进行数据目录化和转换。通过 Athena，您将对数据湖执行强大的查询，而 Redshift 将作为您可扩展分析的数据库。 课程结束时，您不仅将构建一个完整的智能城市数据管道，还将提高故障排除、调试和优化流媒体解决方案的技能。立即注册，开启数据工程领域无限可能的大门！
In this comprehensive Udemy course, you'll be building a sophisticated Smart City End to End Realtime data streaming pipeline, covering every aspect from data ingestion to processing, and finally storage and visualization.Throughout this hands-on course, you'll leverage an arsenal of industry-leading tools and technologies, including Apache Kafka for high-throughput, fault-tolerant messaging, Apache Spark for real-time data processing, Docker for containerization, and a suite of AWS services such as S3, Glue, Athena, IAM, and Redshift for cloud-based data storage, management, and analytics.What's in the course?Architect a Smart City End to End Realtime data streaming pipelineSet up and configure Docker containers for development and deploymentCode IoT service producers for generating diverse data streams including vehicle information, GPS coordinates, traffic updates, weather conditions, and emergency incidentsStream data into Apache Kafka for real-time processing and distributionUtilize AWS services including S3, Glue, Athena, IAM, and Redshift for cloud-based data storage, management, and analyticsConfigure S3 buckets with policies and manage IAM roles and credentials for secure accessUse AWS Glue for data cataloging and transformation, enabling seamless integration with downstream analytics toolsQuery data stored in AWS S3 with Athena, executing powerful SQL queries on your data lakeSet up and manage a scalable data warehouse with AWS Redshift for advanced analytics and visualizationTroubleshoot, debug, and optimize your streaming solution for maximum performance and reliabilityBuild a portfolio project showcasing your proficiency in real-time data engineering and cloud-based analyticsBut that's just the beginning! You'll delve into the intricacies of AWS setup, learning how to configure S3 buckets with policies, manage IAM roles and credentials, and leverage AWS Glue for data cataloging and transformation. With Athena, you'll execute powerful queries on your data lake, while Redshift will serve as your data warehouse for scalable analytics.By the end of this course, you'll not only have built a fully functional Smart City data pipeline, but you'll also have polished your skills in troubleshooting, debugging, and optimizing your streaming solution. Enroll now and unlock the door to endless possibilities in the realm of data engineering!
