|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/spark-performance-tuning-for-data-engineers-part1-storage/
课程评论：没有评论
课程名称：Spark 性能调优：第一部分 - 存储 课程概述：通过掌握与存储相关的性能调优技术，充分发挥 Apache Spark 的潜力。本课程包含丰富的实际场景、指导演示和实践案例，帮助您优化 Spark 存储策略，以提高速度、效率和可扩展性。该课程非常适合中级数据工程师、Spark 开发人员以及希望优化 Spark 作业、降低资源成本、并确保大规模数据应用程序快速、可靠性能的有志建筑师。 课程内容包括： 1. 理解 Apache Spark 内部如何处理存储：内存与磁盘的关系。 2. 学习何时以及如何有效使用 Spark 缓存和持久化。 3. 比较并选择合适的存储级别：MEMORY_ONLY、MEMORY_AND_DISK 等。 4. 使用真实案例和动手演示来基准测试存储决策。 5. 学习如何使用 Spark UI 监控存储指标。 6. 处理内存溢出、磁盘 I/O 瓶颈和集群环境中的存储调优。 7. 在云和本地 Spark 集群中应用存储优化的最佳实践。 为什么参加本课程？ - 100% 实践：专注于实际实施，而不仅仅是理论。 - 设计面向数据工程师、Spark 开发者和大数据从业者。 - 涵盖基础概念和高级调优技术。 - 教导如何使用真实指标衡量性能提升。 - 帮助您做出成本高效的大数据存储决策。 涵盖的工具与技术： - Apache Spark（2.x 和 3.x） - DataBricks - Spark UI - HDFS, 数据湖（存储场景）
Unlock the true potential of Apache Spark by mastering storage-related performance tuning techniques. This hands-on course is packed with real-world scenarios, guided demos, and practical use cases that will help you fine-tune Spark storage strategies for speed, efficiency, and scalability.This course is perfect for Intermediate Data Engineers & Spark Developers as well as Aspiring Achitects who wants to optimize Spark jobs, reduce resource costs, and ensure fast, reliable performance for large-scale data applications.What You'll Learn1. Understand how Apache Spark handles storage internally: memory vs disk2. Learn when and how to use Spark caching and persistence effectively3. Compare and choose the right storage levels: MEMORY_ONLY, MEMORY_AND_DISK, etc.4. Use real-world examples and hands-on demos to benchmark storage decisions5. Learn how to monitor storage metrics using the Spark UI6. Handle memory spills, disk I/O bottlenecks, and storage tuning in cluster environments7. Apply best practices for storage optimization in cloud and on-prem Spark clustersWhy Take This Course?100% Hands-on: Focused on practical implementation, not just theoryDesigned for Data Engineers, Spark Developers, and Big Data PractitionersCovers both foundational concepts and advanced tuning techniquesTeaches how to measure performance gains using real metricsHelps you make cost-efficient decisions for big data storageTools & Technologies CoveredApache Spark (2.x and 3.x)DataBricksSpark UIHDFS, DataLake (for storage scenarios)
