|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/mastering-aws-lambda-functions/
课程评论：没有评论
课程名称：掌握使用Python的数据工程师AWS Lambda函数 课程概述： 本课程旨在教您如何通过构建一个端到端的数据管道来学习AWS Lambda函数，使用Python作为编程语言，并结合重要的AWS服务，如Boto3、S3、DynamoDB、ECR、CloudWatch、Glue Catalog、Athena等。课程内容涵盖如何使用Zip文件、自定义Docker镜像和Layers来开发和部署Lambda函数，并了解如何通过Eventsbridge和Step Functions触发Lambda函数。 您将学习在Windows上设置所需工具以开发基于Python的ETL数据管道代码，包括配置WSL、Docker Desktop和Visual Studio Code等。还将介绍如何创建AWS账号、配置AWS CLI以及查看项目使用的数据集。 核心学习内容包括： - 使用Python Boto3将数据从源头导入AWS S3，结合Pandas进行数据处理，以及通过REST API获取源文件。 - 学习AWS Lambda函数使用Python 3.9运行时环境进行开发，重构应用，将逻辑打包成Zip文件并部署为AWS Lambda函数，同时捕获书签和作业运行详情在DynamoDB中。 - 通过AWS控制台创建和验证使用Zip文件的AWS Lambda函数，调试与AWS Lambda函数相关的问题。 - 为应用构建自定义Docker镜像并将其推送至AWS ECR，利用该镜像创建AWS Lambda函数，并进行验证。 - 理解S3事件通知或基于S3的触发器在Lambda函数中的使用，开发另一款Python应用将数据转换为Parquet格式并写入S3。 - 构建使用AWS S3事件通知的有序管道，并通过AWS EventsBridge调度第一个Lambda函数。 - 在S3位置创建AWS Glue Catalog表，并通过AWS Athena运行SQL查询进行验证。 通过本课程，您将全面了解如何部署和调度Lambda函数，并通过Glue Catalog和AWS Athena验证数据，还将学习如何为Lambda函数使用Layers。 课程主要收获包括： - 开发Python应用并以Zip包或自定义Docker镜像的形式部署为Lambda函数。 - 通过CloudWatch日志监控和排查问题。 - 获取用于演示的整个应用代码及其核心逻辑的笔记本。 - 有能力使用多个AWS服务构建解决方案，如Boto3、S3、DynamoDB、ECR、CloudWatch、Glue Catalog、Athena等。
Do you want to learn AWS Lambda Functions by building an end-to-end data pipeline using Python as Programming Language and other key AWS Services such as Boto3, S3, Dynamodb, ECR, Cloudwatch, Glue Catalog, Athena, etc? Here is one course using which you will learn AWS Lambda Functions by implementing an end-to-end pipeline by using all the services mentioned.As part of this course, you will learn how to develop and deploy lambda functions using the zip files, custom docker images as well as layers. Also, you will understand how to trigger lambda functions from Eventsbridge as well as Step Functions.Set up required tools on Windows to develop the code for ETL Data Pipelines using Python and AWS Services. You will take care of setting up Ubuntu using wsl, Docker Desktop, and Visual Studio Code along with Remote Development Extension Kit so that you can develop Python-based applications using AWS Services.Setup Project or Development Environment to develop applications using Python and AWS Services on Windows and Mac.Getting Started with AWS by creating an account in AWS and also configuring AWS CLI as well as Review Data Sets used for the projectDevelop Core Logic to Ingest Data from source to AWS s3 using Python boto3. The application will be built using Boto3 to interact with AWS Services, Pandas for date arithmetic, and requests to get the files from the source via REST API.Getting Started with AWS Lambda Functions using Python 3.9 Run-time EnvironmentRefactor the application, and build a zip file to deploy as AWS Lambda Function. The application logic includes capturing bookmarks as well as Job Run details in Dynamodb. You will also get an overview of Dynamodb and how to interact with Dynamodb to manage Bookmark as well as Job Run details.Create AWS Lambda Function using a Zip file, deploy using AWS Console and Validate.Troubleshoot issues related to AWS Lambda Functions using AWS CloudwatchBuild a custom docker image for the application and push it to AWS ECRCreate AWS Lambda Function using the custom docker image in AWS ECR and then validate.Get an understanding of AWS s3 Event Notifications or s3-based triggers on Lambda Function.Develop another Python application to transform the data and also write the data in the form of Parquet to s3. The application will be built using Pandas by converting 10,000 records at a time to Parquet.Build orchestrated pipeline using AWS s3 Event Notifications between the two Lambda Functions.Schedule the first lambda function using AWS EventsBridge and then validate.Finally, create an AWS Glue Catalog table on the s3 location which has parquet files, and validate by running SQL Queries using AWS Athena.After going through the complete life cycle of Deploying and Scheduling Lambda Function and also validating the data by using Glue Catalog and AWS Athena, you will also understand how to use Layers for Lambda Function.Here are the key takeaways from this training:Develop Python Applications and Deploy as Lambda Functions by using a Zip-based bundle as well as a custom docker image.Monitor and troubleshoot the issues by going through Cloudwatch logs.The entire application code used for the demo along with the notebook used to come up with core logic.Ability to build solutions using multiple AWS Services such as Boto3, S3, Dynamodb, ECR, Cloudwatch, Glue Catalog, Athena, etc
