|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/supercharged-web-scraping-with-asyncio-and-python/
课程评论：没有评论
课程名称：使用Asyncio和Python的大规模网络爬虫 课程概述：网络爬虫是自动打开任何网站并提取我们认为重要的数据的过程。这对于互联网、搜索引擎、数据科学、自动化、机器学习等领域至关重要。打开网站并提取数据只是网络爬虫的部分内容，数据解析才是其真正的价值所在。本课程涵盖以下内容： - 基础的Python网络爬虫 - 使用Selenium进行网络爬虫 - 同步与异步的比较 - 使用Asyncio进行异步网络爬虫 那么什么是异步代码？它对我们有什么好处？异步代码是一种几乎同时执行多个函数的方法。虽然它们不是完全同时运行，但实际上是并发的。这意味着我们可以在更少的时间内完成更多的任务，特别是在进行数据挖掘或爬取时，这种节省时间的效果尤为显著。 想象一下，如果你要重建谷歌的搜索引擎，你需要定期爬取数万亿个网页，以帮助生成搜索结果。显然，不可能一次性爬取所有这些网页，但如果使用同步爬虫（如使用Python的requests库或Selenium），爬取1,000个页面将非常耗时。 如果你之前有很多网络爬虫的经验，但从未使用Python的asyncio，这门课程将帮助你更好地理解基础知识，并将你的爬虫能力提升到一个新的水平。让我们开始吧！
Web scraping is simply automatically opening up any website and grabbing the data you find important on that website. It's fundamental to the internet, search engines, Data Science, automation, machine learning, and much more.Opening websites and extracting data are only part of what makes web scraping great. It's the parsing of the data that's where the value is.This project will cover:Basic web scraping with PythonWeb scraping with SeleniumSync vs AsyncAsynchronous Web scraping with AsyncioBut why asynchronous code? What is it? How does it benefit us?Asynchrounous code is a way to execute multiple functions basically at once. It's not actually at the exact same time but it's close. (They actually run concurrently). This means that we can do more things in less time and, when it comes to mining or scraping data, this time saving is absolutely significant.Imagine for a moment you're recreating google's search engine. You'd have to scrape trillions (if not more) web pages on a regular interval to help with the search results. Of course you're not going to be scraping all of the trillions of pages at once but the idea is that scraping event 1,000 pages would take a very long time doing it synchronously (like using Python requests and/or just selenium).If you've done a lot of web scraping before but never used Python's aysncio, this course will help you better understand the fundamentals and bring your scraping game to another level.Let's get started!
