|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/web-scraping-in-nodejs/
课程评论：没有评论
课程名称：使用 Nodejs 和 JavaScript 进行网页抓取 课程概述：本课程将教授如何使用 JavaScript 和 Node.js 来抓取网站，包括实际示例和真实网站的应用。课程内容涵盖了 Nodejs 请求、Cheerio、NightmareJs 和 Puppeteer 等工具的使用。学习过程中将采用最新的 JavaScript ES7 语法，使用 async/await。 在课程中，您将学习如何从 Craigslist 网站抓取软件工程职位的信息。接着，您将掌握如何抓取更复杂的网站，例如 iMDB 和 AirBnB，这些网站需要 JavaScript 的支持。课程还将展示如何通过逆向工程找到隐藏的 API，从而避免浪费时间创建网页抓取程序。 另外，课程将教授如何在开发阶段避免被网站封锁，可以通过创建测试驱动的抓取程序来模拟 HTML，而不是在调试时频繁访问网站。如果您遇到封锁情况，也将提供应对方案，使抓取程序能够正常运作。 您还将学习如何在网络连接不佳的服务器上进行抓取，或在自身连接不佳的情况下进行抓取。抓取的结果可以保存到 CSV 文件或 MongoDB 中。 课程将涵盖如何构建一个每小时抓取一次的网站，以及如何将其部署到云服务器，如 Heroku 或 Google Cloud。此外，课程还将展示如何抓取需要密码的网站（如 Craigslist），并教您如何使用 Nodejs Express 提供抓取结果的 REST API，甚至还有如何构建一个显示结果的 React 前端。 最后，课程中还包含了创建基本 GraphQL API 的部分，以及一个特别环节，展示如何仅通过 Request 抓取 Facebook。若在抓取过程中您遇到问题，也可以随时联系我获取帮助，我很乐意为您指点方向！
In this course you will learn how to scrape a websites, with practical examples on real websites using JavaScript Nodejs Request, Cheerio, NightmareJs and Puppeteer. You will be using the newest JavaScript ES7 syntax with async/await. You will learn how to scrape a Craigslist website for software engineering jobs, using Nodejs Request and Cheerio. You will be using the newest JavaScript ES7 syntax with async/await. You will then learn how to scrape more advanced websites that require JavaScript such as iMDB and AirBnB using NighmareJs and Puppeteer.I'm gong to also show you with a practical real-life website, how you can even avoid wasting time on creating a web scraper in the first place, by reverse engineering websites and finding their hidden API's!Learn how to avoid being blocked from websites when developing out your scraper, by building out the scraper in a test-driven way with mocked html, rather than hitting the website every time as you're debugging and developing it. You'll also learn what you can do if you're blocked and your alternatives to get your scraper up and running regardless!You will also learn how to scrape on a server with a bad connection, or even if you have a bad connection. You'll even learn how to save your results to a CSV file and MongoDB!How do you build a scraper that scrapes every 1 hour (or other interval), and deploy it do a cloud host like Heroku or Google Cloud? Let me show you, quick and easy!How do you scrape a site requiring passwords? I'm going to show you that too with a real website (Craigslist)!How do you serve your scraping results in a REST API with Nodejs Express? And how can we build a React frontend that's showing the results? You'll learn that too, in the quickest and simplest way possible!Plus, a section covering how to make a basic GraphQL API is included in the course.As a last cherry on the top, I have a section containing a secret backdoor showing you how to scrape Facebook using only Request!If you have issues regarding a site you're trying to scrape yourself, it's totally okay to reach out to me for some help. I'd be happy to point you in the right direction! Whatever issues my students are facing, I use that to expand on my course!
