|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/scrape-websites-phantomjs-casperjs/
课程评论：没有评论
课程名称：使用 PhantomJS 和 CasperJS 抓取网站数据 课程概述：在本课程中，您将学习如何使用 CasperJS 从网页中抓取数据。课程包含5个示例项目，帮助您全面理解使用 CasperJS API 的无头浏览器的强大功能。 您将学习到： - 高级网络抓取概念的深入理解 - 如何使用 CasperJS 进行 DOM 操作和 UI 交互测试 课程内容： - 首先，将介绍 PhantomJS 和 CasperJS 的工作原理以及如何安装这些框架。 - 接下来，讨论我们的工作流程以及可以传递到 Casper 对象的选项。 - 然后，我们将深入本课程的核心内容，通过5个项目进行探索。 项目内容将包括： - 如何抓取使用 JavaScript 渲染的网站，而非标准 HTML - 如何在抓取元素前等待 AJAX 加载的数据出现 - 如何提交表单进行授权和搜索 - 如何定义导航步骤，如登录、点击按钮和跟随链接 - 如何编写和保存指定数据到表格，并输出为 .html 文件或 JSON 格式 - 如何截取整个网页和特定容器的屏幕截图 什么是 PhantomJS？ PhantomJS 是一个完整的 Web 堆栈，采用无头浏览器。它使我们能够在网页上执行许多有趣的操作，如页面操作、模拟用户交互以及动态捕获和保存网站数据的能力。 什么是 CasperJS？ CasperJS 是一个建立在 PhantomJS 之上的独立框架，兼容大多数操作系统。课程将主要集中于 Casper API，我们将使用该 API 编写所有网络抓取脚本。 您需要知道的： 您应该具备 JavaScript 基础知识，包括回调函数的概念。如果您了解一些 jQuery，将会有所帮助。课程中借助 lodash 替代了原生 JavaScript API 中的内置 Map 方法。
In this course you will learn how to scrape data from web pages using CasperJS. This course consists of 5 example projects to help you fully understand the powers of the headless browser using the CasperJS API. What You Will Learn You will gain a thorough understanding of advanced web scraping concepts and also gain an insight into how to use the CasperJS for Testing DOM manipulation and UI interaction. What to Expect We'll begin with an overview of how both PhantomJS and CasperJS works along with how to install these frameworks. Next, we'll discuss what our workflow will look like and the options we can pass into a Casper object. Then we'll dive into the meat of this course by working through 5 projects. The Projects Will Cover How to scrape websites that are rendered with JavaScript instead of standard HTMLHow to wait for AJAX loaded data to appear before scraping elementsHow to submit forms both for Authorization and when making searchesHow to define navigation Steps - like logging into a site, clicking a button and following linksHow to write and save specified data in tables then output as an.html file or as JSON.And how to take screenshots both of full web pages and specific containers What is PhantomJS? PhantomJS is a Full Web Stack that employs a headless browser. Phantom gives us the power to perform many interesting actions on a web page, such as: performing page manipulation, simulating user interaction and the ability to dynamically capture and save website data. What is CasperJS? CasperJS is a stand-alone framework built on top Phantom and is compatible with most operating systems. The focus of this course will be on the Casper API and we'll be using this API to write all our web scraping scripts. What You Should Know You should already know JavaScript basics. Including what a callback function is. It will help if you know some jQuery. We use lodash in of our examples but only as a replacement for the built-in Map method that's part of the native Javascript API.
