|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/practical-text-analytics-using-spacy-v3/
课程评论：没有评论
课程名称：使用spaCy v3.0进行实用文本分析 课程概述：文本分析是什么？可以这样定义：“文本分析是将非结构化文本文档转换为可用的结构化数据的过程。文本分析通过将句子和短语分解成其组成部分，然后使用复杂的软件规则和机器学习算法评估每个部分的角色和含义。”（来源：Lexalytics网站）在spaCy中，您可以通过两种方式使用机器学习算法： 1. 使用spaCy和其他组织提供的预训练模型——例如，本课程中使用的en_core_web_md就是由创建spaCy的Explosion公司提供的一个预训练模型。 2. 在您的数据上训练自定义机器学习模型——在文档中通常被称为“统计模型”。 为什么要使用统计模型？spaCy的开发者在其文档中表示：“对于复杂任务，通常训练一个统计实体识别模型是更好的选择。然而，统计模型需要训练数据，因此在许多情况下，基于规则的方法更为实用。尤其是在项目初期：您可以使用基于规则的方法作为数据收集过程的一部分，以帮助您“引导”一个统计模型的建立。当您有一些示例并希望系统能够根据这些示例进行泛化时，训练模型是非常有用的。如果在局部上下文中有线索，它的效果尤其好。例如，如果您尝试检测人名或公司名，您的应用可能会从统计命名实体识别模型中获益。基于规则的系统是一个良好的选择，如果您希望在数据中找到的示例数量相对有限，或者如果有非常明确的、可以用令牌规则或正则表达式表示的结构性模式。例如，国家名称、IP地址或URL等内容，您可能可以使用纯基于规则的方法很好地处理。 需要澄清的是，我并不反对开发统计模型——但正如文档所明确指出的，通常从基于规则的系统开始更为实用。本课程的主要目标之一是提供关于仅使用规则基础系统能够做什么和不能做什么的清晰理解。实际上，本课程中仅使用一个数据集，这使得学生容易区分这两者。当您将基于规则的系统与本课程中描述的数据可视化技术相结合时，您还将获得对数据集的非常好的理解。然后，您可以利用这种理解来改善您选择建立的统计模型。依我看，大多数人在使用spaCy规则进行文本分析时仅仅只是触及表面。我希望本课程能为他们在处理这一任务时提供许多新的见解。
What is text analytics?I like this definition: "Text analytics is the process of transforming unstructured text documents into usable, structured data. Text analysis works by breaking apart sentences and phrases into their components, and then evaluating each part's role and meaning using complex software rules and machine learning algorithms." [Source: Lexalytics website]In spaCy, you can use machine learning algorithms in two ways1) pretrained models provided by spaCy and other organizations - for example the en_core_web_md, which I use in this course, is a pretrained model provided by Explosion, the company which created spaCy2) custom machine learning models that you train on your data - which is often referred to in the documentation as "statistical models"Why not statistical models?This is what the makers of spaCy say in their documentation:"For complex tasks, it's usually better to train a statistical entity recognition model. However, statistical models require training data, so for many situations, rule-based approaches are more practical. This is especially true at the start of a project: you can use a rule-based approach as part of a data collection process, to help you "bootstrap" a statistical model.Training a model is useful if you have some examples and you want your system to be able to generalize based on those examples. It works especially well if there are clues in the local context. For instance, if you're trying to detect person or company names, your application may benefit from a statistical named entity recognition model.Rule-based systems are a good choice if there's a more or less finite number of examples that you want to find in the data, or if there's a very clear, structured pattern you can express with token rules or regular expressions. For instance, country names, IP addresses or URLs are things you might be able to handle well with a purely rule-based approach."Just to clarify, I am not against developing statistical models - but as the documentation states quite clearly, it is often more practical to start with rules based systems. One of my main aims in this course is to provide a solid understanding of what you can and cannot do using just a rules based system - in fact I use only one dataset in this entire course so it is a lot easier for the students to make this distinction.When you combine a rules based system with the data visualization technique I describe in this course, you will also gain a very good understanding of your dataset. You can then use this understanding to improve your statistical model if you choose to build one. In my view, most people barely scratch the surface when it comes to using spaCy rules for text analytics. I hope this course will provide them a lot of new insight into how to approach this task.
