|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/ahol-dl4nlp3/
课程评论：没有评论
课程名称：深度学习与自然语言处理 - 第三部分 课程概述： 本课程是“深度学习与自然语言处理”系列的一部分。主要介绍句子嵌入和生成式变换模型等概念，这些概念是理解现代自然语言生成的高级深度学习模型的基础。课程分为两个主要部分。 第一部分：句子嵌入 我们将从基本的词袋方法入手，通过对组成单词的词嵌入进行聚合来获得句子嵌入。具体讨论方法包括平均词袋、词移动距离、SIF和幂均值方法。接着，我们将介绍两个无监督学习方法：Doc2Vec和SkipThought。此外，还将探讨几种监督句子嵌入方法，包括递归神经网络、深度平均网络和InferSent。我们也将学习如何使用卷积神经网络（CNN）计算两个文本字符串之间的语义相似性，讨论深度语义相似模型（DSSM）。最后，还将介绍三个多任务学习方法，包括通用句子编码和MT-DNN，以及SentenceBERT。 第二部分：生成式变换模型 我们将从UniLM开始，讲解段落复发和相对位置嵌入在Transformer-XL中的应用。接下来，我们将探讨XLNet，该模型结合了Transformer-XL和排列语言建模。然后，将理解MASS中的跨度掩膜，以及BART的各种噪声处理方法。我们还将讨论受控自然语言生成方法CTRL，以及T5如何将每个学习任务视为文本到文本的任务。最后，将介绍ProphetNet如何将XLNet的双通道注意力建模扩展到n通道注意力建模，从而实现n-gram预测。 本课程将为想要深入了解自然语言处理及其深度学习应用的学习者提供丰富的知识背景。
This course is a part of "Deep Learning for NLP" Series. In this course, I will introduce concepts like Sentence embeddings and Generative Transformer Models. These concepts form the base for good understanding of advanced deep learning models for modern Natural Language Generation.The course consists of two main sections as follows. In the first section, I will talk about sentence embeddings. We will start with basic bag of words methods where sentence embedddings are obtained using an aggregation over word embeddings of constituent words. We will talk about averaged bag of words, word mover's distance, SIF and Power means method. Then we will discuss two unsupervised methods: Doc2Vec and SkipThought. Further, we will discuss about supervised sentence embedding methods like recursive neural networks, deep averaging networks and InferSent. CNNs can also be used for computing semantic similarity between two text strings; we will talk about DSSMs for the same. We will also discuss 3 multi-task learning methods including Universal Sentence Encodings and MT-DNN. Lastly, I will talk about SentenceBERT.In the second section, I will talk about multiple Generative Transformer Models. We will start with UniLM. Then we will talk about segment recurrence and relative position embeddings in Transformer-XL. Then get to XLNets which use Transformer-XL along with permutation language modeling. Next we will understand span masking in MASS and also discuss various noising methods on BART. We will then discuss about controlled natural language generation using CTRL. We will discuss how T5 models every learning task as a text-to-text task. Finally, we will discuss how ProphetNet extends 2-stream attention modeling from XLNet to n-stream attention modeling, thereby enabling n-gram predictions.
