|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/ahol-dl4nlp2/
课程评论：没有评论
课程总结：深度学习在自然语言处理中的应用 - 第二部分 本课程是“深度学习在自然语言处理中的应用”系列的一部分。在本课程中，我们将介绍一些关键概念，如编码器-解码器注意力模型、ELMo、GLUE、变换器（Transformers）、GPT和BERT。这些概念是理解现代自然语言处理中的高级深度学习模型的基础。 课程结构主要分为两个部分： 第一部分，我们将探讨编码器-解码器模型在机器翻译中的应用，以及如何实现束搜索解码器。接着，我们将深入讲解编码器-解码器注意力的概念，并介绍多种类型的注意力机制，如全局注意力、局部注意力、层次注意力，以及利用卷积神经网络（CNN）和长短期记忆网络（LSTM）处理句对的注意力。我们还将讨论注意力的可视化，最后讲解ELMo，这是一种使用递归模型计算上下文敏感的词嵌入方法。 第二部分将集中讨论GLUE基准测试中的各种任务以及其他自然语言处理任务相关的基准数据集。随后，我们将开启现代自然语言处理的旅程，深入理解编码器-解码器变换器模型的各个部分。我们将详细探讨变换器的概念，如自注意力、多头注意力、位置嵌入、残差连接和掩蔽注意力。接下来，我们将介绍两个最流行的变换器模型：GPT和BERT。在GPT部分，我们将讨论GPT的训练方法以及GPT2和GPT3之间的差异。在BERT部分，我们将探讨BERT与GPT的不同之处，如何通过掩蔽语言模型和下一句预测任务进行预训练，并简单介绍BERT的微调以及多语言BERT。 通过本课程的学习，学员将掌握现代自然语言处理中的重要技术，能够更好地理解和应用这些深度学习模型。
This course is a part of "Deep Learning for NLP" Series. In this course, I will introduce concepts like Encoder-decoder attention models, ELMo, GLUE, Transformers, GPT and BERT. These concepts form the base for good understanding of advanced deep learning models for modern Natural Language Processing.The course consists of two main sections as follows. In the first section, I will talk about Encoder-decoder models in the context of machine translation and how beam search decoder works. Next, I will talk about the concept of encoder-decoder attention. Further, I will elaborate on different types of attention like Global attention, local attention, hierarchical attention, and attention for sentence pairs using CNNs as well as LSTMs. We will also talk about attention visualization. Finally, we will discuss ELMo which is a way of using recurrent models to compute context sensitive word embeddings.In the second section, I will talk about details about the various tasks which are a part of the GLUE benchmark and details about other benchmark NLP datasets across tasks. Then we will start our modern NLP journey with understanding different parts of an encoder-decoder Transformer model. We will delve into details of Transformers in terms of concepts like self attention, multi-head attention, positional embeddings, residual connections, and masked attention. After that I will talk about two most popular Transformer models: GPT and BERT. In the GPT part, we will discuss how is GPT trained and what are differences in variants like GPT2 and GPT3. In the BERT part, we will discuss how BERT is different from GPT, how it is pretrained using the masked language modeling and next sentence prediction tasks. We will also quickly talk about finetuning for BERT and multilingual BERT.
