|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/transformers-in-computer-vision/
课程评论：没有评论
课程名称：计算机视觉中的变压器 课程概述：变压器网络是当前深度学习领域的新趋势。自2017年以来，变压器模型在自然语言处理（NLP）领域迅速崛起，几乎成为所有NLP任务的主流模型。然而，在计算机视觉（CV）领域，变压器的应用起步较晚，但自2020年以来开始逐渐占据重要地位。本课程将从注意力机制和变压器网络的介绍开始，由于变压器最初是在NLP领域提出，因此将通过NLP示例来简要说明其工作原理。接着，我们将讨论这一架构的优缺点，并探讨无监督或半监督预训练对变压器架构的重要性，简要介绍大规模语言模型（如BERT和GPT）。这一部分为后续探讨变压器在计算机视觉中的应用奠定基础。 我们将研究如何将注意力机制扩展到图像的二维空间域，讨论自注意力如何推广卷积，并在编码器-解码器元架构中实现这一点。我们将看到这一通用架构在图像和文本/NLP中的相似性，从而使变压器成为一种通用的函数逼近器。此外，还将讨论通道注意力和空间注意力、局部与全局注意力等主题。 在接下来的三个模块中，我们将重点讨论解决计算机视觉中主要问题的特定网络：分类、目标检测和分割。将讨论来自Google的视觉变压器（ViT）、来自微软的滑动窗口变压器（SWIN）、来自Facebook研究的检测变压器（DETR）和分割变压器（SETR）等多个网络。之后，我们还会探讨变压器在视频处理中的应用，重点是时空变压器在移动目标检测及多任务学习中的应用。 最后，我们将展示如何使用著名的Huggingface库和Pipeline接口轻松地将这些预训练架构应用于实际项目中。
Transformer Networks are the new trend in Deep Learning nowadays. Transformer models have taken the world of NLP by storm since 2017. Since then, they become the mainstream model in almost ALL NLP tasks. Transformers in CV are still lagging, however they started to take over since 2020. We will start by introducing attention and the transformer networks. Since transformers were first introduced in NLP, they are easier to be described with some NLP example first. From there, we will understand the pros and cons of this architecture. Also, we will discuss the importance of unsupervised or semi supervised pre-training for the transformer architectures, discussing Large Scale Language Models (LLM) in brief, like BERT and GPT.This will pave the way to introduce transformers in CV. Here we will try to extend the attention idea into the 2D spatial domain of the image. We will discuss how convolution can be generalized using self attention, within the encoder-decoder meta architecture. We will see how this generic architecture is almost the same in image as in text and NLP, which makes transformers a generic function approximator. We will discuss the channel and spatial attention, local vs. global attention among other topics.In the next three modules, we will discuss the specific networks that solve the big problems in CV: classification, object detection and segmentation. We will discuss Vision Transformer (ViT) from Google, Shifter Window Transformer (SWIN) from Microsoft, Detection Transformer (DETR) from Facebook research, Segmentation Transformer (SETR) and many others. Then we will discuss the application of Transformers in video processing, through Spatio-Temporal Transformers with application to Moving Object Detection, along with Multi-Task Learning setup.Finally, we will show how those pre-trained arcthiectures can be easily applied in practice using the famous Huggingface library using the Pipeline interface.
