|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/curiosity-driven-deep-reinforcement-learning/
课程评论：没有评论
课程名称：以好奇心驱动的深度强化学习 课程概述：在追求人工通用智能的过程中，强化学习必须能够应对奖励稀疏或完全缺失的环境。大多数现实系统的奖励往往在多个时间步后才出现，这使得代理在构建成功策略时信息十分有限。基于好奇心的强化学习通过赋予代理对其世界的内在好奇心，解决了这一问题，使其能够探索并学习成功的导航策略。在这门深入的深度强化学习高级课程中，富有动力的学生将学习如何从零开始实现前沿的人工智能研究论文。该课程节奏较快，适合那些已经有独立实现演员-评论家代理经验的学生。 我们将用流行的PyTorch框架实现两篇论文。第一篇论文涵盖了深度强化学习的异步方法，也被称为流行的异步优势演员-评论家算法（A3C）。在这里，学生将发现一个不需要GPU的新学习框架。我们将学习如何在Python中实现多线程，并利用它来并行训练多个演员-评论家代理。我们将超越论文中的基本实现，介绍强化学习的最新改进：广义优势估计。在OpenAI Gym的Atari库中的Pong环境下测试我们的代理，几小时内即可达到近乎世界级的性能。 接着，我们进入课程的核心：在奖励稀疏或完全缺失的环境中学习。这一新范式利用代理对环境的好奇心作为内在奖励，激励代理探索并学习可泛化的技能。我们将实现内在好奇心模块（ICM），这是为任何深度强化学习算法设计的附加模块。我们将在一个迷宫式环境中训练和测试我们的代理，该环境仅在代理达到目标时才会给予奖励。我们将明确展示相较于基础A3C算法的性能提升，充分证明好奇心驱动的深度强化学习的强大能力。 注意，这是一门针对富有动力和经验丰富学生的快速课程。课程将仅简要回顾强化学习和演员-评论家方法的基本概念，然后直接进入论文阅读和实现。无论是ICM还是异步方法的美在于，这些范式可以应用于几乎任何其他强化学习算法。两者都是高度可适应的，且可以在几乎不修改的情况下与近端策略优化、软演员-评论家或深度Q学习等算法结合。 在本课程中，学生将学习如何： - 实现深度强化学习论文 - 在Python中利用多核CPU进行并行处理 - 从零开始编码A3C算法 - 从基本原理编码ICM - 编码广义优势估计 - 修改OpenAI Gym Atari库 - 编写可扩展的模块化代码 本课程将以PyTorch实现启动，随后会推出TensorFlow 2版本。期待在课程中见到你！
If reinforcement learning is to serve as a viable path to artificial general intelligence, it must learn to cope with environments with sparse or totally absent rewards. Most real life systems provided rewards that only occur after many time steps, leaving the agent with little information to build a successful policy on. Curiosity based reinforcement learning solves this problem by giving the agent an innate sense of curiosity about its world, enabling it to explore and learn successful policies for navigating the world.In this advanced course on deep reinforcement learning, motivated students will learn how to implement cutting edge artificial intelligence research papers from scratch. This is a fast paced course for those that are experienced in coding up actor critic agents on their own. We'll code up two papers in this course, using the popular PyTorch framework. The first paper covers asynchronous methods for deep reinforcement learning; also known as the popular asynchronous advantage actor critic algorithm (A3C). Here students will discover a new framework for learning that doesn't require a GPU. We will learn how to implement multithreading in Python and use that to train multiple actor critic agents in parallel. We will go beyond the basic implementation from the paper and implement a recent improvement to reinforcement learning known as generalized advantage estimation. We will test our agents in the Pong environment from the Open AI Gym's Atari library, and achieve nearly world class performance in just a few hours. From there, we move on to the heart of the course: learning in environments with sparse or totally absent rewards. This new paradigm leverages the agent's curiosity about the environment as an intrinsic reward that motivates the agent to explore and learn generalizable skills. We'll implement the intrinsic curiosity module (ICM), which is a bolt-on module for any deep reinforcement learning algorithm. We will train and test our agent in an maze like environment that only yields rewards when the agent reaches the objective. A clear performance gain over the vanilla A3C algorithm will be demonstrated, conclusively showing the power of curiosity driven deep reinforcement learning.Please keep in mind this is a fast paced course for motivated and advanced students. There will be only a very brief review of the fundamental concepts of reinforcement learning and actor critic methods, and from there we will jump right into reading and implementing papers. The beauty of both the ICM and asynchronous methods is that these paradigms can be applied to nearly any other reinforcement learning algorithm. Both are highly adaptable and can be plugged in with little modification to algorithms like proximal policy optimization, soft actor critic, or deep Q learning.Students will learn how to:Implement deep reinforcement learning papersLeverage multi core CPUs with parallel processing in PythonCode the A3C algorithm from scratchCode the ICM from first principlesCode generalized advantage estimationModify the Open AI Gym Atari LibraryWrite extensible modular codeThis course is launching with the PyTorch implementation, with a Tensorflow 2 version coming. I'll see you on the inside.
