|
所在平台: Udemy |
课程主页: https://www.udemy.com/course/actor-critic-methods-from-paper-to-code-with-pytorch/
课程评论：没有评论
课程名称：现代强化学习：演员-评论家代理 课程概述：在这门高级深度强化学习课程中，您将学习如何在挑战性的环境（如OpenAI Gym）中实现策略梯度、演员-评论家、深度确定性策略梯度（DDPG）、双延迟深度确定性策略梯度（TD3）和软演员-评论家（SAC）算法。课程重点关注处理连续动作空间的环境，这对希望在机器人控制领域使用深度强化学习进行研究的学生尤为重要。您将学习自主阅读深度强化学习研究论文并从头实现这些算法，掌握快速实现高级研究论文中算法的可重复框架。掌握本课程的内容将使您在人工智能工程师的能力上有质的飞跃，使您在依赖他人解释复杂概念的学生中脱颖而出。 课程开始时，我们将快速回顾强化学习的核心主题，包括贝尔曼方程、马尔可夫决策过程、蒙特卡罗预测、蒙特卡罗控制、时间差分预测TD(0)以及基于Q学习的时间差分控制。随后，我们将开始编程第一个代理：一个玩黑杰克的人工智能。接着，我们将教授一个代理使用Q学习平衡杠杆。掌握基础知识后，课程进入快节奏的政策梯度方法介绍，我们将使用REINFORCE算法让人工智能在moon lander环境中着陆。 接下来，我们将实现一步演员-评论家算法，以再次战胜lunar lander。基础知识掌握后，我们将进入更难的项目：实现深度强化学习研究论文。我们首先实现深度确定性策略梯度（DDPG），该算法用于教导机器人在各种连续控制任务中表现优秀。DDPG结合了深度Q学习和传统演员-评论家方法的进展，在连续动作空间中实现最先进的结果。 然后，我们将实现一项最先进的人工智能算法：双延迟深度确定性策略梯度（TD3）。此算法在连续机器人控制任务中设定了新的性能基准。TD3是基于DDPG算法，但解决了一些导致其性能差的近似问题。最后，我们将实现软演员-评论家算法（SAC）。SAC从一个完全不同的角度看待深度强化学习：通过考虑熵最大化而非分数最大化作为可行目标，从而增加代理的探索能力，并在多个重要的OpenAI Gym环境中实现世界级的表现。 通过本课程，您将能回答以下演员-评论家方法的基本问题： - 为何我们在深度Q学习成功时仍需使用演员-评论家方法？ - 深度Q学习中的进展可以应用于强化学习的其他领域吗？ - 如何用确定性策略解决探索-利用困境？ - 如何解决演员-评论家方法中的高估偏差？ - 我们如何处理深度神经网络固有的近似误差？ 本课程适合高度积极和进阶的学生。要成功完成，您必须具备以下主题的先前课程基础： - 大学水平的微积分 - 强化学习 - 深度学习 课程节奏快速，主题处于深度强化学习研究的前沿，但最终的收益是您将学会如何迅速阅读研究论文并将其转化为功能性代码，不再依赖不可靠的博客文章。
In this advanced course on deep reinforcement learning, you will learn how to implement policy gradient, actor critic, deep deterministic policy gradient (DDPG), twin delayed deep deterministic policy gradient (TD3), and soft actor critic (SAC) algorithms in a variety of challenging environments from the Open AI gym. There will be a strong focus on dealing with environments with continuous action spaces, which is of particular interest for those looking to do research into robotic control with deep reinforcement learning.Rather than being a course that spoon feeds the student, here you are going to learn to read deep reinforcement learning research papers on your own, and implement them from scratch. You will learn a repeatable framework for quickly implementing the algorithms in advanced research papers. Mastering the content in this course will be a quantum leap in your capabilities as an artificial intelligence engineer, and will put you in a league of your own among students who are reliant on others to break down complex ideas for them.Fear not, if it's been a while since your last reinforcement learning course, we will begin with a briskly paced review of core topics.The course begins with a practical review of the fundamentals of reinforcement learning, including topics such as:The Bellman EquationMarkov Decision ProcessesMonte Carlo PredictionMonte Carlo ControlTemporal Difference Prediction TD(0)Temporal Difference Control with Q LearningAnd moves straight into coding up our first agent: a blackjack playing artificial intelligence. From there we will progress to teaching an agent to balance the cart pole using Q learning. After mastering the fundamentals, the pace quickens, and we move straight into an introduction to policy gradient methods. We cover the REINFORCE algorithm, and use it to teach an artificial intelligence to land on the moon in the lunar lander environment from the Open AI gym. Next we progress to coding up the one step actor critic algorithm, to again beat the lunar lander.With the fundamentals out of the way, we move on to our harder projects: implementing deep reinforcement learning research papers. We will start with Deep Deterministic Policy Gradients (DDPG), which is an algorithm for teaching robots to excel at a variety of continuous control tasks. DDPG combines many of the advances of Deep Q Learning with traditional actor critic methods to achieve state of the art results in environments with continuous action spaces.Next, we implement a state of the art artificial intelligence algorithm: Twin Delayed Deep Deterministic Policy Gradients (TD3). This algorithm sets a new benchmark for performance in continuous robotic control tasks, and we will demonstrate world class performance in the Bipedal Walker environment from the Open AI gym. TD3 is based on the DDPG algorithm, but addresses a number of approximation issues that result in poor performance in DDPG and other actor critic algorithms.Finally, we will implement the soft actor critic algorithm (SAC). SAC approaches deep reinforcement learning from a totally different angle: by considering entropy maximization, rather than score maximization, as a viable objective. This results in increased exploration by our agent, and world class performance in a number of important Open AI Gym environments.By the end of the course, you will know the answers to the following fundamental questions in Actor-Critic methods:Why should we bother with actor critic methods when deep Q learning is so successful?Can the advances in deep Q learning be used in other fields of reinforcement learning?How can we solve the explore-exploit dilemma with a deterministic policy?How do we get and deal with overestimation bias in actor-critic methods?How do we deal with the inherent approximation errors in deep neural networks?This course is for the highly motivated and advanced student. To succeed, you must have prior course work in all the following topics:College level calculusReinforcement learningDeep learningThe pace of the course is brisk and the topics are at the cutting edge of deep reinforcement learning research, but the payoff is that you will come out knowing how to read research papers and turn them into functional code as quickly as possible. You'll never have to rely on dodgy medium blog posts again.
