|
所在平台: Coursera |
课程主页: https://www.coursera.org/learn/complete-reinforcement-learning-system
课程评论：没有评论
课程名称：完整的强化学习系统 (Capstone) 概述：在本课程中，学员将整合第一、二和第三门课程的知识，实施一个完整的强化学习（RL）解决方案。这个Capstone项目将帮助学员了解问题的表述、算法选择、参数选择和表示设计等各个组件如何协同配合，形成一个完整的解决方案，并在现实世界中成功部署RL时做出适当的选择。学员需要实现一个环境以模拟所选择的问题，并设计一个具有神经网络功能逼近的控制代理。此外，还将进行学习系统的科学研究，以提高评估RL代理稳健性的能力。要在现实世界中使用RL，关键在于：(a) 将问题适当地正式化为马尔可夫决策过程（MDP），(b) 选择合适的算法，(c) 识别实施中对性能影响较大的选择，以及(d) 验证算法的预期行为。这个Capstone对计划利用RL解决实际问题的任何人都非常有价值。学员需完成前3门课程或具备同等知识才能顺利完成。 学习目标：到课程结束时，学员将能够完成一个RL问题的解决方案，从问题表述、算法的选择与实施，到对解决方案有效性的实证研究。 课程大纲： 1. 欢迎来到最终Capstone课程：介绍强化学习专项的最终课程。 2. 里程碑1：将词问题形式化为MDP：阅读问题描述，并将其转化为MDP，完成环境的骨架代码。 3. 里程碑2：选择正确的算法：从三种算法中选择，学习环境的策略，并讨论每种算法在该环境中的适用性。 4. 里程碑3：识别关键性能参数：识别影响代理性能的关键参数，以便后续更深入分析。 5. 里程碑4：实现代理：使用期望Sarsa或Q-learning与RMSProp和神经网络来实现代理，并验证代理的正确性。 6. 里程碑5：提交参数研究：选择要研究的参数，并运行复合参数的脚本，以评估代理性能，同时观察学习成果。 这门课程为希望真正解决问题的学习者提供了实用的指导和实践机会。
Name:Welcome to the Final Capstone Course!
Description:Welcome to the final capstone course of the Reinforcement Learning Specialization!!
Name:Milestone 1: Formalize Word Problem as MDP
Description:This week you will read a description of a problem, and translate it into an MDP. You will complete skeleton code for this environment, to obtain a complete MDP for use in this capstone project.
Name:Milestone 2: Choosing The Right Algorithm
Description:This week you will select from three algorithms, to learn a policy for the environment. You will reflect on and discuss the appropriateness of each algorithm for this environment.
Name:Milestone 3: Identify Key Performance Parameters
Description:This week you will identify key parameters that affect the performance of your agent. The goal is to understand the space of options, to later enable you to choose which parameter you will investigate in-depth for your agent.
Name:Milestone 4: Implement Your Agent
Description:This week, you will implement your agent using Expected Sarsa or Q-learning with RMSProp and Neural Networks. To use NNs, you will have to use a more careful stepsize selection strategy, which is why you will use RMSProp. You will also verify the correctness of your agent.
Name:Milestone 5: Submit Your Parameter Study!
Description:This week you will identify a parameter to study, for your agent. Once you select the parameter to study, we will provide you with a range of values and specific values for other parameters. You will write a script to run your agent and environment on the set of parameters, to determine performance across these parameters. You will gain insight into the impact of parameters on agent performance. You will also get to visualize the agents that you learn. Your parameter study will consist of an array of values that we will check for correctness.
In this final course, you will put together your knowledge from Courses 1, 2 and 3 to implement a complete RL solution to a problem. This capstone will let you see how each component---problem formulation, algorithm selection, parameter selection and representation design---fits together into a complete solution, and how to make appropriate choices when deploying RL in the real world. This project will require you to implement both the environment to stimulate your problem, and a control agent with Neural Network function approximation. In addition, you will conduct a scientific study of your learning system to develop your ability to assess the robustness of RL agents. To use RL in the real world, it is critical to (a) appropriately formalize the problem as an MDP, (b) select appropriate algorithms, (c ) identify what choices in your implementation will have large impacts on performance and (d) validate the expected behaviour of your algorithms. This capstone is valuable for anyone who is planning on using RL to solve real problems. To be successful in this course, you will need to have completed Courses 1, 2, and 3 of this Specialization or the equivalent. By the end of this course, you will be able to: Complete an RL solution to a problem, starting from problem formulation, appropriate algorithm selection and implementation and empirical study into the effectiveness of the solution.
