PDF Publication Title:
Text from PDF Page: 026
3 This chapter studies how to develop policy optimization methods that lead to mono- tonically improving performance and make efficient use of data. As we argued in the Introduction, in order to optimize function approximators, we need to reduce the re- inforcement learning problem to a series of optimization problems. This reduction is nontrivial in reinforcement learning because the state distribution depends on the policy. This chapter shows that to update the policy, we should improve a certain surrogate objective as much as possible, while changing the policy as little as possible, where this change is measured as a KL divergence between action distributions. We show that by bounding the size of the policy update, we can bound the change in state distributions, guaranteeing policy improvement despite non-trivial step sizes. Following this theoretical analysis, we make a series of approximations to the theoretically- justified algorithm, yielding a practical algorithm that we call trust region policy opti- mization (TRPO). We describe two variants of this algorithm: first, the single-path method, which can be applied in the model-free setting; second, the vine method, which requires the system to be restored to particular states, which is typically only possible in simu- lation. These algorithms are scalable and can optimize nonlinear policies with tens of thousands of parameters, which have previously posed a major challenge for model-free policy search [DNP13]. In our experiments, we show that the same TRPO methods can learn complex policies for swimming, hopping, and walking, as well as playing Atari games directly from raw images. 18 TRUST REGION POLICY OPTIMIZATION 3.1 overviewPDF Image | OPTIMIZING EXPECTATIONS: FROM DEEP REINFORCEMENT LEARNING TO STOCHASTIC COMPUTATION GRAPHS
PDF Search Title:
OPTIMIZING EXPECTATIONS: FROM DEEP REINFORCEMENT LEARNING TO STOCHASTIC COMPUTATION GRAPHSOriginal File Name Searched:
thesis-optimizing-deep-learning.pdfDIY PDF Search: Google It | Yahoo | Bing
Cruise Ship Reviews | Luxury Resort | Jet | Yacht | and Travel Tech More Info
Cruising Review Topics and Articles More Info
Software based on Filemaker for the travel industry More Info
The Burgenstock Resort: Reviews on CruisingReview website... More Info
Resort Reviews: World Class resorts... More Info
The Riffelalp Resort: Reviews on CruisingReview website... More Info
| CONTACT TEL: 608-238-6001 Email: greg@cruisingreview.com | RSS | AMP |