๐Ÿ“–
Local Policy Search in a Convex Space and Conservative Policy Iteration as Boosted Policy Search
Bruno Scherrer, Matthieu Geist

Local Policy Search in a Convex Space and Conservative Policy Iteration as Boosted Policy Search

Author / Organization:
Science & Mathematics Published 2014 Verification Pending

Overview & Synopsis

Local Policy Search is a popular reinforcement learning approach for handling large state spaces. Formally, it searches locally in a parameterized policy space in order to maximize the associated value function averaged over some pre-defined distribution. The best one can hope in general from such an approach is to get a local optimum of this criterion. The first contribution of this article is the following surprising result: if the policy space is convex, any (approximate) local optimum enjoys a global performance guarantee. Unfortunately, the convexity assumption is strong: it is not satisfied by commonly used parameterizations and designing a parameterization that induces this property seems hard. A natural so-lution to alleviate this issue consists in deriving an algorithm that solves the local policy search problem using a boosting approach (constrained to the convex hull of the policy space). The resulting algorithm turns out to be a slight generalization of conservative policy iteration; thus, our second contribution is to highlight an original connection between local policy search and approximate dynamic pro-gramming.

Indexed from repository: HAL Open Science (hal.science)
Loading document from verified source...
Keyboard Shortcuts: ← / → for pages • + / - for zoom • 0 to fit width Powered by Mozilla PDF.js v3.11

Document Information

Document Title Local Policy Search in a Convex Space and Conservative Policy Iteration as Boosted Policy Search
Author / Organization Bruno Scherrer, Matthieu Geist
Subject & Discipline Multidisciplinary Academic Open Research
Categories Science & Mathematics
Publication Year 2014
License / Rights Open Access / Creative Commons
Resource Verification Listed PDF (Verification Pending)
Original Host Website HAL Open Science (hal.science)
File Format application/pdf (Portable Document Format)
Language EN
Catalog ID DOC-255309
Original Source Link Visit Source Page on hal.science »