English
 
User Manual Privacy Policy Disclaimer Contact us
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
  Predictive Representations for Policy Gradient in POMDPs

Boularias, A., & Chaib-draa, B. (2009). Predictive Representations for Policy Gradient in POMDPs. In A. Danyluk, L. Bottou, & M. Littman (Eds.), ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning (pp. 65-72). New York, NY, USA: ACM Press.

Item is

Basic

show hide
Item Permalink: http://hdl.handle.net/11858/00-001M-0000-0013-C4A1-C Version Permalink: http://hdl.handle.net/21.11116/0000-0002-F91F-4
Genre: Conference Paper

Files

show Files

Locators

show
hide
Description:
-

Creators

show
hide
 Creators:
Boularias, A1, Author              
Chaib-draa, B, Author
Affiliations:
1External Organizations, ou_persistent22              

Content

show
hide
Free keywords: -
 Abstract: We consider the problem of estimating the policy gradient in Partially Observable Markov Decision Processes (POMDPs) with a special class of policies that are based on Predictive State Representations (PSRs). We compare PSR policies to Finite-State Controllers (FSCs), which are considered as a standard model for policy gradient methods in POMDPs. We present a general Actor- Critic algorithm for learning both FSCs and PSR policies. The critic part computes a value function that has as variables the parameters of the policy. These latter parameters are gradually updated to maximize the value function. We show that the value function is polynomial for both FSCs and PSR policies, with a potentially smaller degree in the case of PSR policies. Therefore, the value function of a PSR policy can have less local optima than the equivalent FSC, and consequently, the gradient algorithm is more likely to converge to a global optimal solution.

Details

show
hide
Language(s):
 Dates: 2009-06
 Publication Status: Published in print
 Pages: -
 Publishing info: -
 Table of Contents: -
 Rev. Type: -
 Identifiers: DOI: 10.1145/1553374.1553383
BibTex Citekey: 6827
 Degree: -

Event

show
hide
Title: 26th International Conference on Machine Learning (ICML 2009)
Place of Event: Montreal, Canada
Start-/End Date: 2009-06-14 - 2009-06-18

Legal Case

show

Project information

show

Source 1

show
hide
Title: ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning
Source Genre: Proceedings
 Creator(s):
Danyluk, A, Editor
Bottou, L, Editor
Littman, M, Editor
Affiliations:
-
Publ. Info: New York, NY, USA : ACM Press
Pages: - Volume / Issue: - Sequence Number: - Start / End Page: 65 - 72 Identifier: ISBN: 978-1-60558-516-1

Source 2

show
hide
Title: ACM International Conference Proceeding Series
Source Genre: Series
 Creator(s):
Affiliations:
Publ. Info: -
Pages: - Volume / Issue: 382 Sequence Number: - Start / End Page: - Identifier: -