English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
 
 
DownloadE-Mail
 Local TagsRelease HistoryDetailsSummary
  Predictive Representations for Policy Gradient in POMDPs

Boularias, A., & Chaib-draa, B. (2009). Predictive Representations for Policy Gradient in POMDPs. In A. Danyluk, L. Bottou, & M. Littman (Eds.), ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning (pp. 65-72). New York, NY, USA: ACM Press.

Item is

 

show all hide all

Basic

show hide
Item Permalink: https://hdl.handle.net/11858/00-001M-0000-0013-C4A1-C Version Permalink: https://hdl.handle.net/21.11116/0000-0002-F91F-4
Genre: Conference Paper

Files

show Files

Locators

show
hide
Locator:
https://dl.acm.org/citation.cfm?doid=1553374.1553383 (Publisher version)
Description:
-
OA-Status:

Creators

show
hide
 Creators:
Boularias, A1, Author           
Chaib-draa, B, Author
Affiliations:
1External Organizations, ou_persistent22              

Content

show
hide
Free keywords: -
 Abstract: We consider the problem of estimating
the policy gradient in Partially Observable
Markov Decision Processes (POMDPs) with
a special class of policies that are based
on Predictive State Representations (PSRs).
We compare PSR policies to Finite-State
Controllers (FSCs), which are considered as a
standard model for policy gradient methods
in POMDPs. We present a general Actor-
Critic algorithm for learning both FSCs and
PSR policies. The critic part computes a
value function that has as variables the parameters
of the policy. These latter parameters
are gradually updated to maximize the value
function. We show that the value function
is polynomial for both FSCs and PSR policies,
with a potentially smaller degree in the
case of PSR policies. Therefore, the value
function of a PSR policy can have less local
optima than the equivalent FSC, and consequently,
the gradient algorithm is more likely
to converge to a global optimal solution.

Details

show
hide
Language(s):
 Dates: 2009-06
 Publication Status: Issued
 Pages: -
 Publishing info: -
 Table of Contents: -
 Rev. Type: -
 Identifiers: DOI: 10.1145/1553374.1553383
BibTex Citekey: 6827
 Degree: -

Event

show
hide
Title: 26th International Conference on Machine Learning (ICML 2009)
Place of Event: Montreal, Canada
Start-/End Date: 2009-06-14 - 2009-06-18

Legal Case

show

Project information

show

Source 1

show
hide
Title: ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning
Source Genre: Proceedings
 Creator(s):
Danyluk, A, Editor
Bottou, L, Editor
Littman, M, Editor
Affiliations:
-
Publ. Info: New York, NY, USA : ACM Press
Pages: - Volume / Issue: - Sequence Number: - Start / End Page: 65 - 72 Identifier: ISBN: 978-1-60558-516-1

Source 2

show
hide
Title: ACM International Conference Proceeding Series
Source Genre: Series
 Creator(s):
Affiliations:
Publ. Info: -
Pages: - Volume / Issue: 382 Sequence Number: - Start / End Page: - Identifier: -