English
 
User Manual Privacy Policy Disclaimer Contact us
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
  Relative Entropy Inverse Reinforcement Learning

Boularias, A., Kober, J., & Peters, J. (2011). Relative Entropy Inverse Reinforcement Learning. In G. Gordon, D. Dunson, & M. Dudik (Eds.), JMLR Workshop and Conference Proceedings (pp. 182-189). Cambridge, MA, USA: MIT Press.

Item is

Basic

show hide
Item Permalink: http://hdl.handle.net/11858/00-001M-0000-0013-BC40-3 Version Permalink: http://hdl.handle.net/21.11116/0000-0002-0D69-B
Genre: Conference Paper

Files

show Files

Locators

show
hide
Description:
-

Creators

show
hide
 Creators:
Boularias, A1, Author              
Kober, J1, Author              
Peters, J1, Author              
Affiliations:
1Dept. Empirical Inference, Max Planck Institute for Intelligent Systems, Max Planck Society, ou_1497647              

Content

show
hide
Free keywords: -
 Abstract: We consider the problem of imitation learning where the examples, demonstrated by an expert, cover only a small part of a large state space. Inverse Reinforcement Learning (IRL) provides an efficient tool for generalizing the demonstration, based on the assumption that the expert is optimally acting in a Markov Decision Process (MDP). Most of the past work on IRL requires that a (near)-optimal policy can be computed for different reward functions. However, this requirement can hardly be satisfied in systems with a large, or continuous, state space. In this paper, we propose a model-free IRL algorithm, where the relative entropy between the empirical distribution of the state-action trajectories under a uniform policy and their distribution under the learned policy is minimized by stochastic gradient descent. We compare this new approach to well-known IRL algorithms using approximate MDP models. Empirical results on simulated car racing, gridworld and ball-in-a-cup problems show that our approach is able to learn good policies from a small number of demonstrations.

Details

show
hide
Language(s):
 Dates: 2011-04
 Publication Status: Published in print
 Pages: -
 Publishing info: -
 Table of Contents: -
 Rev. Type: -
 Identifiers: URI: http://jmlr.csail.mit.edu/proceedings/papers/v15/boularias11a.html
BibTex Citekey: BoulariasKP2011
 Degree: -

Event

show
hide
Title: Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2011)
Place of Event: Ft. Lauderdale, FL, USA
Start-/End Date: 2011-04-11 - 2011-04-13

Legal Case

show

Project information

show

Source 1

show
hide
Title: JMLR Workshop and Conference Proceedings
Source Genre: Proceedings
 Creator(s):
Gordon, G, Editor
Dunson, D, Editor
Dudik, M, Editor
Affiliations:
-
Publ. Info: Cambridge, MA, USA : MIT Press
Pages: - Volume / Issue: 15 Sequence Number: - Start / End Page: 182 - 189 Identifier: -