English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
 
 
DownloadE-Mail
  Relative Entropy Inverse Reinforcement Learning

Boularias, A., Kober, J., & Peters, J. (2011). Relative Entropy Inverse Reinforcement Learning.

Item is

Basic

show hide
Genre: Conference Paper
Alternative Title : JMLR: W&CP

Files

show Files

Locators

show

Creators

show
hide
 Creators:
Boularias, A.1, Author           
Kober, J.1, Author           
Peters, J.1, Author           
Affiliations:
1Dept. Empirical Inference, Max Planck Institute for Intelligent Systems, Max Planck Society, ou_1497647              

Content

show
hide
Free keywords: MPI für Intelligente Systeme; Abt. Schölkopf;
 Abstract: We consider the problem of imitation learning where the examples, demonstrated by an expert, cover only a small part of a large state space. Inverse Reinforcement Learning (IRL) provides an efficient tool for generalizing the demonstration, based on the assumption that the expert is optimally acting in a Markov Decision Process (MDP). Most of the past work on IRL requires that a (near)-optimal policy can be computed for different reward functions. However, this requirement can hardly be satisfied in systems with a large, or continuous, state space. In this paper, we propose a model-free IRL algorithm, where the relative entropy between the empirical distribution of the state-action trajectories under a uniform policy and their distribution under the learned policy is minimized by stochastic gradient descent. We compare this new approach to well-known IRL algorithms using approximate MDP models. Empirical results on simulated car racing, gridworld and ball-in-a-cup problems show that our approach is able to learn good policies from a small number of demonstrations.

Details

show
hide
Language(s):
 Dates: 2011
 Publication Status: Issued
 Pages: -
 Publishing info: -
 Table of Contents: -
 Rev. Type: -
 Degree: -

Event

show
hide
Title: Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS)
Place of Event: Fort Lauderdale, FL, USA
Start-/End Date: 2011-04-11 - 2011-04-13

Legal Case

show

Project information

show

Source 1

show
hide
Title: Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS)
Source Genre: Issue
 Creator(s):
Gordon, G., Editor
Dunson, D., Editor
M., Dudík., Editor
Affiliations:
-
Publ. Info: -
Pages: 7 Volume / Issue: - Sequence Number: - Start / End Page: 182 - 189 Identifier: -

Source 2

show
hide
Title: JMLR: Workshop and Conference Proceedings
  Alternative Title : JMLR: W&CP
Source Genre: Series
 Creator(s):
Affiliations:
Publ. Info: -
Pages: 7 Volume / Issue: 15 Sequence Number: - Start / End Page: - Identifier: -