Relative Entropy Inverse Reinforcement Learning

Boularias, A; Kober, J; Peters, J

Local TagsRelease HistoryDetailsSummary

Relative Entropy Inverse Reinforcement Learning

Boularias, A., Kober, J., & Peters, J. (2011). Relative Entropy Inverse Reinforcement Learning. In G. Gordon, D. Dunson, & M. Dudik (Eds.), JMLR Workshop and Conference Proceedings (pp. 182-189). Cambridge, MA, USA: MIT Press.

Item is Released

show all hide all

Basic

show hide

Item Permalink: https://hdl.handle.net/11858/00-001M-0000-0013-BC40-3 Version Permalink: https://hdl.handle.net/21.11116/0000-0002-0D69-B

Genre: Conference Paper

Files

show Files

Locators

show

hide

Locator:
http://proceedings.mlr.press/v15/boularias11a/boularias11a.pdf (Publisher version) Open Access status unknown

Description:
-

OA-Status:

Creators

show

hide

Creators:
Boularias, A¹, Author
Kober, J¹, Author
Peters, J¹, Author

Affiliations:
1Dept. Empirical Inference, Max Planck Institute for Intelligent Systems, Max Planck Society, ou_1497647

Content

show

hide

Free keywords: -

Abstract: We consider the problem of imitation learning where the examples, demonstrated by an expert, cover only a small part of a large state space. Inverse Reinforcement Learning (IRL) provides an efficient tool for generalizing the demonstration, based on the assumption that the expert is optimally acting in a Markov Decision Process (MDP). Most of the past work on IRL requires that a (near)-optimal policy can be computed for different reward functions. However, this requirement can hardly be satisfied in systems with a large, or continuous, state space. In this paper, we propose a model-free IRL algorithm, where the relative entropy between the empirical distribution of the state-action trajectories under a uniform policy and their distribution under the learned policy is minimized by stochastic gradient descent. We compare this new approach to well-known IRL algorithms using approximate MDP models. Empirical results on simulated car racing, gridworld and ball-in-a-cup problems show that our approach is able to learn good policies from a small number of demonstrations.

Details

show

hide

Language(s):

Dates: Date issued: 2011-04

Publication Status: Issued

Pages: -

Publishing info: -

Table of Contents: -

Rev. Type: -

Identifiers: URI: http://jmlr.csail.mit.edu/proceedings/papers/v15/boularias11a.html
BibTex Citekey: BoulariasKP2011

Degree: -

Event

show

hide

Title: Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2011)

Place of Event: Ft. Lauderdale, FL, USA

Start-/End Date: 2011-04-11 - 2011-04-13

Legal Case

show

Project information

show

Source 1

show

hide

Title: JMLR Workshop and Conference Proceedings

Source Genre: Proceedings

Creator(s):
Gordon, G, Editor
Dunson, D, Editor
Dudik, M, Editor

Affiliations:
-

Publ. Info: Cambridge, MA, USA : MIT Press

Pages: - Volume / Issue: 15 Sequence Number: - Start / End Page: 182 - 189 Identifier: -