Bootstrapping Apprenticeship Learning

Boularias, A; Chaib-Draa, B

Local TagsRelease HistoryDetailsSummary

Bootstrapping Apprenticeship Learning

Boularias, A., & Chaib-Draa, B. (2011). Bootstrapping Apprenticeship Learning. Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, 289-297.

Item is Released

show all hide all

Basic

show hide

Item Permalink: https://hdl.handle.net/11858/00-001M-0000-0013-BB72-C Version Permalink: https://hdl.handle.net/21.11116/0000-0002-0AAA-4

Genre: Conference Paper

Files

show Files

Locators

show

hide

Locator:
https://papers.nips.cc/paper/4160-bootstrapping-apprenticeship-learning.pdf (Publisher version) Open Access status unknown

Description:
-

OA-Status:

Creators

show

hide

Creators:
Boularias, A^{1, 2}, Author
Chaib-Draa, B, Author

Affiliations:
1Department Empirical Inference, Max Planck Institute for Biological Cybernetics, Max Planck Society, ou_1497795
2Max Planck Institute for Biological Cybernetics, Max Planck Society, Spemannstrasse 38, 72076 Tübingen, DE, ou_1497794

Content

show

hide

Free keywords: -

Abstract: We consider the problem of apprenticeship learning where the examples, demonstrated by an expert, cover only a small part of a large state space. Inverse Reinforcement Learning (IRL) provides an efficient tool for generalizing the demonstration, based on the assumption that the expert is maximizing a utility function that is a linear combination of state-action features. Most IRL algorithms use a simple Monte Carlo estimation to approximate the expected feature counts under the expert's policy. In this paper, we show that the quality of the learned policies is highly sensitive to the error in estimating the feature counts. To reduce this error, we introduce a novel approach for bootstrapping the demonstration by assuming that: (i), the expert is (near-)optimal, and (ii), the dynamics of the system is known. Empirical results on gridworlds and car racing problems show that our approach is able to learn good policies from a small number of demonstrations.

Details

show

hide

Language(s):

Dates: Date issued: 2011-06

Publication Status: Issued

Pages: -

Publishing info: -

Table of Contents: -

Rev. Type: -

Identifiers: BibTex Citekey: 6826

Degree: -

Event

show

hide

Title: Twenty-Fourth Annual Conference on Neural Information Processing Systems (NIPS 2010)

Place of Event: Vancouver, BC, Canada

Start-/End Date: 2010-12-06 - 2010-12-11

Legal Case

show

Project information

show

Source 1

show

hide

Title: Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010

Source Genre: Journal

Creator(s):
Lafferty, J, Editor
Williams, CKI, Editor
Shawe-Taylor, J, Editor
Zemel, RS, Editor
Culotta, A, Editor

Affiliations:
-

Publ. Info: Red Hook, NY, USA : Curran

Pages: - Volume / Issue: - Sequence Number: - Start / End Page: 289 - 297 Identifier: ISBN: 978-1-617-82380-0