Exploration-Exploitation in a Contextual Multi-Armed Bandit Task

Schulz, E; Konstantinidis, E; Speekenbrink, M

Item

ITEM ACTIONSEXPORT

Add to Basket

Local TagsStatisticsRelease HistoryDetailsSummary

Released

Conference Paper

Exploration-Exploitation in a Contextual Multi-Armed Bandit Task

MPS-Authors

There are no MPG-Authors available

External Ressource

http://www.cognitive-modeling.com/proceedings/ICCM2015_proceedings.pdf
(Publisher version)

Fulltext (public)

There are no public fulltexts stored in PuRe

Supplementary Material (public)

There is no public supplementary material available

Citation

Schulz, E., Konstantinidis, E., & Speekenbrink, M. (2015). Exploration-Exploitation in a Contextual Multi-Armed Bandit Task. In N. Taatgen, M. van Vugt, J. Borst, & K. Mehlhorn (Eds.), 13th International Conference on Cognitive Modeling (ICCM 2015) (pp. 118-123).

Cite as: http://hdl.handle.net/21.11116/0000-0006-B447-F

Abstract

We introduce the Contextual Multi-Armed Bandit task as a method to assess decision making in uncertain environments and test how participants behave in this task. Within an experimental paradigm named Mining in Space, participants see 4 different planets that are described by 3 different binary elements (the context) and then have to decide on which planet they want to mine (which arm to play). We find that participants adapt their decisions to the context well and can best be described by a Contextual Gaussian Process algorithm that probability matches according to expected outcomes. We conclude that humans are well-adapted to contextualized bandit problems even in potentially non-stationary environments through probability matching, a heuristic that used to be described as biased behavior. We argue that Contextual Bandit problems can provide further insight into how people make decisions in real world scenarios.