User Manual Privacy Policy Disclaimer Contact us
  Advanced SearchBrowse




Conference Paper

Exploration-Exploitation in a Contextual Multi-Armed Bandit Task

There are no MPG-Authors available
Fulltext (public)
There are no public fulltexts stored in PuRe
Supplementary Material (public)
There is no public supplementary material available

Schulz, E., Konstantinidis, E., & Speekenbrink, M. (2015). Exploration-Exploitation in a Contextual Multi-Armed Bandit Task. In N. Taatgen, M. van Vugt, J. Borst, & K. Mehlhorn (Eds.), 13th International Conference on Cognitive Modeling (ICCM 2015) (pp. 118-123).

Cite as: http://hdl.handle.net/21.11116/0000-0006-B447-F
We introduce the Contextual Multi-Armed Bandit task as a method to assess decision making in uncertain environments and test how participants behave in this task. Within an experimental paradigm named Mining in Space, participants see 4 different planets that are described by 3 different binary elements (the context) and then have to decide on which planet they want to mine (which arm to play). We find that participants adapt their decisions to the context well and can best be described by a Contextual Gaussian Process algorithm that probability matches according to expected outcomes. We conclude that humans are well-adapted to contextualized bandit problems even in potentially non-stationary environments through probability matching, a heuristic that used to be described as biased behavior. We argue that Contextual Bandit problems can provide further insight into how people make decisions in real world scenarios.