Item

Abstract

To what extent do human reward learning and decision-making rely on the ability to represent and generate richly structured relationships between options? We provide evidence that structure learning and the principle of compositionality play crucial roles in human reinforcement learning. In a new multi-armed bandit paradigm, termed the compositionally-structured multi-armed bandit task, we found evidence that participants are able to learn representations of different latent reward structures and combine them to make correct generalizations about options in novel contexts. Moreover, we found substantial evidence that participants transferred knowledge of simpler reward structures, to make informed, compositional generalizations about rewards in complex contexts. We also provide a computational model which is able to generalize and compose knowledge of complex reward structures using a grammar over structures and show how such compositional inductive biases can be learned by meta-reinforcement learning agents.