Item

Abstract

Human reinforcement learning (RL) is characterized by different challenges. Exploration has been studied extensively in multi-armed bandits, while planning has been investigated in multi-step decision tasks. More recent work added structure >to bandits to study generalization. However, most studies focus on a single aspect of learning, making it hard to compare and integrate results. Here, we propose a generative model for constructing Correlated Trees, which provide a unified and scalable method for studying exploration, planning, and generalization in a single task. In an online experiment, we found that, when provided, people use structure to generalize and perform uncertainty-directed exploration, with structure helping more in larger environments. In environments without structure, exploration becomes more random and more planning is needed. All behavioral effects are captured in a single model with recoverable parameters. In conclusion, our results connect past research on human RL in one framework using Correlated Trees.