Item

Abstract

Reversal learning is one of the most venerable paradigms for studying the acquisition, extinction,and reacquisition of knowledge in humans and other animals. It has been of particular value in askingquestions about the roles played by prefrontal structures such as the orbitofrontal cortex (OFC). Indeed,evidence from rats and monkeys suggests that these areas are involved in various forms of context-sensitiveinference about the contingencies linking cues and actions over time to the value and identity of predictedoutcomes. In order to explore these roles in depth, we fit data from a substantial behavioural neurosciencestudy in rodents who experienced blocks of free- and forced-choice instrumental learning trials with identityor value reversals at each block transition. We constructed two classes of models, fit their parametersusing a random effects treatment, tested their generative competence, and selected between them based on acomplexity-sensitive integrated Bayesian Information Criteria score. One class of ‘return’-based models wasbased on elaborations of a standard Q-learning algorithm, including parameters such as different learningrates or combination rules for forced- and fixed-choice trials, behavioural lapses, and eligibility traces. Theother novel class of ‘income’-based models exploited the weak notion of contingency over time advocatedby Walton et al (2010) in their analysis of the choices of monkeys with OFC lesions. We show that income-based and return-based models are both able to predict the behaviour well, and examine their performanceand implications for reinforcement learning. The outcome of this study sets the stage for the next phase ofthe research that will attempt to correlate the values of the parameters to neural recordings taken in the ratswhile performing the task.