Abstract
OBJECTIVE: It is widely established that reward prediction error signals can be observed in the human brain: When participants believe there is a given percentage chance of earning a reward and then receive it, areas such as the ventral striatum and the ventromedial prefrontal cortex show BOLD activations reflecting the reward prediction error, relative to the reward expectation. This study aims to determine where and how these reward expectations are updated. Specifically, it asks whether BOLD signals in the ventral striatum and ventromedial PFC, known to encode reward prediction errors (Schultz, Dayan, and Montague, 1997; Caplin et al., 2010; Rutledge et al., 2010; Niv et al., 2012), also reflect updates to a participant’s reward expectations in the absence of actual rewards. METHODS: In an fMRI experiment, participants (N=14) faced binary lotteries represented as partially occluded pie charts. The pie chart’s composition on each trial was revealed only gradually to participants, so that they could sequentially update their belief about the probability of winning the prize. At a random time during this gradual resolution (and hence at different levels of ambiguity), we elicited participants’ valuation of the current pie chart-lottery using an incentive-compatible procedure yielding their probability equivalent. RESULTS: In line with theoretical predictions, participants’ valuations in absence of ambiguity reveal (1) that their subjective beliefs closely track the lottery’s winning probability, and (2) their degree of risk aversion, with a coefficient of relative risk aversion of 0.93. In the presence of ambiguity, participants’ revealed beliefs are slightly more pessimistic, reflecting their degree of ambiguity aversion. Relating the neuroimaging data to behaviorally revealed belief updates, we then examine the extent to which BOLD activations in reward-prediction areas track changing reward expectations. CONCLUSION: This study examines whether areas encoding reward prediction errors in response to received rewards also represent updates to reward expectations, mirroring punishment expectation errors seen in similar locations (Seymour et al., 2003). Promising a measurable trace of belief updates, this approach offers novel empirical constraints on models of hitherto unobservable evolving beliefs, and provides a rigorous way to test neuroeconomic theories that make joint predictions about choice behavior and neural data.
