hide
Free keywords:
-
Abstract:
A cornerstone of current theorizing about dopamine’s computational role is the idea that the phasic activity of
dopamine neurons represents a temporal difference (TD) prediction error. While substantial evidence supports
this mapping, recent reports of ramp-like increases in accumbens dopamine concentration when animals are
about to act, or are about to reach rewards, pose an important challenge to this TD hypothesis. This is because,
under a TD account, the implied activity underlying such ramps is persistently predictable by preceding stimuli
and as such, should be largely predicted away. Nevertheless, we suggest that dopamine ramps are largely
reconcilable with standard theory, and offer three, non-mutually exclusive accounts of these phenomena. Firstly,
at a computational level, we propose, along with Berke, that ramping may arise as a form of state prediction. In
average-reward analyses, average reward rate has been suggested to (i) be a comparison point for the TD error,
(ii) control instrumental vigour, and (iii) be represented by tonic dopamine levels. We determine that a suitable
counterpart in the discounted case is a state-dependent quantity which, carried by dopamine, would manifest as
a ’quasi-tonic’ signal with similar properties to those observed experimentally. Secondly, at an algorithmic level,
we suggest that ramping observed just before execution of an instrumental action for reward may be caused by
uncertainty about when the action will occur. TD errors occasioned by resolution of such uncertainty, such as may
occur just prior to action execution, may explain these signals. Thirdly, at an implementational level, we observe
that ramping may arise if dopamine has a direct influence on the timecourse of choice, such as setting the gain
of an accumulative decision-making process. Even assuming purely noise-driven fluctuations in dopamine levels,
resulting correlated dynamics entail an average dopamine signal which appears to ramp up to the time of decision.
