Abstract
The observation that the phasic activity of dopamine neurons resembles closely an appetitivetemporal difference prediction error associated with conditioned sensory cues does not exhaust either thecharacteristics of the signal or its putative role in influencing behaviour. In particular, experiments usingoperant paradigms and fast timescale measurements of the concentration of dopamine in one of its keytargets, the nucleus accumbens, have shown transient increases just prior to the emission of actions thatdeliver rewards or the avoidance of punishment. This signal might play a causal role in driving behaviour, for instance through an effect on basal ganglia dynamics. It might also be a consequence of locally gated releaseat the level of the striatum, without any change in phasic activity of the dopamine neurons, as has beenargued for the case of dopamine ramps. However, we study a third possibility that it reflects the outcomeof an internal decision to respond. This conceives of the systems controlling dopamine as monitoring theinternal state of the subject, and responding when this state implies that the appetitive outcome consequenton the action is impending. We consider the implications of this view for the informational relationshipbetween the predictive critic and a temporally-sophisticated actor.
