Abstract
Complexities in the relationship between value and choice are two central sources of anomalies. First, the different systems can disagree about their values. Actions involve such things as picking a stimulus or pressing a button. The environment specifies a set of rules governing the transitions between states depending on the action chosen. The trouble is that a tree typically grows exponentially with the number of layers considered, making this extremely difficult. Model-based and model-free controls are ways of doing this, which differ in the information about the environment they use and the computations they perform. The model-free system would learn the utility of pressing the lever but would not have the informational wherewithal to realize that this utility had changed when the cheese had been poisoned. Pavlovian control is also based on predictions of affectively important outcomes such as rewards and punishments. However, rather than determining the choices that would lead to the acquisition or avoidance of these outcomes, it expresses a set of hard-wired preparatory and consummatory choices.
