hide
Free keywords:
-
Abstract:
Cognitive flexibility, the ability to acquire, adapt, combine and recombine behaviors appropriate to everchanging
tasks, is a hallmark of intelligent behavior. In mammals, flexibility is likely to depend crucially on
mechanisms and representations within the prefrontal cortex (PFC), and on the PFC’s extensive connections
with structures such as the basal ganglia (BG) and hippocampus. To study flexibility, it is necessary at the
very least to present collections of related tasks; unfortunately, most experimental, and almost all computational,
approaches have hitherto focused on learning single underlying tasks, albeit with subtly changing
contingencies. In this work, we consider a foundational form of flexibility – the way that separate behavioral
components can be acquired through shaping and then combined to solve an overall task.
We study the 12-AX task, which was proposed and then modelled by O’Reilly et al[2] as a rich test bed for
analysing PFC-PFC and PFC-BG interactions. Subjects are presented sequentially with letters or the digits
’1’ or ’2’. If the most recent digit they have seen was a ’1’, they have to provide a non-default response only
to the sequence segment ‘AX’; if it was a ’2’, then they must react only to ‘BY’. Storing ’1’ or ’2’ is an outer
working memory loop defining a cognitive context; storing either ‘A’ or ‘B’ is an inner loop. O’Reilly et
al modeled a complex reinforcement-based learning process for this task, and showed that it out-performed
a standard architecture for learning to use working memory in tasks. However, in both cases, the networks
had to learn the full task monolithically, in one fell swoop. Instead, we considered the consequences of
shaping, by training individual subcomponents separately, and learning their combination. We performed
our shaping in an LSTM[1] network.
