English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT

Released

Journal Article

An anatomical substrate of credit assignment in reinforcement learning

MPS-Authors
/persons/resource/persons125729

Kornfeld,  Jörgen
Research Group: Circuits of Birdsong / Kornfeld, MPI of Neurobiology, Max Planck Society;

/persons/resource/persons208338

Schubert,  Philipp J.
Department: Electrons-Photons-Neurons / Denk, MPI of Neurobiology, Max Planck Society;

/persons/resource/persons128986

Denk,  Winfried
Department: Electrons-Photons-Neurons / Denk, MPI of Neurobiology, Max Planck Society;

External Resource
No external resources are shared
Fulltext (restricted access)
There are currently no full texts shared for your IP range.
Fulltext (public)
There are no public fulltexts stored in PuRe
Supplementary Material (public)
There is no public supplementary material available
Citation

Kornfeld, J., Januszewski, M., Schubert, P. J., Jain, V., Denk, W., & Fee, M. S. (2024). An anatomical substrate of credit assignment in reinforcement learning. bioRxiv: the preprint server for biology. doi:10.1101/2020.02.18.954354.


Cite as: https://hdl.handle.net/21.11116/0000-0009-764C-E
Abstract
Learning turns experience into better decisions. A key problem in learning is credit assignment—knowing how to change parameters, such as synaptic weights deep within a neural network, in order to improve behavioral performance. Artificial intelligence owes its recent bloom largely to the error-backpropagation algorithm1, which estimates the contribution of every synapse to output errors and allows rapid weight adjustment. Biological systems, however, lack an obvious mechanism to backpropagate errors. Here we show, by combining high-throughput volume electron microscopy2 and automated connectomic analysis3–5, that the synaptic architecture of songbird basal ganglia supports local credit assignment using a variant of the node perturbation algorithm proposed in a model of songbird reinforcement learning6, 7. We find that key predictions of the model hold true: first, cortical axons that encode exploratory motor variability terminate predominantly on dendritic shafts of striatal spiny neurons, while cortical axons that encode song timing terminate almost exclusively on spines. Second, synapse pairs that share a presynaptic cortical timing axon and a postsynaptic spiny dendrite are substantially more similar in size than expected, indicating Hebbian plasticity8, 9. Combined with numerical simulations, these findings provide strong evidence for a biologically plausible credit assignment mechanism6.