Literature DB >> 24255115

A multiplicative reinforcement learning model capturing learning dynamics and interindividual variability in mice.

Brice Bathellier1, Sui Poh Tee, Christina Hrovat, Simon Rumpel.   

Abstract

Both in humans and in animals, different individuals may learn the same task with strikingly different speeds; however, the sources of this variability remain elusive. In standard learning models, interindividual variability is often explained by variations of the learning rate, a parameter indicating how much synapses are updated on each learning event. Here, we theoretically show that the initial connectivity between the neurons involved in learning a task is also a strong determinant of how quickly the task is learned, provided that connections are updated in a multiplicative manner. To experimentally test this idea, we trained mice to perform an auditory Go/NoGo discrimination task followed by a reversal to compare learning speed when starting from naive or already trained synaptic connections. All mice learned the initial task, but often displayed sigmoid-like learning curves, with a variable delay period followed by a steep increase in performance, as often observed in operant conditioning. For all mice, learning was much faster in the subsequent reversal training. An accurate fit of all learning curves could be obtained with a reinforcement learning model endowed with a multiplicative learning rule, but not with an additive rule. Surprisingly, the multiplicative model could explain a large fraction of the interindividual variability by variations in the initial synaptic weights. Altogether, these results demonstrate the power of multiplicative learning rules to account for the full dynamics of biological learning and suggest an important role of initial wiring in the brain for predispositions to different tasks.

Entities:  

Keywords:  behavior; cue competition; memory; savings

Mesh:

Year:  2013        PMID: 24255115      PMCID: PMC3856837          DOI: 10.1073/pnas.1312125110

Source DB:  PubMed          Journal:  Proc Natl Acad Sci U S A        ISSN: 0027-8424            Impact factor:   11.205


  27 in total

1.  Reinforcement learning in populations of spiking neurons.

Authors:  Robert Urbanczik; Walter Senn
Journal:  Nat Neurosci       Date:  2009-02-15       Impact factor: 24.884

2.  Stress, genotype and norepinephrine in the prediction of mouse behavior using reinforcement learning.

Authors:  Gediminas Luksys; Wulfram Gerstner; Carmen Sandi
Journal:  Nat Neurosci       Date:  2009-08-16       Impact factor: 24.884

3.  Synapse specificity of long-term potentiation breaks down at short distances.

Authors:  F Engert; T Bonhoeffer
Journal:  Nature       Date:  1997-07-17       Impact factor: 49.962

Review 4.  A neural substrate of prediction and reward.

Authors:  W Schultz; P Dayan; P R Montague
Journal:  Science       Date:  1997-03-14       Impact factor: 47.728

5.  Exploring a latent cause theory of classical conditioning.

Authors:  Samuel J Gershman; Yael Niv
Journal:  Learn Behav       Date:  2012-09       Impact factor: 1.986

6.  Discrete neocortical dynamics predict behavioral categorization of sounds.

Authors:  Brice Bathellier; Lyubov Ushakova; Simon Rumpel
Journal:  Neuron       Date:  2012-10-17       Impact factor: 17.173

Review 7.  Behavioral profiles of inbred strains on novel olfactory, spatial and emotional tests for reference memory in mice.

Authors:  A Holmes; C C Wrenn; A P Harris; K E Thayer; J N Crawley
Journal:  Genes Brain Behav       Date:  2002-01       Impact factor: 3.449

8.  Visual discrimination and reversal learning in the aged monkey (Macaca mulatta).

Authors:  P R Rapp
Journal:  Behav Neurosci       Date:  1990-12       Impact factor: 1.912

9.  Correlated connectivity and the distribution of firing rates in the neocortex.

Authors:  Alexei A Koulakov; Tomás Hromádka; Anthony M Zador
Journal:  J Neurosci       Date:  2009-03-25       Impact factor: 6.167

10.  Increased axonal bouton dynamics in the aging mouse cortex.

Authors:  Federico W Grillo; Sen Song; Leonor M Teles-Grilo Ruivo; Lieven Huang; Ge Gao; Graham W Knott; Bohumil Maco; Valentina Ferretti; Dawn Thompson; Graham E Little; Vincenzo De Paola
Journal:  Proc Natl Acad Sci U S A       Date:  2013-03-29       Impact factor: 11.205

View more
  13 in total

1.  Rethinking dopamine as generalized prediction error.

Authors:  Matthew P H Gardner; Geoffrey Schoenbaum; Samuel J Gershman
Journal:  Proc Biol Sci       Date:  2018-11-21       Impact factor: 5.349

2.  Temporal chunking as a mechanism for unsupervised learning of task-sets.

Authors:  Flora Bouchacourt; Stefano Palminteri; Etienne Koechlin; Srdjan Ostojic
Journal:  Elife       Date:  2020-03-09       Impact factor: 8.140

3.  Unique features of stimulus-based probabilistic reversal learning.

Authors:  Carl Harris; Claudia Aguirre; Saisriya Kolli; Kanak Das; Alicia Izquierdo; Alireza Soltani
Journal:  Behav Neurosci       Date:  2021-08       Impact factor: 2.154

4.  Introducing Clicker Training as a Cognitive Enrichment for Laboratory Mice.

Authors:  Charlotte Leidinger; Felix Herrmann; Christa Thöne-Reineke; Nadine Baumgart; Jan Baumgart
Journal:  J Vis Exp       Date:  2017-03-06       Impact factor: 1.355

5.  Temporal asymmetries in auditory coding and perception reflect multi-layered nonlinearities.

Authors:  Thomas Deneux; Alexandre Kempf; Aurélie Daret; Emmanuel Ponsot; Brice Bathellier
Journal:  Nat Commun       Date:  2016-09-01       Impact factor: 14.919

6.  Reversal Learning in Humans and Gerbils: Dynamic Control Network Facilitates Learning.

Authors:  Christian Jarvers; Tobias Brosch; André Brechmann; Marie L Woldeit; Andreas L Schulz; Frank W Ohl; Marcel Lommerzheim; Heiko Neumann
Journal:  Front Neurosci       Date:  2016-11-17       Impact factor: 4.677

7.  Improving Accuracy and Temporal Resolution of Learning Curve Estimation for within- and across-Session Analysis.

Authors:  Matthias Deliano; Karsten Tabelow; Reinhard König; Jörg Polzehl
Journal:  PLoS One       Date:  2016-06-15       Impact factor: 3.240

8.  Learning-related population dynamics in the auditory thalamus.

Authors:  Ariel Gilad; Ido Maor; Adi Mizrahi
Journal:  Elife       Date:  2020-07-08       Impact factor: 8.140

9.  Striatal low-threshold spiking interneurons locally gate dopamine.

Authors:  Elizabeth N Holly; M Felicia Davatolhagh; Rodrigo A España; Marc V Fuccillo
Journal:  Curr Biol       Date:  2021-07-23       Impact factor: 10.900

10.  Structural and Functional Brain Remodeling during Pregnancy with Diffusion Tensor MRI and Resting-State Functional MRI.

Authors:  Russell W Chan; Leon C Ho; Iris Y Zhou; Patrick P Gao; Kevin C Chan; Ed X Wu
Journal:  PLoS One       Date:  2015-12-10       Impact factor: 3.240

View more

北京卡尤迪生物科技股份有限公司 © 2022-2023.