Literature DB >> 33444700

Why and how the brain weights contributions from a mixture of experts.

John P O'Doherty1, Sang Wan Lee2, Reza Tadayonnejad3, Jeff Cockburn4, Kyo Iigaya4, Caroline J Charpentier4.   

Abstract

It has long been suggested that human behavior reflects the contributions of multiple systems that cooperate or compete for behavioral control. Here we propose that the brain acts as a "Mixture of Experts" in which different expert systems propose strategies for action. It will be argued that the brain determines which experts should control behavior at any one moment in time by keeping track of the reliability of the predictions within each system, and by allocating control over behavior in a manner that depends on the relative reliabilities across experts. fMRI and neurostimulation studies suggest a specific contribution of the anterior prefrontal cortex in this process. Further, such a mechanism also takes into consideration the complexity of the expert, favoring simpler over more cognitively complex experts. Results from the study of different expert systems in both experiential and social learning domains hint at the possibility that this reliability-based control mechanism is domain general, exerting control over many different expert systems simultaneously in order to produce sophisticated behavior.
Copyright © 2021 Elsevier Ltd. All rights reserved.

Entities:  

Keywords:  Basal ganglia; Cognitive control; Decision-making; Prefrontal cortex; Theoretical neuroscience

Mesh:

Year:  2021        PMID: 33444700      PMCID: PMC8040830          DOI: 10.1016/j.neubiorev.2020.10.022

Source DB:  PubMed          Journal:  Neurosci Biobehav Rev        ISSN: 0149-7634            Impact factor:   8.989


  40 in total

1.  Mixture of experts classification using a hierarchical mixture model.

Authors:  Michalis K Titsias; Aristidis Likas
Journal:  Neural Comput       Date:  2002-09       Impact factor: 2.026

2.  Separate neural systems value immediate and delayed monetary rewards.

Authors:  Samuel M McClure; David I Laibson; George Loewenstein; Jonathan D Cohen
Journal:  Science       Date:  2004-10-15       Impact factor: 47.728

3.  Separate encoding of model-based and model-free valuations in the human brain.

Authors:  Ulrik R Beierholm; Cedric Anen; Steven Quartz; Peter Bossaerts
Journal:  Neuroimage       Date:  2011-07-02       Impact factor: 6.556

Review 4.  The expected value of control: an integrative theory of anterior cingulate cortex function.

Authors:  Amitai Shenhav; Matthew M Botvinick; Jonathan D Cohen
Journal:  Neuron       Date:  2013-07-24       Impact factor: 17.173

5.  Cost-Benefit Arbitration Between Multiple Reinforcement-Learning Systems.

Authors:  Wouter Kool; Samuel J Gershman; Fiery A Cushman
Journal:  Psychol Sci       Date:  2017-07-21

6.  Adaptive Mixtures of Local Experts.

Authors:  Robert A Jacobs; Michael I Jordan; Steven J Nowlan; Geoffrey E Hinton
Journal:  Neural Comput       Date:  1991       Impact factor: 2.026

Review 7.  Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration.

Authors:  Jonathan D Cohen; Samuel M McClure; Angela J Yu
Journal:  Philos Trans R Soc Lond B Biol Sci       Date:  2007-05-29       Impact factor: 6.237

8.  Differential effects of two ways of devaluing the unconditioned stimulus after Pavlovian appetitive conditioning.

Authors:  P C Holland; J J Straub
Journal:  J Exp Psychol Anim Behav Process       Date:  1979-01

9.  Risk, unexpected uncertainty, and estimation uncertainty: Bayesian learning in unstable settings.

Authors:  Elise Payzan-LeNestour; Peter Bossaerts
Journal:  PLoS Comput Biol       Date:  2011-01-20       Impact factor: 4.475

10.  Mapping value based planning and extensively trained choice in the human brain.

Authors:  Klaus Wunderlich; Peter Dayan; Raymond J Dolan
Journal:  Nat Neurosci       Date:  2012-03-11       Impact factor: 24.884

View more
  2 in total

1.  Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T.

Authors:  Jaron T Colas; Neil M Dundon; Raphael T Gerraty; Natalie M Saragosa-Harris; Karol P Szymula; Koranis Tanwisuth; J Michael Tyszka; Camilla van Geen; Harang Ju; Arthur W Toga; Joshua I Gold; Dani S Bassett; Catherine A Hartley; Daphna Shohamy; Scott T Grafton; John P O'Doherty
Journal:  Hum Brain Mapp       Date:  2022-07-21       Impact factor: 5.399

2.  Meta-control of social learning strategies.

Authors:  Anil Yaman; Nicolas Bredeche; Onur Çaylak; Joel Z Leibo; Sang Wan Lee
Journal:  PLoS Comput Biol       Date:  2022-02-28       Impact factor: 4.475

  2 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.