Literature DB >> 29463751

Within- and across-trial dynamics of human EEG reveal cooperative interplay between reinforcement learning and working memory.

Anne G E Collins1,2, Michael J Frank3.   

Abstract

Learning from rewards and punishments is essential to survival and facilitates flexible human behavior. It is widely appreciated that multiple cognitive and reinforcement learning systems contribute to decision-making, but the nature of their interactions is elusive. Here, we leverage methods for extracting trial-by-trial indices of reinforcement learning (RL) and working memory (WM) in human electro-encephalography to reveal single-trial computations beyond that afforded by behavior alone. Neural dynamics confirmed that increases in neural expectation were predictive of reduced neural surprise in the following feedback period, supporting central tenets of RL models. Within- and cross-trial dynamics revealed a cooperative interplay between systems for learning, in which WM contributes expectations to guide RL, despite competition between systems during choice. Together, these results provide a deeper understanding of how multiple neural systems interact for learning and decision-making and facilitate analysis of their disruption in clinical populations.

Entities:  

Keywords:  EEG; computational model; dynamics; reinforcement learning; working memory

Mesh:

Year:  2018        PMID: 29463751      PMCID: PMC5877949          DOI: 10.1073/pnas.1720963115

Source DB:  PubMed          Journal:  Proc Natl Acad Sci U S A        ISSN: 0027-8424            Impact factor:   11.205


  25 in total

1.  Multiple model-based reinforcement learning.

Authors:  Kenji Doya; Kazuyuki Samejima; Ken-ichi Katagiri; Mitsuo Kawato
Journal:  Neural Comput       Date:  2002-06       Impact factor: 2.026

2.  Testing the reward prediction error hypothesis with an axiomatic model.

Authors:  Robb B Rutledge; Mark Dean; Andrew Caplin; Paul W Glimcher
Journal:  J Neurosci       Date:  2010-10-06       Impact factor: 6.167

3.  Informing cognitive abstractions through neuroimaging: the neural drift diffusion model.

Authors:  Brandon M Turner; Leendert van Maanen; Birte U Forstmann
Journal:  Psychol Rev       Date:  2015-04       Impact factor: 8.934

Review 4.  The cognitive neuroscience of working memory.

Authors:  Mark D'Esposito; Bradley R Postle
Journal:  Annu Rev Psychol       Date:  2014-09-19       Impact factor: 24.137

5.  Neural computations underlying arbitration between model-based and model-free learning.

Authors:  Sang Wan Lee; Shinsuke Shimojo; John P O'Doherty
Journal:  Neuron       Date:  2014-02-05       Impact factor: 17.173

6.  Working memory contributions to reinforcement learning impairments in schizophrenia.

Authors:  Anne G E Collins; Jaime K Brown; James M Gold; James A Waltz; Michael J Frank
Journal:  J Neurosci       Date:  2014-10-08       Impact factor: 6.167

7.  Interactions Among Working Memory, Reinforcement Learning, and Effort in Value-Based Choice: A New Paradigm and Selective Deficits in Schizophrenia.

Authors:  Anne G E Collins; Matthew A Albrecht; James A Waltz; James M Gold; Michael J Frank
Journal:  Biol Psychiatry       Date:  2017-05-31       Impact factor: 13.382

8.  Opponent actor learning (OpAL): modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive.

Authors:  Anne G E Collins; Michael J Frank
Journal:  Psychol Rev       Date:  2014-07       Impact factor: 8.934

9.  Model-based choices involve prospective neural activity.

Authors:  Bradley B Doll; Katherine D Duncan; Dylan A Simon; Daphna Shohamy; Nathaniel D Daw
Journal:  Nat Neurosci       Date:  2015-03-23       Impact factor: 24.884

10.  Modeling choice and reaction time during arbitrary visuomotor learning through the coordination of adaptive working memory and reinforcement learning.

Authors:  Guillaume Viejo; Mehdi Khamassi; Andrea Brovelli; Benoît Girard
Journal:  Front Behav Neurosci       Date:  2015-08-26       Impact factor: 3.558

View more
  21 in total

1.  The importance of agency in human reward processing.

Authors:  Cameron D Hassall; Greg Hajcak; Olave E Krigolson
Journal:  Cogn Affect Behav Neurosci       Date:  2019-12       Impact factor: 3.282

2.  A Control Theoretic Model of Adaptive Learning in Dynamic Environments.

Authors:  Harrison Ritz; Matthew R Nassar; Michael J Frank; Amitai Shenhav
Journal:  J Cogn Neurosci       Date:  2018-06-07       Impact factor: 3.225

3.  How the Level of Reward Awareness Changes the Computational and Electrophysiological Signatures of Reinforcement Learning.

Authors:  Camile M C Correa; Samuel Noorman; Jun Jiang; Stefano Palminteri; Michael X Cohen; Maël Lebreton; Simon van Gaal
Journal:  J Neurosci       Date:  2018-10-16       Impact factor: 6.167

4.  The Role of Executive Function in Shaping Reinforcement Learning.

Authors:  Milena Rmus; Samuel D McDougle; Anne G E Collins
Journal:  Curr Opin Behav Sci       Date:  2020-11-14

5.  What do Reinforcement Learning Models Measure? Interpreting Model Parameters in Cognition and Neuroscience.

Authors:  Maria K Eckstein; Linda Wilbrecht; Anne G E Collins
Journal:  Curr Opin Behav Sci       Date:  2021-07-03

Review 6.  Advances in modeling learning and decision-making in neuroscience.

Authors:  Anne G E Collins; Amitai Shenhav
Journal:  Neuropsychopharmacology       Date:  2021-08-27       Impact factor: 7.853

Review 7.  Personalizing neuromodulation.

Authors:  John D Medaglia; Brian Erickson; Jared Zimmerman; Apoorva Kelkar
Journal:  Int J Psychophysiol       Date:  2019-01-24       Impact factor: 2.997

8.  A new model of decision processing in instrumental learning tasks.

Authors:  Steven Miletić; Russell J Boag; Anne C Trutti; Niek Stevenson; Birte U Forstmann; Andrew Heathcote
Journal:  Elife       Date:  2021-01-27       Impact factor: 8.140

9.  Decision-making ability, psychopathology, and brain connectivity.

Authors:  Michael Moutoussis; Benjamín Garzón; Sharon Neufeld; Dominik R Bach; Francesco Rigoli; Ian Goodyer; Edward Bullmore; Marc Guitart-Masip; Raymond J Dolan
Journal:  Neuron       Date:  2021-05-20       Impact factor: 17.173

Review 10.  Modeling the influence of working memory, reinforcement, and action uncertainty on reaction time and choice during instrumental learning.

Authors:  Samuel D McDougle; Anne G E Collins
Journal:  Psychon Bull Rev       Date:  2021-02
View more

北京卡尤迪生物科技股份有限公司 © 2022-2023.