Literature DB >> 29205151

Towards deep learning with segregated dendrites.

Jordan Guerguiev1,2, Timothy P Lillicrap3, Blake A Richards1,2,4.   

Abstract

Deep learning has led to significant advances in artificial intelligence, in part, by adopting strategies motivated by neurophysiology. However, it is unclear whether deep learning could occur in the real brain. Here, we show that a deep learning algorithm that utilizes multi-compartment neurons might help us to understand how the neocortex optimizes cost functions. Like neocortical pyramidal neurons, neurons in our model receive sensory information and higher-order feedback in electrotonically segregated compartments. Thanks to this segregation, neurons in different layers of the network can coordinate synaptic weight updates. As a result, the network learns to categorize images better than a single layer network. Furthermore, we show that our algorithm takes advantage of multilayer architectures to identify useful higher-order representations-the hallmark of deep learning. This work demonstrates that deep learning can be achieved using segregated dendritic compartments, which may help to explain the morphology of neocortical pyramidal neurons.

Entities:  

Keywords:  computational biology; credit assignment; deep learning; dendritic morphology; feedback alignment; neocortex; neuroscience; none; systems biology; target propagation

Mesh:

Year:  2017        PMID: 29205151      PMCID: PMC5716677          DOI: 10.7554/eLife.22901

Source DB:  PubMed          Journal:  Elife        ISSN: 2050-084X            Impact factor:   8.140


  55 in total

Review 1.  LTP and LTD: an embarrassment of riches.

Authors:  Robert C Malenka; Mark F Bear
Journal:  Neuron       Date:  2004-09-30       Impact factor: 17.173

2.  Mastering the game of Go with deep neural networks and tree search.

Authors:  David Silver; Aja Huang; Chris J Maddison; Arthur Guez; Laurent Sifre; George van den Driessche; Julian Schrittwieser; Ioannis Antonoglou; Veda Panneershelvam; Marc Lanctot; Sander Dieleman; Dominik Grewe; John Nham; Nal Kalchbrenner; Ilya Sutskever; Timothy Lillicrap; Madeleine Leach; Koray Kavukcuoglu; Thore Graepel; Demis Hassabis
Journal:  Nature       Date:  2016-01-28       Impact factor: 49.962

Review 3.  Cortical region interactions and the functional role of apical dendrites.

Authors:  M W Spratling
Journal:  Behav Cogn Neurosci Rev       Date:  2002-09

4.  Reinforcement learning in populations of spiking neurons.

Authors:  Robert Urbanczik; Walter Senn
Journal:  Nat Neurosci       Date:  2009-02-15       Impact factor: 24.884

5.  Learning by the dendritic prediction of somatic spiking.

Authors:  Robert Urbanczik; Walter Senn
Journal:  Neuron       Date:  2014-02-05       Impact factor: 17.173

6.  Rules and mechanisms for efficient two-stage learning in neural circuits.

Authors:  Tiberiu Teşileanu; Bence Ölveczky; Vijay Balasubramanian
Journal:  Elife       Date:  2017-04-04       Impact factor: 8.140

7.  Layer-specific modulation of neocortical dendritic inhibition during active wakefulness.

Authors:  William Muñoz; Robin Tremblay; Daniel Levenstein; Bernardo Rudy
Journal:  Science       Date:  2017-03-03       Impact factor: 47.728

8.  Depression-biased reverse plasticity rule is required for stable learning at top-down connections.

Authors:  Kendra S Burbank; Gabriel Kreiman
Journal:  PLoS Comput Biol       Date:  2012-03-01       Impact factor: 4.475

9.  Context- and Output Layer-Dependent Long-Term Ensemble Plasticity in a Sensory Circuit.

Authors:  Yoshiyuki Yamada; Khaleel Bhaukaurally; Tamás J Madarász; Alexandre Pouget; Ivan Rodriguez; Alan Carleton
Journal:  Neuron       Date:  2017-02-23       Impact factor: 17.173

10.  Deep Neural Networks as a Computational Model for Human Shape Sensitivity.

Authors:  Jonas Kubilius; Stefania Bracci; Hans P Op de Beeck
Journal:  PLoS Comput Biol       Date:  2016-04-28       Impact factor: 4.475

View more
  66 in total

1.  Widespread and Highly Correlated Somato-dendritic Activity in Cortical Layer 5 Neurons.

Authors:  Lou Beaulieu-Laroche; Enrique H S Toloza; Norma J Brown; Mark T Harnett
Journal:  Neuron       Date:  2019-06-06       Impact factor: 17.173

2.  Can the Brain Do Backpropagation? -Exact Implementation of Backpropagation in Predictive Coding Networks.

Authors:  Yuhang Song; Thomas Lukasiewicz; Zhenghua Xu; Rafal Bogacz
Journal:  Adv Neural Inf Process Syst       Date:  2020

Review 3.  A mechanism for learning with sleep spindles.

Authors:  Adrien Peyrache; Julie Seibt
Journal:  Philos Trans R Soc Lond B Biol Sci       Date:  2020-04-06       Impact factor: 6.237

4.  Dendritic Spikes Expand the Range of Well Tolerated Population Noise Structures.

Authors:  Alon Poleg-Polsky
Journal:  J Neurosci       Date:  2019-09-26       Impact factor: 6.167

Review 5.  Backpropagation and the brain.

Authors:  Timothy P Lillicrap; Adam Santoro; Luke Marris; Colin J Akerman; Geoffrey Hinton
Journal:  Nat Rev Neurosci       Date:  2020-04-17       Impact factor: 34.870

6.  Continual Learning in a Multi-Layer Network of an Electric Fish.

Authors:  Salomon Z Muller; Abigail N Zadina; L F Abbott; Nathaniel B Sawtell
Journal:  Cell       Date:  2019-11-14       Impact factor: 41.582

Review 7.  Beyond the feedforward sweep: feedback computations in the visual cortex.

Authors:  Gabriel Kreiman; Thomas Serre
Journal:  Ann N Y Acad Sci       Date:  2020-02-28       Impact factor: 5.691

8.  Enhanced Dendritic Compartmentalization in Human Cortical Neurons.

Authors:  Lou Beaulieu-Laroche; Enrique H S Toloza; Marie-Sophie van der Goes; Mathieu Lafourcade; Derrick Barnagian; Ziv M Williams; Emad N Eskandar; Matthew P Frosch; Sydney S Cash; Mark T Harnett
Journal:  Cell       Date:  2018-10-18       Impact factor: 41.582

9.  Single-cell transcriptomic evidence for dense intracortical neuropeptide networks.

Authors:  Stephen J Smith; Uygar Sümbül; Lucas T Graybuck; Forrest Collman; Sharmishtaa Seshamani; Rohan Gala; Olga Gliko; Leila Elabbady; Jeremy A Miller; Trygve E Bakken; Jean Rossier; Zizhen Yao; Ed Lein; Hongkui Zeng; Bosiljka Tasic; Michael Hawrylycz
Journal:  Elife       Date:  2019-11-11       Impact factor: 8.140

Review 10.  A deep learning framework for neuroscience.

Authors:  Blake A Richards; Timothy P Lillicrap; Denis Therien; Konrad P Kording; Philippe Beaudoin; Yoshua Bengio; Rafal Bogacz; Amelia Christensen; Claudia Clopath; Rui Ponte Costa; Archy de Berker; Surya Ganguli; Colleen J Gillon; Danijar Hafner; Adam Kepecs; Nikolaus Kriegeskorte; Peter Latham; Grace W Lindsay; Kenneth D Miller; Richard Naud; Christopher C Pack; Panayiota Poirazi; Pieter Roelfsema; João Sacramento; Andrew Saxe; Benjamin Scellier; Anna C Schapiro; Walter Senn; Greg Wayne; Daniel Yamins; Friedemann Zenke; Joel Zylberberg
Journal:  Nat Neurosci       Date:  2019-10-28       Impact factor: 24.884

View more

北京卡尤迪生物科技股份有限公司 © 2022-2023.