Overcoming catastrophic forgetting in neural networks

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

Related collections

Most cited references 14

Record: found
Abstract: found
Article: not found

Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory.

James McClelland, Bruce L. McNaughton, Randall C O'Reilly (1995)

Damage to the hippocampal system disrupts recent memory but leaves remote memory intact. The account presented here suggests that memories are first stored via synaptic changes in the hippocampal system, that these changes support reinstatement of recent memories in the neocortex, that neocortical synapses change a little on each reinstatement, and that remote memory is based on accumulated neocortical changes. Models that learn via changes to connections help explain this organization. These models discover the structure in ensembles of items if learning of each item is gradual and interleaved with learning about other items. This suggests that the neocortex learns slowly to discover the structure in ensembles of experiences. The hippocampal system permits rapid learning of new items without disrupting this structure, and reinstatement of new memories interleaves them with others to integrate them into structured neocortical memory systems.

0 comments Cited 513 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: not found
Article: not found

A Practical Bayesian Framework for Backpropagation Networks

David MacKay (1992)

0 comments Cited 476 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Catastrophic forgetting in connectionist networks.

R. French (1999)

All natural cognitive systems, and, in particular, our own, gradually forget previously learned information. Plausible models of human cognition should therefore exhibit similar patterns of gradual forgetting of old information as new information is acquired. Only rarely does new learning in natural cognitive systems completely disrupt or erase previously learned information; that is, natural cognitive systems do not, in general, forget 'catastrophically'. Unfortunately, though, catastrophic forgetting does occur under certain circumstances in distributed connectionist networks. The very features that give these networks their remarkable abilities to generalize, to function in the presence of degraded input, and so on, are found to be the root cause of catastrophic forgetting. The challenge in this field is to discover how to keep the advantages of distributed connectionist networks while avoiding the problem of catastrophic forgetting. In this article the causes, consequences and numerous solutions to the problem of catastrophic forgetting in neural networks are examined. The review will consider how the brain might have overcome this problem and will also explore the consequences of this solution for distributed connectionist networks.

0 comments Cited 318 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Journal

Title: Proceedings of the National Academy of Sciences

Abbreviated Title: Proc Natl Acad Sci USA

Publisher: Proceedings of the National Academy of Sciences

ISSN (Print): 0027-8424

ISSN (Electronic): 1091-6490

Publication date Created: March 28 2017

Publication date (Print): March 28 2017

Volume: 114

Issue: 13

Pages: 3521-3526

Article

DOI: 10.1073/pnas.1611835114

PMC ID: 5380101

PubMed ID: 28292907

SO-VID: 810a4aa5-8455-4b69-b9b4-07f8868626d2

History

Data availability:

Comments

Comment on this article

scite_

Smart Citations

Citing PublicationsSupportingMentioningContrasting

View Citations

See how this article has been cited at scite.ai

scite shows how a scientific paper has been cited by providing the context of the citation, a classification describing whether it supports, mentions, or contrasts the cited claim, and a label indicating in which section the citation was made.

Overcoming catastrophic forgetting in neural networks

Read this article at

Abstract

Related collections

ScienceOpen Research

Most cited references 14

Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory.

A Practical Bayesian Framework for Backpropagation Networks

Catastrophic forgetting in connectionist networks.

Author and article information

Journal

Article

History

Comments

Comment on this article

Similar content 4,865

Cited by 504

Most referenced authors 309