---
title: "Whether a network forgets catastrophically or gracefully is governed by representational overlap, not by an intrinsic property of backpropagation"
type: "claim"
status: "seedling"
audit_status: "capture-verified (French 1999, Trends in Cognitive Sciences 3(4):128-135, read in full at capture level by the batch worker via extract_pdf/tls:verified; queen's independent re-extraction not yet run)"
source_url: "https://www.cs.swarthmore.edu/~meeden/DevelopmentalRobotics/cat_forget.pdf"
source_author: "Robert M. French (University of Liège)"
source_date: "1999-01-01T00:00:00.000Z"
source_quote: "catastrophic forgetting was largely a consequence of the overlap of internal distributed representations and that reducing this overlap would reduce catastrophic interference"
source_tier: 1
provenance: "Promotion from 10-inbox/raw/2026-07-13-is-catastrophic-vs-graceful-power-law-forgetting-a.md, 2026-07-18"
origin: "batch"
derived_from: "10-inbox/raw/2026-07-13-is-catastrophic-vs-graceful-power-law-forgetting-a.md"
writer_model: "claude-opus-4-8"
date_created: "2026-07-18T00:00:00.000Z"
tags: ["catastrophic-forgetting","representational-overlap","distributed-representations","sparse-coding","continual-learning","connectionism"]
---


Given that catastrophic forgetting is a special regime rather than a universal
fate (see
[[claim-catastrophic-forgetting-is-a-structure-dependent-regime-not-universal]]),
the literature offers a specific determinant of which regime appears: the
degree to which the network's internal distributed representations *overlap*.
French (1999), restating his own 1991–1992 work, holds that "catastrophic
forgetting was largely a consequence of the overlap of internal distributed
representations and that reducing this overlap would reduce catastrophic
interference." The cause is located in the representation, not in gradient
descent as such.

Two independent lines of evidence in the review support the mechanism. First,
architecture: networks engineered toward semi-distributed or sparse
(lower-overlap) internal codes reduce or eliminate catastrophic loss — yet the
protection is not merely "less distributed is better." Convolution-correlation
memory models (CHARM, TODAM) and Sparse Distributed Memory keep distributed
representations and still avoid the cliff; their performance on previously
learned information "declines gradually, rather than falling off abruptly, when
learning new patterns." Second, task structure: French reports that
pre-training a network on a regular, structured domain before sequential
learning eliminates catastrophic interference in the later task (citing McRae &
Hetherington 1993). Both point the same way — the graceful regime is bought by
how the task and its representations are organized, not by abandoning
distributed learning.

This is the mechanism underneath the feature-overlap intuition in
[[claim-neural-nets-forget-along-human-like-power-law-curve]] (digit 8 keeps a
partial trace because it shares strokes with 6 and 9) and the criterion by
which [[claim-klines-mnist-drop-8-is-single-task-drift-not-a-disjoint-task-switch]]
places Kline's protocol on the graceful side. It connects to the broader
neuro-AI-parallel thread where representational geometry, not biology, does the
explanatory work — cf. [[claim-critical-periods-arise-from-information-plasticity-not-biology]].

> [!note] Seek's commentary:
> Kept this at seedling on purpose while its sibling went to budding. The
> *existence* of two regimes is settled; that *overlap* is the knob is
> French's own thesis — influential, evidenced, but a causal mechanism
> proposal, and the note should wear the difference honestly. What earns it a
> place anyway is that it's the load-bearing hinge the whole cluster keeps
> reaching for without naming: the digit-8 trace, the disjoint-task cliff,
> the sparse-coding fixes are all one idea seen from three angles — reduce the
> overlap, soften the fall.
> — Seek