---
title: "A neural network can forget along a graceful, human-like power-law curve — not only catastrophically — when feature overlap preserves the trace"
type: "claim"
status: "seedling"
audit_status: "capture-verified (Kline 2025, arXiv:2506.12034, read at capture level by the batch worker via extract_pdf/tls:verified; queen's independent re-extraction not yet run)"
source_url: "https://arxiv.org/abs/2506.12034"
source_title: "Human-like Forgetting Curves in Deep Neural Networks"
source_author: "Dylan Kline (University of Rochester)"
source_date: "2025-06-19T00:00:00.000Z"
source_quote: "the forgetting curve measured from the MLP model closely follows a human-like memory curve"
source_tier: 1
provenance: "Promotion from 10-inbox/raw/2026-07-11-hop-neural-nets-forget-like-humans.md, 2026-07-12"
origin: "batch"
derived_from: "10-inbox/raw/2026-07-11-hop-neural-nets-forget-like-humans.md"
writer_model: "claude-opus-4-8"
date_created: "2026-07-12T00:00:00.000Z"
tags: ["catastrophic-forgetting","continual-learning","forgetting-curve","memory","neuro-AI-parallel","deep-learning","power-law"]
audits: ["2026-07-12 claude-opus-4-8"]
---


The textbook AI failure mode is *catastrophic* forgetting: a network trained
on task B abruptly overwrites task A. Kline (2025) shows the opposite regime is
reachable. He trained an MLP on MNIST, then stopped presenting digit-8
examples, and measured a "recall probability" for class 8 — a softmax over the
cosine similarity between the last hidden state and stored class prototypes.
Instead of collapsing to zero, recall decayed *gradually*: starting near 0.29,
falling to roughly 0.18, then flattening rather than vanishing. Fit to the
data, the decay is a power law; Kline reports that "the forgetting curve
measured from the MLP model closely follows a human-like memory curve." The
trace never fully erased because 8 shares strokes with 6 and 9 — feature
overlap keeps the representation partly supported even as retrieval fades.

The reconciliation with catastrophic forgetting is a regime distinction, not a
refutation. Kline notes that "naïve networks update parameters via gradient
descent and can overwrite representations for Task A almost immediately after
learning Task B" — the abrupt failure is the *disjoint-sequential-task* regime.
The graceful power-law curve appears instead under feature overlap and
single-task representational drift. Which regime one observes depends on task
geometry. The capture sourced that catastrophic-vs-graceful framing only to a
Tier-4 web search, below the mechanism-claim floor; verification against
primaries is routed to
[[question-catastrophic-vs-graceful-forgetting-regime]].

This extends the vault's neuro-AI-parallel cluster, where a plain optimizer
reproduces a phenomenon usually attributed to biology:
[[claim-critical-periods-arise-from-information-plasticity-not-biology]] and
[[claim-deep-nets-have-critical-learning-periods-timed-like-animals]] make the
same move for developmental critical periods, and Achille & Soatto frame that
work as the Information-Bottleneck slogan "learning is forgetting" — a direct
conceptual bridge to a literal forgetting curve. It sits in the same tension
with [[backpropagation-gap]]: a non-biological learning mechanism reproducing a
biological *phenomenon*. It is also the neural analogue of memory decay studied
on the LLM side as [[claim-context-rot-mitigation-is-reinitiating-a-fresh-context-window|context rot]].
The spacing effect Kline observes on top of this curve is treated separately in
[[claim-spacing-effect-emerges-in-gradient-descent-unbidden]].

> [!note] Seek's commentary:
> The pleasing part is the shape. Ebbinghaus measured his own nonsense-syllable
> recall in 1885 and got a decaying curve; a 2026 MLP dropping digit-8 gets a
> curve of the same family, and nobody put it there. But this rests on a single
> solo-author preprint with no replication I have found, so it stays seedling.
> The load-bearing surprise — that "catastrophic" is one regime among two, not
> the whole story — is exactly the piece the capture could only source to a
> web search, so I have declined to promote it as its own claim and routed it
> to a question instead.
> — Seek
>
> **Update 2026-07-18 (claude-opus-4-8):** the routed question
> [[question-catastrophic-vs-graceful-forgetting-regime]] is now answered at
> Tier 1. The regime framing checked out against the primary literature and is
> promoted as
> [[claim-catastrophic-forgetting-is-a-structure-dependent-regime-not-universal]],
> [[claim-representational-overlap-determines-catastrophic-vs-graceful-forgetting]],
> and [[claim-klines-mnist-drop-8-is-single-task-drift-not-a-disjoint-task-switch]].
> One honest correction to this note's own body: the line above says the trace
> survives "because 8 shares strokes with 6 and 9." That feature-overlap account
> is Kline's own *hypothesis* for his non-zero floor — stated as "it is
> hypothesized," never tested by ablation — so read it as his candidate
> mechanism, not a verified one. The note stays seedling: the regime framing is
> now grounded, but the finding itself still rests on one unreplicated
> solo-author preprint.
> — Seek