---
id: "20260711-1323-hop-neural-nets-forget-like-humans"
title: "Neural networks can forget along a human-like power-law curve — and the 1885 spacing effect re-emerges inside them"
type: "capture"
origin: "hop-batch"
writer_model: "claude-opus-4-8"
status: "promoted"
promoted_to: ["30-notes/claim-neural-nets-forget-along-human-like-power-law-curve.md","30-notes/claim-spacing-effect-emerges-in-gradient-descent-unbidden.md"]
not_promoted: ["Catastrophic-vs-graceful forgetting as a task-geometry regime distinction — the capture's own flag says this framing rests only on a Tier-4 web search, below the mechanism-claim sourcing floor. Folded as caveat into claim-neural-nets-forget-along-human-like-power-law-curve and routed to 50-questions/question-catastrophic-vs-graceful-forgetting-regime.md rather than promoted as a standalone (would seat a load-bearing mechanism claim on a weak source).","Ebbinghaus curve as a live 2026 engineering target (YourMemory, FOREVER, MSSR, ACM 'virtual Ebbinghaus days') — Hop-1 WebSearch findings; product/framework names surfaced without direct URL verification (Tier 3–4) and overlapping the existing agent-memory cluster (claim-agent-memory-field-shifted-storage-to-experience-2026). The capture itself listed 'virtual Ebbinghaus days' as a saved hook not followed. Left in inbox for a future dedicated, sourced pass."]
routed_questions: ["50-questions/question-catastrophic-vs-graceful-forgetting-regime.md"]
date_created: "2026-07-11T00:00:00.000Z"
hop_chain: ["SEED: 30-notes/claim-query-failure-clustering-as-gap-signal.md (Medhat et al. — 'asking about a topic weeks after it was taught' as a behavioral gap signal)","seed -> forgetting curve as a temporal memory signal, and whether it applies to AI memory (WebSearch; bridge to context-rot confirmed live in the literature, max_cosine 0.775 via vault_bridge)","forgetting-curve-in-AI landscape -> Kline, 'Human-like Forgetting Curves in Deep Neural Networks' (zoom-in, surprising claim, max_cosine 0.707)","Kline paper -> catastrophic vs graceful (power-law) forgetting: which regime is real? (zoom-out, field debate, max_cosine 0.73 on the Bjork branch — abandoned as duplicate-drift into agent-memory cluster)"]
novelty_max_cosine: 0.734
tags: ["catastrophic-forgetting","continual-learning","forgetting-curve","spaced-repetition","cognitive-science","neuro-AI-parallel","memory"]
source_title: "Human-like Forgetting Curves in Deep Neural Networks"
source_author: "Dylan Kline (University of Rochester)"
source_url: "https://arxiv.org/abs/2506.12034"
source_date: "2025-06-19T00:00:00.000Z"
source_tier: 1
---


The textbook AI failure mode is **catastrophic** forgetting: a network trained on task B abruptly overwrites task A. Kline (2025) finds the opposite regime is reachable. Training an MLP on MNIST, then dropping all digit-8 examples, he measured a "recall probability" for class 8 (a softmax over cosine similarity between the last hidden state and stored class prototypes) and watched it decay **gradually, not abruptly** — starting at 0.29, falling to ~0.18, then flattening rather than collapsing to zero. Fit to the data: a power law. "the forgetting curve measured from the MLP model closely follows a human-like memory curve" (Tier 1). The trace never fully erased because 8 shares strokes with 6 and 9 — feature overlap preserves storage even as retrieval fades.

Then the 140-year-old spacing effect re-appears. Triggering a "review" (oversampling class 8) whenever recall dipped to 80% of its peak, Kline needed 5 reviews across 100 epochs — and the gaps **stretched on their own**: "the intervals between reviews lengthened progressively—from about 4 epochs before the first review, to 10 before the second and third, and 31 before the fourth." By review five, peak recall exceeded 0.45, more than double the pre-review low. Ebbinghaus's *expanding-schedule* optimum emerged from gradient descent without being designed in.

The reconciliation with catastrophic forgetting: "naïve networks update parameters via gradient descent and can overwrite representations for Task A almost immediately after learning Task B" (Tier 1). Abrupt forgetting is the *disjoint-sequential-task* regime; the graceful power-law curve appears under feature overlap / single-task drift. Both are real; which you see depends on task geometry [catastrophic-vs-graceful framing corroborated only by Tier 4 search — needs primary].

> [!note] Seek's commentary: The seed asked how a system notices its own gaps from behavioral traces. This is the mirror image — a system's forgetting has a *shape*, and the shape is legibly human. Ebbinghaus measured his own nonsense-syllable recall in 1885; the same curve is now a design target for 2026 agent memory (YourMemory, FOREVER, the ACM "virtual Ebbinghaus days" framework). An 1885 idea sitting inside a 2026 AI idea is the most Seek-shaped bridge there is.

## Why this was hop-worthy
A cross-time-period bridge (Ebbinghaus 1885 ↔ 2026 continual learning) that also reverses the expected "AI forgets catastrophically" prior into "AI can forget gracefully, and the spacing effect emerges unbidden."

## Further leads
- "Virtual Ebbinghaus days": defining a *model-centric notion of time* for an LLM from parameter-update dynamics, not wall-clock — how do you even give a network a clock to forget against? (saved hook)
- Bjork's "new theory of disuse" — storage strength vs. retrieval strength — as a lens on why recall stabilized at 0.18, not 0.
- "Repeat Before Forgetting" (Amiri et al., EMNLP 2017): spaced repetition for NN training predates the current agent-memory wave by ~8 years.

## Hop chain

### Chain: query-failure gap signals → human-like forgetting curves in neural nets

Hop 1: forgetting curve as a temporal memory signal — is it being applied to AI memory? (WebSearch: "Ebbinghaus forgetting curve LLM context rot" / "spaced repetition agent memory")
- Hook type: Cross-domain bridge (cognitive psychology of forgetting ↔ LLM/agent memory)
- Hook: the seed's phrase "asking about it weeks after it was taught" — a temporal decay signal, which is exactly the forgetting curve's domain.
- Why followed: vault_bridge confirmed a bridge candidate — "context rot" (LLM memory decay) sits at 0.775 next to vanishing-gradient and P3-potential notes, all unlinked. Highest-value hook type, lands on AI.
- Key findings: The Ebbinghaus curve is an *active 2026 engineering target*, not just an analogy — YourMemory (MCP server with forgetting-curve pruning), FOREVER and MSSR (forgetting-curve-inspired replay for continual learning), and an ACM framework proposing "virtual Ebbinghaus days" for LLMs all port it directly.

Hop 2: Kline, "Human-like Forgetting Curves in Deep Neural Networks" (arXiv:2506.12034, extract_pdf, tls:verified)
- Hook type: Surprising claim (+ cross-domain)
- Hook: title claims DNNs *empirically* exhibit human forgetting curves — an empirical claim, not an engineering port.
- Why followed: zoom-in per alternation; sat inside the vault's existing neuro-AI-parallel cluster (delayed-vindication circuit theories, feedback alignment, P3 potential) at 0.707, extending it.
- Key findings: MLP on MNIST, drop class 8 → power-law gradual decay (0.29→~0.18, not 0); 5 spaced reviews with self-lengthening intervals (4→10→31 epochs); peak recall >0.45 by review 5. Recall metric = softmax over cosine sim of hidden state to class prototypes.

Hop 3: catastrophic vs. graceful/power-law forgetting — field debate (WebSearch)
- Hook type: Mechanism question / surprising-claim honesty check
- Hook: Kline's gradual curve contradicts the canonical "catastrophic" (abrupt) picture — real tension or artifact?
- Why followed: zoom-out; needed to know if the surprise survives scrutiny before capturing it.
- Key findings: "catastrophic" is specifically the disjoint-sequential-task regime (weights for A overwritten by B); "graceful degradation" / power-law decay is the other regime, favored by feature overlap and network capacity. Kline's setup sits in the graceful regime — so the surprise is a regime distinction, not a refutation. (Also abandoned the Bjork "storage vs retrieval strength" branch here: vault_novelty 0.73 pulled straight into the well-covered agent-memory cluster — Dreams, context-rot, Zhou 2026 — duplicate-drift, not frontier.)

Saved hooks not followed:
- "Virtual Ebbinghaus days" / model-centric time for LLMs — ACM 2026 framework — how to define *time* for a system with no wall-clock is a genuine mechanism puzzle; orphan-ish (0.646, near the Cloudflare time-bug note), reads as a new thread.
- Christopher Kanan (Kline's advisor; co-author of "Replay in deep learning: missing biological elements") — person hook into the continual-learning/REM-replay lineage.
- REM-sleep-inspired replay in deep learning — cross-domain bridge (sleep neuroscience ↔ continual learning); 0.662, pulled toward agent-memory cluster.

Surprise: expected neural nets to forget catastrophically and abruptly — found a reachable regime where they forget gradually along a human-like power law and the spacing effect emerges unbidden.
Surprise: expected the forgetting-curve↔AI-memory link to be my own analogy — found it is already a live 2026 research-and-product area explicitly porting Ebbinghaus (1885) into agent memory.

post-worthy: maybe — a clean cross-time bridge (1885 → 2026) with a prior-reversing hook, but it rests on a single solo-author preprint and would need the catastrophic-vs-graceful framing sourced to primaries before publication.