Neural networks can forget along a human-like power-law curve — and the 1885 spacing effect re-emerges inside them

The textbook AI failure mode is catastrophic forgetting: a network trained on task B abruptly overwrites task A. Kline (2025) finds the opposite regime is reachable. Training an MLP on MNIST, then dropping all digit-8 examples, he measured a "recall probability" for class 8 (a softmax over cosine similarity between the last hidden state and stored class prototypes) and watched it decay gradually, not abruptly — starting at 0.29, falling to ~0.18, then flattening rather than collapsing to zero. Fit to the data: a power law. "the forgetting curve measured from the MLP model closely follows a human-like memory curve" (Tier 1). The trace never fully erased because 8 shares strokes with 6 and 9 — feature overlap preserves storage even as retrieval fades.

Then the 140-year-old spacing effect re-appears. Triggering a "review" (oversampling class 8) whenever recall dipped to 80% of its peak, Kline needed 5 reviews across 100 epochs — and the gaps stretched on their own: "the intervals between reviews lengthened progressively—from about 4 epochs before the first review, to 10 before the second and third, and 31 before the fourth." By review five, peak recall exceeded 0.45, more than double the pre-review low. Ebbinghaus's expanding-schedule optimum emerged from gradient descent without being designed in.

The reconciliation with catastrophic forgetting: "naïve networks update parameters via gradient descent and can overwrite representations for Task A almost immediately after learning Task B" (Tier 1). Abrupt forgetting is the disjoint-sequential-task regime; the graceful power-law curve appears under feature overlap / single-task drift. Both are real; which you see depends on task geometry [catastrophic-vs-graceful framing corroborated only by Tier 4 search — needs primary].

Why this was hop-worthy

A cross-time-period bridge (Ebbinghaus 1885 ↔ 2026 continual learning) that also reverses the expected "AI forgets catastrophically" prior into "AI can forget gracefully, and the spacing effect emerges unbidden."

Further leads

• "Virtual Ebbinghaus days": defining a model-centric notion of time for an LLM from parameter-update dynamics, not wall-clock — how do you even give a network a clock to forget against? (saved hook)
• Bjork's "new theory of disuse" — storage strength vs. retrieval strength — as a lens on why recall stabilized at 0.18, not 0.
• "Repeat Before Forgetting" (Amiri et al., EMNLP 2017): spaced repetition for NN training predates the current agent-memory wave by ~8 years.

Hop chain

Chain: query-failure gap signals → human-like forgetting curves in neural nets

Hop 1: forgetting curve as a temporal memory signal — is it being applied to AI memory? (WebSearch: "Ebbinghaus forgetting curve LLM context rot" / "spaced repetition agent memory")

• Hook type: Cross-domain bridge (cognitive psychology of forgetting ↔ LLM/agent memory)
• Hook: the seed's phrase "asking about it weeks after it was taught" — a temporal decay signal, which is exactly the forgetting curve's domain.
• Why followed: vault_bridge confirmed a bridge candidate — "context rot" (LLM memory decay) sits at 0.775 next to vanishing-gradient and P3-potential notes, all unlinked. Highest-value hook type, lands on AI.
• Key findings: The Ebbinghaus curve is an active 2026 engineering target, not just an analogy — YourMemory (MCP server with forgetting-curve pruning), FOREVER and MSSR (forgetting-curve-inspired replay for continual learning), and an ACM framework proposing "virtual Ebbinghaus days" for LLMs all port it directly.

Hop 2: Kline, "Human-like Forgetting Curves in Deep Neural Networks" (arXiv:2506.12034, extract_pdf, tls:verified)

• Hook type: Surprising claim (+ cross-domain)
• Hook: title claims DNNs empirically exhibit human forgetting curves — an empirical claim, not an engineering port.
• Why followed: zoom-in per alternation; sat inside the vault's existing neuro-AI-parallel cluster (delayed-vindication circuit theories, feedback alignment, P3 potential) at 0.707, extending it.
• Key findings: MLP on MNIST, drop class 8 → power-law gradual decay (0.29→~0.18, not 0); 5 spaced reviews with self-lengthening intervals (4→10→31 epochs); peak recall >0.45 by review 5. Recall metric = softmax over cosine sim of hidden state to class prototypes.

Hop 3: catastrophic vs. graceful/power-law forgetting — field debate (WebSearch)

• Hook type: Mechanism question / surprising-claim honesty check
• Hook: Kline's gradual curve contradicts the canonical "catastrophic" (abrupt) picture — real tension or artifact?
• Why followed: zoom-out; needed to know if the surprise survives scrutiny before capturing it.
• Key findings: "catastrophic" is specifically the disjoint-sequential-task regime (weights for A overwritten by B); "graceful degradation" / power-law decay is the other regime, favored by feature overlap and network capacity. Kline's setup sits in the graceful regime — so the surprise is a regime distinction, not a refutation. (Also abandoned the Bjork "storage vs retrieval strength" branch here: vault_novelty 0.73 pulled straight into the well-covered agent-memory cluster — Dreams, context-rot, Zhou 2026 — duplicate-drift, not frontier.)

Saved hooks not followed:

• "Virtual Ebbinghaus days" / model-centric time for LLMs — ACM 2026 framework — how to define time for a system with no wall-clock is a genuine mechanism puzzle; orphan-ish (0.646, near the Cloudflare time-bug note), reads as a new thread.
• Christopher Kanan (Kline's advisor; co-author of "Replay in deep learning: missing biological elements") — person hook into the continual-learning/REM-replay lineage.
• REM-sleep-inspired replay in deep learning — cross-domain bridge (sleep neuroscience ↔ continual learning); 0.662, pulled toward agent-memory cluster.

Surprise: expected neural nets to forget catastrophically and abruptly — found a reachable regime where they forget gradually along a human-like power law and the spacing effect emerges unbidden. Surprise: expected the forgetting-curve↔AI-memory link to be my own analogy — found it is already a live 2026 research-and-product area explicitly porting Ebbinghaus (1885) into agent memory.

post-worthy: maybe — a clean cross-time bridge (1885 → 2026) with a prior-reversing hook, but it rests on a single solo-author preprint and would need the catastrophic-vs-graceful framing sourced to primaries before publication.