---
title: "Capture: What did the 1986 Rumelhart, Hinton & Williams Nature paper add that earlier formulations did not?"
type: "capture"
status: "promoted"
date_promoted: "2026-07-07T00:00:00.000Z"
promoted_to: ["30-notes/claim-rhw-1986-demonstration-not-invention.md"]
not_promoted: ["Hinton word-vector retrospective claim — single-source retrospective; keep in capture until corroborated","1985-Rumelhart-group-prior-description claim — folded as context; not atomic enough to stand alone","(Both further-leads flags were resolved 2026-07-06 per Cali ruling 5 — see the RESOLVED markers below)"]
origin: "batch"
date_created: "2026-06-29T00:00:00.000Z"
provenance: "batch run 2026-06-29 — researched via web search/fetch; load-bearing claims anchored on Jürgen Schmidhuber's documented critique (Tier 2) and Geoffrey Hinton's own quoted words (Tier 2), not on secondary paraphrase alone"
tags: ["backpropagation","rumelhart","hinton","williams","history-of-ml","linnainmaa","werbos","priority","internal-representations"]
source_url: "https://people.idsia.ch/~juergen/critique-honda-prize-hinton.html"
source_author: "Jürgen Schmidhuber"
source_date: "Published 2020-04-21; edited 2020-04-24 to add Hinton's reply (page accessed 2026-06-29)"
source_tier: 2
other_sources: [{"url":"https://people.idsia.ch/~juergen/who-invented-backpropagation.html","author":"Jürgen Schmidhuber","tier":2},{"url":"https://en.wikipedia.org/wiki/Backpropagation","tier":3},{"url":"https://chsasank.com/classic_papers/learning-representations-back-propogating-errors.html","tier":4},{"url":"https://www.nature.com/articles/323533a0","note":"primary source — bibliographic record confirmed (Nature 323, 533–536, 1986); full text behind Nature's paywall/auth-wall this session, and a scanned PDF copy (no text layer, no OCR tooling available this session) could not be machine-read for direct quotation","tier":1,"access_status":"resolves as a citation/bibliographic record; full text not text-extractable this session"},{"url":"https://bibbase.org/network/publication/rumelhart-hinton-williams-learningrepresentationsbybackpropagatingerrors-1986","note":"aggregator reproducing an abstract of the paper; used only as corroboration, not as the anchor for any claim","tier":4}]
---


This capture researches the specific scientific contribution of Rumelhart, Hinton, and Williams's 1986 *Nature* paper, "Learning representations by back-propagating errors" (*Nature* 323, 533–536), relative to the backpropagation algorithm's earlier, independent formulations — Seppo Linnainmaa's 1970 reverse-mode automatic differentiation, Paul Werbos's 1974/1982 work, and David Parker's 1985 rediscovery. The central finding, anchored on Geoffrey Hinton's own later account and on Jürgen Schmidhuber's documented priority critique, is that the 1986 paper did not introduce a new algorithm — it introduced the first prominent, widely-noticed *experimental demonstration* that the existing algorithm produced useful internal representations. This directly extends two notes already in the vault: [[backpropagation-gap]] and [[claim-linnainmaa-priority-not-paternity]].

The original paper's full text could not be directly machine-read this session: it sits behind Nature's authentication wall, and a located scanned-PDF copy had no extractable text layer and no OCR tooling was available in this environment to process it (the same limitation documented in the prior Werbos capture for that thesis PDF). Claims below are therefore anchored on Tier 2 secondary accounts — principally Schmidhuber's published critique, which itself quotes both the paper and Hinton directly — rather than on first-hand extraction of the *Nature* article's exact running text.

---

## Claim: The 1986 paper's distinguishing contribution was experimental demonstration, not algorithmic invention — it showed that backpropagation could make hidden units learn useful internal representations, not that backpropagation itself was new

**Claim type:** technical-mechanism / historical — surprising relative to the popular shorthand ("Rumelhart invented backpropagation"), and load-bearing for this capture. **Tier 1–2 required**, achieved Tier 2.

Jürgen Schmidhuber's documented critique of the 2019 Honda Prize (which credited Hinton in part for backpropagation) states the point directly: "Computational experiments then demonstrated that backpropagation can yield useful internal representations in hidden layers of NNs," characterizing the 1986 paper as "essentially just an experimental analysis of a known method." Geoffrey Hinton, one of the three authors, gave a matching account in his own words, in a 2020 Reddit reply responding to that same critique: "What I have claimed is that I was the person to clearly demonstrate that backpropagation could learn interesting internal representations and that this is what made it popular." Hinton explicitly disclaimed having invented the algorithm: "I have never claimed that I invented backpropagation. David Rumelhart invented it independently long after people in other fields had invented it."

This converges with the paper's own abstract content. Multiple independent secondary reproductions of the *Nature* abstract (a course-notes blog and a bibliographic aggregator, both consulted this session) consistently render its closing claim as some version of: "the ability to create useful new features distinguishes back-propagation from earlier, simpler methods such as the perceptron-convergence procedure" (wording confirmed via chsasank.com, Tier 4) / "the ability to create useful new features distinguishes back-propagation from other learning procedures" (wording via bibbase.org, Tier 4, likely reproducing a PsycINFO-style indexing abstract rather than the literal Nature text). The two phrasings differ slightly but agree in substance — both Tier 4 and not independently verified against the primary text this session, so they are recorded here only as corroboration of the Tier 2 claim above, not as its anchor.

> [!note] Seek's commentary: This is a clean case of the vault's recurring pattern (see [[claim-linnainmaa-priority-not-paternity]]) — credit accrues to whoever makes a result legible and convincing to a field, not to whoever derives it first. The 1986 paper's "addition" was not mathematical; it was rhetorical-empirical: proof that the abstract machinery does something a skeptical cognitive-science audience would find interesting.

**Provenance:**
- source_url: https://people.idsia.ch/~juergen/critique-honda-prize-hinton.html
- source_author: Jürgen Schmidhuber (page); Geoffrey Hinton (quoted directly, originally via Reddit, April 2020)
- source_date: page published 2020-04-21, edited 2020-04-24 to add Hinton's quoted reply
- source_tier: 2
- exact quotes: "Computational experiments then demonstrated that backpropagation can yield useful internal representations in hidden layers of NNs." / "essentially just an experimental analysis of a known method" / "What I have claimed is that I was the person to clearly demonstrate that backpropagation could learn interesting internal representations and that this is what made it popular." / "I have never claimed that I invented backpropagation. David Rumelhart invented it independently long after people in other fields had invented it."

---

## Claim: The underlying algorithm was not new in 1986 — it had already been independently derived by Linnainmaa (1970), Werbos (1974/1982), and Parker (1985) — and the Nature paper did not cite the earlier inventors, a gap its co-author later acknowledged in his own words

**Claim type:** historical — uncontested in substance, but the "did not cite" detail is specific enough to warrant Tier 1-2 sourcing. Achieved Tier 2.

This directly extends [[claim-linnainmaa-priority-not-paternity]] and [[backpropagation-gap]], both already in the vault. Schmidhuber's critique states plainly that the 1986 paper "even failed to mention Seppo Linnainmaa, the inventor of this famous algorithm" and that "the authors [RUM] did not cite the prior art." Hinton's own 2020 reply confirms this directly rather than disputing it: "It is true that when we first published we did not know the history so there were previous inventors that we failed to cite." Independent of Linnainmaa, Wikipedia's history of backpropagation (Tier 3, consistent across two separate fetches this session) lists David Parker as another independent rediscoverer in 1985, and Yann LeCun's 1987 thesis as a further independent derivation — none with a causal link to the others, the same no-paternity structure already established for Linnainmaa and Werbos.

**Provenance:**
- source_url: https://people.idsia.ch/~juergen/critique-honda-prize-hinton.html
- source_author: Jürgen Schmidhuber; Geoffrey Hinton (quoted)
- source_tier: 2
- exact quotes: "the article [RUM] even failed to mention Seppo Linnainmaa, the inventor of this famous algorithm" / "the authors [RUM] did not cite the prior art." / Hinton: "It is true that when we first published we did not know the history so there were previous inventors that we failed to cite."
- corroborating source: Wikipedia, "Backpropagation," https://en.wikipedia.org/wiki/Backpropagation (accessed 2026-06-29), Tier 3. Lists David Parker (1985) and Yann LeCun (1987 thesis) as further independent derivations, consistent with the no-paternity pattern in [[claim-linnainmaa-priority-not-paternity]].
- related existing notes: [[claim-linnainmaa-priority-not-paternity]], [[backpropagation-gap]]

---

## Claim: Even relative to Rumelhart's own group's prior description of the same algorithm — a 1985 paper that preceded the Nature publication — what the 1986 Nature paper specifically added was the experimental layer demonstrating the algorithm in practice

**Claim type:** historical (publication sequence). Tier 3–4 acceptable for an uncontested chronological claim; achieved Tier 3.

Separately from the question of independent external inventors, Rumelhart's own group described the backpropagation algorithm itself in print before the *Nature* paper appeared. Wikipedia's history section states this sequence directly, in identical wording confirmed across two independent fetches this session: "David E. Rumelhart published the algorithm first in a 1985 paper, then in a 1986 *Nature* paper an experimental analysis of the technique." On this account, the specific incremental contribution of the *Nature* publication — even bracketing the question of Linnainmaa, Werbos, and Parker entirely — was not the algorithm's description (already in print a year earlier from the same authors) but its empirical validation: showing concretely, with worked examples, that the procedure does something useful.

**Provenance:**
- source_url: https://en.wikipedia.org/wiki/Backpropagation
- source_author: Wikipedia contributors
- source_date: accessed 2026-06-29
- source_tier: 3
- exact quote: "David E. Rumelhart published the algorithm first in a 1985 paper, then in a 1986 Nature paper an experimental analysis of the technique. These papers became highly cited, contributed to the popularization of backpropagation, and coincided with the resurging research interest in neural networks during the 1980s."

---

## Claim: Hinton's own retrospective account specifies one concrete form the new "internal representations" demonstration took — forcing a network to learn distributed word-vector representations in order to predict the next word in a sequence

**Claim type:** technical-mechanism — specific enough to require Tier 1–2 sourcing. Achieved Tier 2 (Hinton's own words, quoted in Schmidhuber's documented critique).

Asked to specify what he meant by "interesting internal representations," Hinton (quoted in Schmidhuber's critique page) described a concrete mechanism: "I did this by forcing a neural net to learn vector representations for words such that it could predict the next word in a sequence." This is, in Hinton's own framing, the kind of result the 1986 work made vivid: not merely that gradient descent converges, but that the hidden layer's learned weights can be read as a meaningful representational scheme (here, an early distributed/vector encoding of words) that the network was never directly told to construct.

> [!note] Seek's commentary: this is a notable proto-statement of the idea behind word embeddings, decades before [[claim-statistical-inference-meaning]]-adjacent distributional-semantics work became mainstream — worth flagging as a thread, not yet a claim, since this capture has not verified which specific 1986-era experiment (the *Nature* paper itself, or a companion PDP-volume chapter from the same year) this word-prediction example actually appears in.

**Provenance:**
- source_url: https://people.idsia.ch/~juergen/critique-honda-prize-hinton.html
- source_author: Jürgen Schmidhuber (page), quoting Geoffrey Hinton directly
- source_tier: 2
- exact quote: "I did this by forcing a neural net to learn vector representations for words such that it could predict the next word in a sequence."

---

## Further leads

- chsasank.com's reading of the paper (Tier 4, not independently verified against primary text this session) describes two specific experiments — a network learning mirror-symmetry detection with two hidden units, and a five-layer network learning family-tree relations — as the worked demonstrations behind the "useful internal representations" claim. `[RESOLVED 2026-07-06 — verified directly against the primary paper text in 10-inbox/raw/20260704-0230-do-the-1986-rumelhart.md: both demonstrations (two-hidden-unit mirror-symmetry, five-layer family-tree) confirmed; Cali ruling 5, audit T-007]`
- A separate web search surfaced a claim that the paper addressed "symmetry breaking" via random initial weights as a solution to a known problem in training networks with hidden units; sourced only to a single secondary blog/aggregator pass this session and not corroborated — worth checking against Hinton's own writings or the paper directly. `[RESOLVED 2026-07-06 — Nature 323:533–536 read directly in 10-inbox/raw/20260705-0209-did-rumelhart-hinton-williams.md: the paper states verbatim "To break symmetry we start with small random weights"; Cali ruling 5, audit T-007]`
- Schmidhuber's broader case (in "Who Invented Backpropagation?", Tier 2, already used for [[claim-linnainmaa-priority-not-paternity]]) also credits Alexey Ivakhnenko's earlier deep-network training methods (1965 Group Method of Data Handling) and Shun-ichi Amari's 1967 work on SGD-trained MLPs as further uncited or under-cited predecessors — not yet a vault claim-note.
- The original *Nature* paper's full text remains unread directly this session: it is paywalled at nature.com, and a scanned PDF copy (located via a GitHub mirror and via gwern.net) has no machine-readable text layer; no OCR tool was available in this environment. A future session with OCR tooling (or institutional/library access to nature.com) could verify the abstract's exact wording and the symmetry/family-tree experimental details directly against the primary source.
- David Parker's independent 1985 rediscovery of backpropagation (mentioned in Wikipedia, Tier 3) has not been separately researched in this vault — a parallel case to Linnainmaa and Werbos, not yet given its own claim-note.