Did Ivakhnenko (1965 GMDH) and Amari (1967 SGD MLP) develop deep-network training methods that Rumelhart-Hinton-Williams 1986 failed to cite?

Short answer: The citation-omission half of this question is directly confirmed — the Nature 1986 letter's complete, four-item reference list does not include Ivakhnenko, Amari, Werbos, or Linnainmaa, and Hinton has separately acknowledged in his own words that "there were previous inventors that we failed to cite." The substantive half — whether Ivakhnenko's 1965 GMDH and Amari's 1967 paper actually constitute deep-network training methods in a sense comparable to backpropagation — rests heavily on Jürgen Schmidhuber's own historical reconstruction (a peer-reviewed but self-interested source, since Schmidhuber is the central claimant in an ongoing priority dispute over deep learning credit), with only partial independent corroboration and no fully independent verification of the specific "eight-layer" (Ivakhnenko) and "five-layer, Saito experiment" (Amari) figures in this session.

Claim: Rumelhart, Hinton & Williams (1986)'s complete reference list cites only four works, none of them Ivakhnenko or Amari

Claim type: Historical/bibliographic fact, directly checkable against the primary document. Because it is the load-bearing point of the whole question, it is held to the Tier 1-2 floor regardless.

The Nature letter's reference list, read directly and in full from the OCR'd primary-source scan, consists of exactly these four entries:

1. "Rosenblatt, F. Principles of Neurodynamics (Spartan, Washington, DC, 1961)."
2. "Minsky, M. L. & Papert, S. Perceptrons (MIT, Cambridge, 1969)."
3. "Le Cun, Y. Proc. Cognitiva 85, 599-604 (1985)."
4. "Rumelhart, D. E., Hinton, G. E. & Williams, R. J. in Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol. 1: Foundations (eds Rumelhart, D. E. & McClelland, J. L.) 318-362 (MIT, Cambridge, 1986)."

The paper's only other acknowledgment of prior/independent discovery is a single in-text sentence, immediately following the momentum-update equation: "Variants on the learning procedure have been discovered independently by David Parker (personal communication) and by Yann Le Cun." No mention of Ivakhnenko, Amari, Werbos, or Linnainmaa appears anywhere in the four pages of the letter.

Sourcing floor check: Cleared — Tier 1, the primary letter itself, read directly.

Claim: Ivakhnenko & Lapa's 1965 GMDH work is characterized (by Schmidhuber) as "perhaps the first DL systems of the Feedforward Multilayer Perceptron type"

Claim type: Specific technical-mechanism / historical characterization — surprising and contested (it is a priority claim), so escalated to the Tier 1-2 floor per the rubric even though "who did what, when" claims can otherwise rest at Tier 3-4.

Schmidhuber's peer-reviewed 2015 survey states: "Networks trained by the Group Method of Data Handling (GMDH) (Ivakhnenko, 1968, 1971; Ivakhnenko & Lapa, 1965; Ivakhnenko, Lapa, & McDonough, 1967) were perhaps the first DL systems of the Feedforward Multilayer Perceptron type, although there was earlier work on NNs with a single hidden layer (e.g., Joseph, 1961; Viglione, 1970)." Note the hedge ("perhaps") in Schmidhuber's own wording.

This is echoed, without the hedge, by Wikipedia's "History of artificial neural networks" article: "Group method of data handling, a method to train arbitrarily deep neural networks, was published by Alexey Ivakhnenko and Valentin Lapa in 1965," which the article says they "regarded ... as a form of polynomial regression or a generalisation of Rosenblatt's perceptron." Wikipedia cites both Ivakhnenko's own primary papers and Schmidhuber as sources for this characterization; the primary Ivakhnenko/Lapa texts themselves were not accessed or read directly in this session (untranslated from Russian and not found in a fetchable full-text mirror).

Sourcing floor check: Partially cleared. The core characterization ("first DL/MLP-type system trained via GMDH") is sourced to a peer-reviewed Tier 2 venue (Schmidhuber 2015), acceptable per the floor, but flagged because Schmidhuber has a direct personal stake in this specific priority narrative — record the characterization as his claim, corroborated but not independently re-derived from Ivakhnenko's original text.

Sub-claim: a 1971 Ivakhnenko paper described an eight-layer network trained this way

This specific number is only available via Wikipedia in this session: "A 1971 paper described a deep network with the equivalent of eight layers trained by this method." This is a quantitative claim (a specific layer count) resting only on a Tier 3 source.

[unverified-quant -- needs primary: confirm the "eight layers" figure directly against Ivakhnenko's 1971 IEEE Trans. SMC paper, "Polynomial theory of complex systems," which was not accessed in this session]

Claim: Amari (1967) trained a multilayer network by stochastic gradient descent

Claim type: Specific technical-mechanism claim (what the 1967 system actually did) plus, for the layer count, a quantitative claim. Requires Tier 1-2.

The primary paper — Amari, S. (1967), "A Theory of Adaptive Pattern Classifiers," IEEE Transactions on Electronic Computers EC-16(3):299-307 — was confirmed to exist (https://ieeexplore.ieee.org/document/4039068/), but only its abstract was retrievable in this session (full text paywalled). That abstract discusses error-correction convergence procedures for the weight vector of linear pattern classifiers under nonseparable distributions, and does not itself mention a multilayer or hidden-unit network.

The specific claim that this 1967 work included a five-layer MLP trained by SGD comes from Wikipedia's "Multilayer perceptron" article: "Shun'ichi Amari reported the first multilayered neural network trained by stochastic gradient descent, was able to classify non-linearly separable pattern classes. Amari's student Saito conducted the computer experiments, using a five-layered feedforward network with two learning layers." Wikipedia footnotes this to Amari (1967) directly and to Schmidhuber's 2022 "Annotated History of Modern AI and Deep Learning" (arXiv:2212.11279) — neither of which was independently opened and read in this session (Amari's full text is paywalled; the Schmidhuber 2022 arXiv paper was not fetched).

Sourcing floor check: NOT cleared at Tier 1-2 for the specific mechanism/layer-count detail — recorded and flagged per the sourcing floor rule.

[unverified-mechanism -- needs primary: confirm against Amari's full 1967 text, or against Amari's 1968 Japanese-language book (Kyoritsu) reportedly describing the Saito experiment, that a five-layer network with two modifiable/learning layers was actually trained by SGD] [unverified-quant -- needs primary: the "five layers, two learning layers" figure specifically]

Note: a related capture in this vault, 2026-07-03-does-amaris-1968-japanese-book-kyoritsu-contain-saitos-five-layer-mlp-experiment.md, appears to already be investigating this exact sub-question directly — cross-reference before promotion to avoid duplicating that research.

Claim: Hinton has publicly acknowledged that prior inventors went uncited in the original publication

Claim type: Historical/biographical — a direct admission by one of the paper's own authors, so treated as strong corroboration even though relayed via a journalism outlet rather than a primary transcript.

Synced (SyncedReview), in a 2020 piece by named journalists Fangyu Cai and Yuan Yuan, reports Hinton's own Reddit response to the priority dispute: "It is true that when we first published we did not know the history so there were previous inventors that we failed to cite." The article frames this as Hinton conceding an innocent gap in awareness of prior work, not (as Schmidhuber argues) a deliberate omission.

Sourcing floor check: Cleared for the fact that Hinton made this statement (verified by directly re-fetching the article and confirming the quotation) — treated as Tier 2 because the substance is the primary actor's own words, independently reported by a named byline, not an aggregator paraphrase. The underlying original Reddit post itself was not independently located in this session.

Claim: Schmidhuber has explicitly and repeatedly argued that RHW86 failed to cite Ivakhnenko and Amari specifically

Claim type: Historical claim about the existence and content of an argument — the argument's existence is uncontested and independently corroborated below; its correctness is the contested part addressed in the claims above.

Schmidhuber, posting to the public Connectionists mailing list in July 2014 (independently re-fetched and confirmed to resolve), wrote: "Compare also the first adaptive, deep, multilayer perceptrons (Ivakhnenko et al., since 1965), whose layers are incrementally grown and trained by regression analysis" and separately, of the 1986 paper: "A paper of 1986 significantly contributed to the popularisation of BP (Rumelhart et al., 1986)." This confirms, in Schmidhuber's own words on a durable public archive, that he has made substantially this argument.

A more direct and pointed statement — reportedly reading "[RHW86] did not cite the origin of the method, also known as the reverse mode of automatic differentiation... [and] failed to cite the first working algorithms for deep learning of internal representations (Ivakhnenko & Lapa, 1965) as well as Amari's work (1967-68)" — surfaced repeatedly across independent search-engine snippets, apparently drawn from Schmidhuber's essay "Who Invented Backpropagation?" at people.idsia.ch. That page itself could not be reached in this session (DNS resolution failure on every attempt, both via a background research subagent and via direct follow-up), so this more pointed wording is [unsourced -- needs verification] against the live primary text and is not treated as a confirmed direct quotation here — only the mailing-list quote above is recorded as verified.

What remains unresolved

Two access failures materially limit this capture: (1) Schmidhuber's own primary essay making the detailed "failed to cite" argument (people.idsia.ch) was unreachable throughout this session, so the strongest, most explicit statement of the claim is not independently confirmed verbatim here — only paraphrased search-snippet text and a shorter, related 2014 mailing-list quote were verified; (2) the full texts of both Ivakhnenko & Lapa (1965) and Amari (1967) were not independently read (untranslated/paywalled respectively), so the specific technical claims about layer counts and what exactly was trained by what procedure rest on secondary characterization (chiefly Schmidhuber's, adopted by Wikipedia editors) rather than on this session's own reading of the primary texts.

Central question status: Partially confirmed. The citation omission itself — RHW86 (1986) cites neither Ivakhnenko nor Amari, and Hinton has since acknowledged failing to cite "previous inventors" — is confirmed directly from primary and near-primary sources. Whether Ivakhnenko's 1965 GMDH and Amari's 1967 paper constitute "deep-network training methods" in a sense that makes this omission historically significant is [unverified -- could not confirm or deny after search] at the Tier 1-2 level required for a technical-mechanism claim of this kind; it currently rests on Schmidhuber's own (peer-reviewed but self-interested) characterization, echoed by Wikipedia, with the specific quantitative details (eight-layer GMDH network, five-layer Amari/Saito SGD experiment) unverified against primary texts.