Did Robbins & Monro (1951) originate stochastic gradient descent, and is Schmidhuber correct that Amari applied this specific method to MLPs in 1967?

Short answer: The question has two parts with two different confidence levels. Part one — that Robbins & Monro's 1951 paper originated stochastic approximation, the mathematical family SGD belongs to — is well-supported: the paper's bibliographic facts are confirmed at the journal-of-record (Tier 1), and Wikipedia's own SGD history section (Tier 3, uncontested claim) credits it as the precursor. Part two — that Amari specifically applied this Robbins-Monro-style method to MLP training in 1967 — is confirmed only as Schmidhuber's claim, sourced verbatim to Schmidhuber's own pages (Tier 1-2 for "this is what Schmidhuber asserts"). It could not be independently verified against Amari's own primary text this session: the 1967 paper's own indexed abstract, pulled from Amari's own university's institutional repository, describes the paper as being about linear and piecewise-linear pattern classifiers, with no mention of multilayer networks, five layers, a student named Saito, or stochastic gradient descent. Compounding the gap, the two Wikipedia articles that appear to "corroborate" Schmidhuber's claim show direct textual fingerprints (a shared citation error, a shared misspelling) indicating they were copied from Schmidhuber's own text rather than independently researched. This sub-claim is marked [unverified-mechanism — needs primary].

Claim: Robbins & Monro (1951) is the founding paper of stochastic approximation, the method family that stochastic gradient descent belongs to

Claim type: Historical/definitional (uncontested, load-bearing for this note — escalated to Tier 1 per the rubric's "escalate if load-bearing" clause).

Herbert Robbins and Sutton Monro published "A Stochastic Approximation Method" in The Annals of Mathematical Statistics, volume 22, issue 3, pages 400–407, September 1951 (DOI 10.1214/aoms/1177729586). This bibliographic record is confirmed directly at Project Euclid, the journal's official host. Wikipedia's stochastic gradient descent article states in its History section: "In 1951, Herbert Robbins and Sutton Monro introduced the earliest stochastic approximation methods, preceding stochastic gradient descent," citing the same paper. Stochastic approximation is the general root-finding/optimization framework of which modern SGD is now understood as a special case (approximating a gradient with noisy per-sample estimates rather than the full-batch derivative).

Notably, Wikipedia's dedicated SGD article's History section does not mention Amari or multilayer perceptrons anywhere — it moves from Robbins-Monro (1951) directly to Rosenblatt's perceptron and then to Rumelhart, Hinton & Williams (1986). This is a negative result worth recording: the Amari-in-1967 claim is not part of the mainstream Wikipedia narrative of SGD's own history, even though a different Wikipedia article (on Amari himself, and on the history of ANNs — see below) does carry it.

Sourcing floor check: Historical claim, uncontested that this paper exists and is credited as founding stochastic approximation; clears Tier 1 via the journal-of-record bibliographic listing, and the "precursor to SGD" framing is corroborated by Wikipedia (Tier 3, acceptable for an uncontested historical/definitional claim, further backed by the Tier-1 bibliographic confirmation of the paper's existence and date).

Claim: Schmidhuber asserts, in his own words, that Amari (1967) trained MLPs by the Robbins-Monro/SGD method

Claim type: Historical/technical-mechanism — but note the precise object of this claim is "what Schmidhuber says," not "what is objectively true of the 1967 paper." As a claim about Schmidhuber's own assertion, it is sourced directly to his own primary-authored pages (Tier 1-2).

Schmidhuber's "Who Invented Backpropagation?" page states: "However, already in 1967, Amari suggested to train deep multilayer perceptrons (MLPs) with many layers in non-incremental end-to-end fashion from scratch by stochastic gradient descent (SGD) [GD1], a method proposed in 1951 [STO51-52]." His companion "Annotated History of Modern AI and Deep Learning" page states the same claim in near-identical wording: "In 1967, however, Shun-Ichi Amari suggested to train MLPs with many layers in non-incremental end-to-end fashion from scratch by stochastic gradient descent (SGD), a method proposed in 1951 by Robbins & Monro."

Schmidhuber's own reference-list entry for the underlying paper reads: "[GD1] S. I. Amari (1967). A theory of adaptive pattern classifier, IEEE Trans, EC-16, 279-307 (Japanese version published in 1965). PDF. Probably the first paper on using stochastic gradient descent[STO51-52] for learning in multilayer neural networks" (emphasis added to flag Schmidhuber's own hedge — his prose text states the claim more flatly than his footnoted reference entry does).

Sourcing floor check: Clears the floor as a claim about what Schmidhuber himself asserts — Tier 1-2, his own words, quoted verbatim, on his own institutional page. This does not by itself establish that the underlying historical claim is true; see the next two claims for the independent-verification gap.

Claim: Amari's 1967 paper's own indexed abstract, at his own university's repository, describes it as a paper about linear/piecewise-linear pattern classifiers — with no mention of multilayer networks, five layers, Saito, or SGD

Claim type: Technical-mechanism (this is the direct test of Schmidhuber's characterization against the closest available primary-adjacent record).

The Teikyo University institutional repository (Amari's own academic home institution's official publication record) gives the following as the paper's abstract: "This paper describes error-correction adjustment procedures for determining the weight vector of linear pattern classifiers under general pattern distribution." The associated keyword list is: "Accuracy of learning, adaptive pattern classifier, convergence of learning, learning under nonseparable pattern distribution, linear decision function, piecewise-linear decision function, rapidity of learning." IEEE Xplore's and DBLP's independent bibliographic records give the identical title, journal, volume, and page range (EC-16(3):299–307), but without full abstract text reproduced in the fetched pages.

This does not necessarily contradict Schmidhuber's characterization — 1960s-era IEEE abstracts were often terse, and a specific worked example (an MLP experiment attributed to a student, Saito) could appear in the paper's body without surfacing in the indexed abstract or keywords. But it means the specific mechanism claim — SGD applied end-to-end to a multilayer network — is not confirmed by anything short of Schmidhuber's own paraphrase in the material accessible this session. The primary PDF itself (hosted by Schmidhuber at people.idsia.ch/~juergen/amari1967.pdf) is a scanned image file that could not be OCR'd by the tools available in this session, repeating the exact access failure recorded in the 2026-06-29 capture on the same topic.

Sourcing floor check: Technical-mechanism claim; the abstract quote clears Tier 1 (Amari's own institution's official record), but it is being used here to establish a gap rather than to positively confirm the MLP/SGD/Saito mechanism. That specific mechanism claim remains [unverified-mechanism — needs primary]: the only source for it is Schmidhuber's paraphrase, and the primary text that would settle it has not been read by anyone in this research chain (this session or the 2026-06-29 session).

Claim: The Wikipedia passages that appear to "corroborate" Schmidhuber's Amari claim show direct textual evidence of having been copied from Schmidhuber rather than independently derived

Claim type: Technical/historical claim about citation provenance — a comparison of primary bibliographic records against secondary text, verifiable by direct quotation on both sides.

Wikipedia's "History of artificial neural networks" article states: "In 1967, Shun'ichi Amari reported the first multilayered neural network trained by stochastic gradient descent, was able to classify non-linearily separable pattern classes. Amari's student Saito conducted the computer experiments, using a five-layered feedforward network with two learning layers," and cites the source as "Amari, Shun'ichi (1967). 'A theory of adaptive pattern classifier'. IEEE Transactions. EC (16): 279-307."

Every independently-sourced bibliographic record checked this session — IEEE Xplore, DBLP, and Amari's own university repository (Teikyo) — gives the paper's page range as 299-307, not 279-307. Schmidhuber's own reference list ([GD1], quoted above) is the only source found that gives "279-307." The Wikipedia sentence also reproduces the word "non-linearily" — a non-standard spelling ("non-linearly" is standard) — which is the exact same spelling used in Schmidhuber's "Amari's implementation... was trained to classify non-linearily separable pattern classes" on his own page. The co-occurrence of (a) an identical, otherwise-unattested page-range figure and (b) an identical unusual misspelling is strong circumstantial evidence that this Wikipedia passage was copied or closely paraphrased from Schmidhuber's text rather than independently researched from the primary paper or from an independent secondary source.

This matters directly for how much evidentiary weight the "multiple sources confirm this" framing can bear: what looks like convergent corroboration from an independent tertiary source (Wikipedia) is, on this evidence, more likely to be a single claim propagating through a citation chain that traces back to one historian.

Sourcing floor check: This is itself a well-sourced claim — it rests on directly comparing quoted text and bibliographic figures from Tier 1 sources (DBLP, Teikyo) against Tier 2 (Schmidhuber) and Tier 3 (Wikipedia) sources, all quoted verbatim above. The comparison itself, not a single source, is what supports the conclusion.

Claim: No independent (non-Schmidhuber-derived) source, and no specific published rebuttal, was found regarding this particular claim this session

Claim type: Historical / search-completeness note (not a positive factual claim — a documented negative result).

Targeted searches for critics or independent assessments of the Amari-1967-SGD-MLP claim specifically (e.g. "Schmidhuber Amari dispute," "Schmidhuber Amari overstated") turned up general, well-documented controversy about Schmidhuber's priority claims in general — for instance, AI Expert Magazine's profile "The Heretic: Jürgen Schmidhuber and the Contentious History of Deep Learning" reports critics arguing that "while his lab did indeed produce many important early ideas... science is not just about having the first idea" — but no source was found that specifically challenges the Amari/Robbins-Monro/1967 claim by name. Andrey Kurenkov's independent deep-learning-history series, a natural place to look for an alternative narrative, could not be accessed this session (redirect stub / mirror error). Grokipedia's Amari page returned 403. A widely-circulated characterization of Yann LeCun's views on Schmidhuber's self-credit-claiming pattern surfaced in search results but was not independently fetched from a primary article this session, so it is not recorded here as a sourced claim.

Sourcing floor check: N/A — this is a documented absence, not a claim requiring a tier. Recorded so a future researcher does not re-run the same unsuccessful searches without knowing they were already tried.

Further leads

• The primary 1967 Amari paper (people.idsia.ch/~juergen/amari1967.pdf) remains unread by any session in this vault's research chain — it is a scanned PDF requiring OCR tooling (e.g. poppler/pdftotext) not available in the current sandbox. Resolving this would settle the central open question directly.
• The peer-reviewed version of Schmidhuber's history claim — "Deep Learning in Neural Networks: An Overview," Neural Networks 61 (2015), pp. 85-117 — could not be fetched this session (unreadable binary). Checking whether the peer-reviewed 2015 text states the Amari/MLP/SGD claim with the same confidence as the self-published HTML pages would be informative: peer review is a different (if weak) check than a personal website.
• Andrey Kurenkov's "A 'Brief' History of Neural Nets and Deep Learning" is a plausible independent historical narrative to check for Amari coverage; inaccessible this session due to a redirect/mirror failure, not yet ruled out as a lead.
• Robbins & Monro's own paper text (not just its bibliographic identity) has still never been directly quoted in this vault's research chain — both this session and standard hosts (Project Euclid, Columbia course archive) returned unreadable PDF content. The specific algorithmic description of "stochastic approximation" in their own words remains unverified beyond title/abstract-level bibliographic facts.

Central question status: Part one (Robbins & Monro originated stochastic approximation, the family SGD belongs to) is supported at Tier 1. Part two (Amari specifically applied this method to MLP training in 1967, per Schmidhuber) is confirmed only as an accurately-quoted claim by Schmidhuber, not as an independently verified historical fact — [unverified-mechanism — needs primary], pending either a readable copy of Amari's 1967 paper or an independent (non-Schmidhuber-derived) secondary source that has not yet surfaced.