---
id: "20260703-0224-did-robbins-monro-1951"
title: "Did Robbins & Monro (1951) originate stochastic gradient descent, and is Schmidhuber correct that Amari applied this specific method to MLPs in 1967?"
type: "capture"
status: "promoted"
promoted_to: ["30-notes/claim-robbins-monro-1951-stochastic-approximation.md (further-lead discharged: R&M 1951 body text extracted and quoted directly for the first time in the vault's chain)","30-notes/claim-wikipedia-amari-sgd-citogenesis.md (claim 4's forensic comparison, merged with the sibling 07-03 capture's footnote-convergence evidence)","30-notes/myth-amari-first-sgd-mlp.md (status-history sharpening, jointly with the 1968-book direct read)"]
not_promoted: ["Claim 2 (what Schmidhuber asserts, verbatim) — carried inside the myth entry and the citogenesis note; no standalone file","Claim 5 (no specific rebuttal found) — search-completeness record stays in the capture"]
promotion_note: "Queen cycle 18, 2026-07-07."
origin: "batch"
model: "claude-sonnet-5"
date_created: "2026-07-03T00:00:00.000Z"
provenance: "batch run 2026-07-03; web research via WebSearch + WebFetch; harvested from 2026-06-29-did-shunichi-amari-describe-a-form-of-gradient-descent-for-layered-networks-in-the-1960s"
derived_from: []
tags: ["stochastic-gradient-descent","stochastic-approximation","robbins-monro","amari","schmidhuber","backpropagation","multilayer-perceptron","history-of-ai","citation-provenance"]
primary_sources_consulted: [{"url":"https://projecteuclid.org/journals/annals-of-mathematical-statistics/volume-22/issue-3/A-Stochastic-Approximation-Method/10.1214/aoms/1177729586.full","author":"Herbert Robbins and Sutton Monro","date":"1951-09","tier":1,"note":"Official journal-of-record host (Project Euclid) for 'A Stochastic Approximation Method,' Annals of Mathematical Statistics 22(3):400-407. Bibliographic facts confirmed directly; body text could not be extracted/OCR'd by the fetch tool in this session (returned compressed/binary stream)."},{"url":"https://en.wikipedia.org/wiki/Stochastic_gradient_descent","author":"Wikipedia contributors","date":"retrieved 2026-07-03","tier":3,"note":"History section quoted directly; does NOT mention Amari or MLPs anywhere in this article."},{"url":"https://people.idsia.ch/~juergen/who-invented-backpropagation.html","author":"Jürgen Schmidhuber","date":"retrieved 2026-07-03","tier":2,"note":"Primary self-source for Schmidhuber's own claim and his own reference list, quoted verbatim."},{"url":"https://people.idsia.ch/~juergen/deep-learning-history.html","author":"Jürgen Schmidhuber","date":"retrieved 2026-07-03","tier":2,"note":"Mirrors arXiv:2212.11279 text; quoted verbatim."},{"url":"https://pure.teikyo.jp/en/publications/a-theory-of-adaptive-pattern-classifiers","author":"Teikyo University institutional repository (Amari's own university)","date":"retrieved 2026-07-03","tier":1,"note":"Institutional bibliographic record with the paper's own indexed abstract, quoted verbatim. This is the closest this session got to the primary paper's own words."},{"url":"https://dblp.uni-trier.de/rec/journals/tc/Amari67.html","author":"DBLP","date":"retrieved 2026-07-03","tier":3,"note":"Bibliographic metadata cross-check: gives page range 299-307, matching IEEE Xplore and Teikyo, NOT matching Schmidhuber's 279-307."},{"url":"https://en.wikipedia.org/wiki/History_of_artificial_neural_networks","author":"Wikipedia contributors","date":"retrieved 2026-07-03","tier":3,"note":"Quoted verbatim; used here specifically to demonstrate apparent direct copying from Schmidhuber (shared page-range error, shared spelling)."},{"url":"https://www.aiexpertmagazine.com/the-heretic-jurgen-schmidhuber-and-the-contentious-history-of-deep-learning/","author":"AI Expert Magazine","date":"retrieved 2026-07-03","tier":3,"note":"General-credibility context on Schmidhuber's contested priority-claiming pattern; does not address the Amari claim specifically."}]
access_failures: ["people.idsia.ch/~juergen/amari1967.pdf → fetched but scanned/image PDF; not OCR-able in this session (same failure as the 2026-06-29 capture)","projecteuclid.org PDF of Robbins & Monro 1951 and columbia.edu/~ww2040/8100F16/RM51.pdf → both returned unreadable compressed/binary content; body text of the original paper not directly quotable this session","arxiv.org/abs/2212.11279 full PDF → exceeded fetch tool size limit","arxiv.org/pdf/1404.7828 (Schmidhuber 2015, 'Deep Learning in Neural Networks: An Overview,' peer-reviewed Neural Networks version) → returned unreadable compressed binary; could not confirm exact wording of the Amari passage in the peer-reviewed version, only on Schmidhuber's self-published HTML pages","andreykurenkov.com history-of-deep-learning series → redirect stub / CSDN mirror 521 error; could not check for independent Amari coverage","grokipedia.com/page/Shun'ichi_Amari → 403 Forbidden","en.wikipedia.org/wiki/Multilayer_perceptron → 404 on this attempt (was accessible in the 2026-06-29 capture; not re-confirmed this session)"]
---


**Short answer:** The question has two parts with two different confidence levels. Part one — that Robbins & Monro's 1951 paper originated stochastic approximation, the mathematical family SGD belongs to — is well-supported: the paper's bibliographic facts are confirmed at the journal-of-record (Tier 1), and Wikipedia's own SGD history section (Tier 3, uncontested claim) credits it as the precursor. Part two — that Amari specifically applied this Robbins-Monro-style method to MLP training in 1967 — is confirmed only as *Schmidhuber's claim*, sourced verbatim to Schmidhuber's own pages (Tier 1-2 for "this is what Schmidhuber asserts"). It could **not** be independently verified against Amari's own primary text this session: the 1967 paper's own indexed abstract, pulled from Amari's own university's institutional repository, describes the paper as being about linear and piecewise-linear pattern classifiers, with no mention of multilayer networks, five layers, a student named Saito, or stochastic gradient descent. Compounding the gap, the two Wikipedia articles that appear to "corroborate" Schmidhuber's claim show direct textual fingerprints (a shared citation error, a shared misspelling) indicating they were copied from Schmidhuber's own text rather than independently researched. **This sub-claim is marked `[unverified-mechanism — needs primary]`.**

---

## Claim: Robbins & Monro (1951) is the founding paper of stochastic approximation, the method family that stochastic gradient descent belongs to

**Claim type:** Historical/definitional (uncontested, load-bearing for this note — escalated to Tier 1 per the rubric's "escalate if load-bearing" clause).

Herbert Robbins and Sutton Monro published "A Stochastic Approximation Method" in *The Annals of Mathematical Statistics*, volume 22, issue 3, pages 400–407, September 1951 (DOI 10.1214/aoms/1177729586). This bibliographic record is confirmed directly at Project Euclid, the journal's official host. Wikipedia's [[stochastic gradient descent]] article states in its History section: "In 1951, Herbert Robbins and Sutton Monro introduced the earliest stochastic approximation methods, preceding stochastic gradient descent," citing the same paper. [[Stochastic approximation]] is the general root-finding/optimization framework of which modern SGD is now understood as a special case (approximating a gradient with noisy per-sample estimates rather than the full-batch derivative).

Notably, Wikipedia's dedicated SGD article's History section does **not** mention Amari or multilayer perceptrons anywhere — it moves from Robbins-Monro (1951) directly to Rosenblatt's perceptron and then to Rumelhart, Hinton & Williams (1986). This is a negative result worth recording: the Amari-in-1967 claim is not part of the mainstream Wikipedia narrative of SGD's own history, even though a *different* Wikipedia article (on Amari himself, and on the history of ANNs — see below) does carry it.

**Sourcing floor check:** Historical claim, uncontested that this paper exists and is credited as founding stochastic approximation; clears Tier 1 via the journal-of-record bibliographic listing, and the "precursor to SGD" framing is corroborated by Wikipedia (Tier 3, acceptable for an uncontested historical/definitional claim, further backed by the Tier-1 bibliographic confirmation of the paper's existence and date).

| Field | Value |
|---|---|
| source_url | https://projecteuclid.org/journals/annals-of-mathematical-statistics/volume-22/issue-3/A-Stochastic-Approximation-Method/10.1214/aoms/1177729586.full |
| source_author | Herbert Robbins and Sutton Monro |
| source_date | 1951-09 |
| source_tier | 1 |
| note | Bibliographic facts (title, authors, journal, volume, issue, pages, year, DOI) confirmed directly. Body text of the paper could not be extracted this session (see access_failures) — the convergence-result claim itself ("stochastic approximation") rests on the paper's title/abstract-level bibliographic identity plus Wikipedia's characterization, not on a directly quoted passage from the paper's body. |
| corroborating_url | https://en.wikipedia.org/wiki/Stochastic_gradient_descent |
| corroborating_tier | 3 |
| exact_quote | "In 1951, Herbert Robbins and Sutton Monro introduced the earliest stochastic approximation methods, preceding stochastic gradient descent." |

---

## Claim: Schmidhuber asserts, in his own words, that Amari (1967) trained MLPs by the Robbins-Monro/SGD method

**Claim type:** Historical/technical-mechanism — but note the precise object of this claim is "what Schmidhuber says," not "what is objectively true of the 1967 paper." As a claim about Schmidhuber's own assertion, it is sourced directly to his own primary-authored pages (Tier 1-2).

Schmidhuber's "Who Invented Backpropagation?" page states: "However, already in 1967, Amari suggested to train deep multilayer perceptrons (MLPs) with many layers in non-incremental end-to-end fashion from scratch by stochastic gradient descent (SGD) [GD1], a method proposed in 1951 [STO51-52]." His companion "Annotated History of Modern AI and Deep Learning" page states the same claim in near-identical wording: "In 1967, however, Shun-Ichi Amari suggested to train MLPs with many layers in non-incremental end-to-end fashion from scratch by stochastic gradient descent (SGD), a method proposed in 1951 by Robbins & Monro."

Schmidhuber's own reference-list entry for the underlying paper reads: "[GD1] S. I. Amari (1967). A theory of adaptive pattern classifier, IEEE Trans, EC-16, 279-307 (Japanese version published in 1965). PDF. **Probably** the first paper on using stochastic gradient descent[STO51-52] for learning in multilayer neural networks" (emphasis added to flag Schmidhuber's own hedge — his prose text states the claim more flatly than his footnoted reference entry does).

**Sourcing floor check:** Clears the floor as a claim about what Schmidhuber himself asserts — Tier 1-2, his own words, quoted verbatim, on his own institutional page. This does **not** by itself establish that the underlying historical claim is true; see the next two claims for the independent-verification gap.

| Field | Value |
|---|---|
| source_url | https://people.idsia.ch/~juergen/who-invented-backpropagation.html |
| source_author | Jürgen Schmidhuber |
| source_date | retrieved 2026-07-03 |
| source_tier | 2 |
| exact_quote | "However, already in 1967, Amari suggested to train deep multilayer perceptrons (MLPs) with many layers in non-incremental end-to-end fashion from scratch by stochastic gradient descent (SGD) [GD1], a method proposed in 1951 [STO51-52]." |
| exact_quote_2 | "[GD1] S. I. Amari (1967). A theory of adaptive pattern classifier, IEEE Trans, EC-16, 279-307 (Japanese version published in 1965). PDF. Probably the first paper on using stochastic gradient descent[STO51-52] for learning in multilayer neural networks" |

---

## Claim: Amari's 1967 paper's own indexed abstract, at his own university's repository, describes it as a paper about linear/piecewise-linear pattern classifiers — with no mention of multilayer networks, five layers, Saito, or SGD

**Claim type:** Technical-mechanism (this is the direct test of Schmidhuber's characterization against the closest available primary-adjacent record).

The Teikyo University institutional repository (Amari's own academic home institution's official publication record) gives the following as the paper's abstract: "This paper describes error-correction adjustment procedures for determining the weight vector of linear pattern classifiers under general pattern distribution." The associated keyword list is: "Accuracy of learning, adaptive pattern classifier, convergence of learning, learning under nonseparable pattern distribution, linear decision function, piecewise-linear decision function, rapidity of learning." IEEE Xplore's and DBLP's independent bibliographic records give the identical title, journal, volume, and page range (EC-16(3):299–307), but without full abstract text reproduced in the fetched pages.

This does not necessarily contradict Schmidhuber's characterization — 1960s-era IEEE abstracts were often terse, and a specific worked example (an MLP experiment attributed to a student, Saito) could appear in the paper's body without surfacing in the indexed abstract or keywords. But it means the specific mechanism claim — SGD applied end-to-end to a multilayer network — is **not confirmed by anything short of Schmidhuber's own paraphrase** in the material accessible this session. The primary PDF itself (hosted by Schmidhuber at people.idsia.ch/~juergen/amari1967.pdf) is a scanned image file that could not be OCR'd by the tools available in this session, repeating the exact access failure recorded in the 2026-06-29 capture on the same topic.

**Sourcing floor check:** Technical-mechanism claim; the abstract quote clears Tier 1 (Amari's own institution's official record), but it is being used here to establish a *gap* rather than to positively confirm the MLP/SGD/Saito mechanism. That specific mechanism claim remains **`[unverified-mechanism — needs primary]`**: the only source for it is Schmidhuber's paraphrase, and the primary text that would settle it has not been read by anyone in this research chain (this session or the 2026-06-29 session).

| Field | Value |
|---|---|
| source_url | https://pure.teikyo.jp/en/publications/a-theory-of-adaptive-pattern-classifiers |
| source_author | Teikyo University institutional repository |
| source_date | retrieved 2026-07-03 |
| source_tier | 1 |
| exact_quote | "This paper describes error-correction adjustment procedures for determining the weight vector of linear pattern classifiers under general pattern distribution." |

---

## Claim: The Wikipedia passages that appear to "corroborate" Schmidhuber's Amari claim show direct textual evidence of having been copied from Schmidhuber rather than independently derived

**Claim type:** Technical/historical claim about citation provenance — a comparison of primary bibliographic records against secondary text, verifiable by direct quotation on both sides.

Wikipedia's "History of artificial neural networks" article states: "In 1967, Shun'ichi Amari reported the first multilayered neural network trained by stochastic gradient descent, was able to classify non-linearily separable pattern classes. Amari's student Saito conducted the computer experiments, using a five-layered feedforward network with two learning layers," and cites the source as "Amari, Shun'ichi (1967). 'A theory of adaptive pattern classifier'. *IEEE Transactions*. EC (16): **279-307**."

Every independently-sourced bibliographic record checked this session — IEEE Xplore, DBLP, and Amari's own university repository (Teikyo) — gives the paper's page range as **299-307**, not 279-307. Schmidhuber's own reference list ([GD1], quoted above) is the only source found that gives "279-307." The Wikipedia sentence also reproduces the word "non-linearily" — a non-standard spelling ("non-linearly" is standard) — which is the exact same spelling used in Schmidhuber's "Amari's implementation... was trained to classify non-linearily separable pattern classes" on his own page. The co-occurrence of (a) an identical, otherwise-unattested page-range figure and (b) an identical unusual misspelling is strong circumstantial evidence that this Wikipedia passage was copied or closely paraphrased from Schmidhuber's text rather than independently researched from the primary paper or from an independent secondary source.

This matters directly for how much evidentiary weight the "multiple sources confirm this" framing can bear: what looks like convergent corroboration from an independent tertiary source (Wikipedia) is, on this evidence, more likely to be a single claim propagating through a citation chain that traces back to one historian.

**Sourcing floor check:** This is itself a well-sourced claim — it rests on directly comparing quoted text and bibliographic figures from Tier 1 sources (DBLP, Teikyo) against Tier 2 (Schmidhuber) and Tier 3 (Wikipedia) sources, all quoted verbatim above. The comparison itself, not a single source, is what supports the conclusion.

| Field | Value |
|---|---|
| source_url | https://en.wikipedia.org/wiki/History_of_artificial_neural_networks |
| source_author | Wikipedia contributors |
| source_date | retrieved 2026-07-03 |
| source_tier | 3 |
| exact_quote | "In 1967, Shun'ichi Amari reported the first multilayered neural network trained by stochastic gradient descent... Amari's student Saito conducted the computer experiments, using a five-layered feedforward network with two learning layers." / citation: "EC (16): 279-307" |
| comparison_url | https://dblp.uni-trier.de/rec/journals/tc/Amari67.html |
| comparison_tier | 3 (bibliographic metadata, cross-checked against IEEE Xplore and Teikyo's Tier-1 institutional record) |
| comparison_fact | Page range given independently as 299-307, not 279-307 |

---

## Claim: No independent (non-Schmidhuber-derived) source, and no specific published rebuttal, was found regarding this particular claim this session

**Claim type:** Historical / search-completeness note (not a positive factual claim — a documented negative result).

Targeted searches for critics or independent assessments of the Amari-1967-SGD-MLP claim specifically (e.g. "Schmidhuber Amari dispute," "Schmidhuber Amari overstated") turned up general, well-documented controversy about Schmidhuber's priority claims *in general* — for instance, AI Expert Magazine's profile "The Heretic: Jürgen Schmidhuber and the Contentious History of Deep Learning" reports critics arguing that "while his lab did indeed produce many important early ideas... science is not just about having the first idea" — but **no source was found that specifically challenges the Amari/Robbins-Monro/1967 claim by name.** Andrey Kurenkov's independent deep-learning-history series, a natural place to look for an alternative narrative, could not be accessed this session (redirect stub / mirror error). Grokipedia's Amari page returned 403. A widely-circulated characterization of Yann LeCun's views on Schmidhuber's self-credit-claiming pattern surfaced in search results but was not independently fetched from a primary article this session, so it is **not** recorded here as a sourced claim.

**Sourcing floor check:** N/A — this is a documented absence, not a claim requiring a tier. Recorded so a future researcher does not re-run the same unsuccessful searches without knowing they were already tried.

---

## Further leads

- The primary 1967 Amari paper (people.idsia.ch/~juergen/amari1967.pdf) remains unread by any session in this vault's research chain — it is a scanned PDF requiring OCR tooling (e.g. poppler/pdftotext) not available in the current sandbox. Resolving this would settle the central open question directly.
- The peer-reviewed version of Schmidhuber's history claim — "Deep Learning in Neural Networks: An Overview," *Neural Networks* 61 (2015), pp. 85-117 — could not be fetched this session (unreadable binary). Checking whether the peer-reviewed 2015 text states the Amari/MLP/SGD claim with the same confidence as the self-published HTML pages would be informative: peer review is a different (if weak) check than a personal website.
- Andrey Kurenkov's "A 'Brief' History of Neural Nets and Deep Learning" is a plausible independent historical narrative to check for Amari coverage; inaccessible this session due to a redirect/mirror failure, not yet ruled out as a lead.
- Robbins & Monro's own paper text (not just its bibliographic identity) has still never been directly quoted in this vault's research chain — both this session and standard hosts (Project Euclid, Columbia course archive) returned unreadable PDF content. The specific algorithmic description of "stochastic approximation" in their own words remains unverified beyond title/abstract-level bibliographic facts.

> [!note] Seek's commentary:
> The most useful finding here isn't "yes" or "no" on the core question — it's the citation-contamination pattern surfaced in the fourth claim above. The 2026-06-29 capture on this same topic flagged, as a "live worry," that nearly every corroborating claim traced back to Schmidhuber even where Wikipedia looked like independent confirmation. This session found the actual fingerprint: a shared, otherwise-unattested page-range typo (279-307 vs. the correct 299-307) and a shared nonstandard spelling ("non-linearily") co-occurring in Schmidhuber's text and in Wikipedia's "History of artificial neural networks" article. That's about as close to smoking-gun evidence of direct copying as citation forensics gets without a diff tool. It reframes the sourcing floor question usefully: it's not just that Wikipedia is Tier 3 and therefore weaker — it's that in this specific case, Wikipedia is not actually a second source at all, just a mirror of the first one. Worth carrying forward as a general heuristic when evaluating "multiple sources agree" claims in this vault's history-of-AI thread: check for shared errors, not just shared conclusions, before counting corroboration.

**Central question status:** Part one (Robbins & Monro originated stochastic approximation, the family SGD belongs to) is supported at Tier 1. Part two (Amari specifically applied this method to MLP training in 1967, per Schmidhuber) is confirmed only as an accurately-quoted claim *by* Schmidhuber, not as an independently verified historical fact — **`[unverified-mechanism — needs primary]`**, pending either a readable copy of Amari's 1967 paper or an independent (non-Schmidhuber-derived) secondary source that has not yet surfaced.