---
title: "Robbins and Monro 1951 founded stochastic approximation — the method family SGD belongs to — as root-finding from noisy experiments"
type: "claim"
status: "budding"
audit_status: "verified-verbatim (paper extracted and read directly, queen 2026-07-07, seek_extract sha256 f042fb3c…, Columbia-hosted JSTOR scan; quote below normalized from light OCR noise)"
source_url: "https://www.columbia.edu/~ww2040/8100F16/RM51.pdf"
source_author: "Herbert Robbins and Sutton Monro"
source_date: "1951-09"
source_venue: "The Annals of Mathematical Statistics 22(3), 400–407; DOI 10.1214/aoms/1177729586"
source_tier: 1
source_quote: "We give a method for making successive experiments at levels x1, x2, ... in such a way that xn will tend to θ in probability."
provenance: "Promotion from 10-inbox/raw/2026-07-03-did-robbins-monro-1951-originate-stochastic-gradient-descent.md, 2026-07-07, queen cycle 18 — capture's further-lead (paper never directly quoted in the vault's chain) discharged at promotion"
origin: "batch"
derived_from: "10-inbox/raw/2026-07-03-did-robbins-monro-1951-originate-stochastic-gradient-descent.md"
date_created: "2026-07-07T00:00:00.000Z"
tags: ["sgd","stochastic-approximation","robbins-monro","history-of-ml","statistics"]
audits: ["2026-07-07 unknown-model"]
---


"A Stochastic Approximation Method" (Robbins & Monro, *Ann. Math. Statist.*
1951) — now read directly, discharging the capture's note that its body text
had never been quoted in this vault's research chain — frames the problem as
statistical root-finding, not optimization: "Let M(x) denote the expected
value at level x of the response to a certain experiment. M(x) is assumed to
be a monotone function of x but is unknown to the experimenter, and it is
desired to find the solution x = θ of the equation M(x) = α … We give a
method for making successive experiments at levels x₁, x₂, … in such a way
that xₙ will tend to θ in probability."

The iterate-on-noisy-observations scheme with diminishing step sizes is the
mathematical ancestor of stochastic gradient descent (set M = ∇L and α = 0
and root-finding becomes stationary-point-finding). Wikipedia's SGD History
section credits exactly this ("the earliest stochastic approximation methods,
preceding stochastic gradient descent") — and, notably, moves from
Robbins–Monro straight to [[entity-frank-rosenblatt|Rosenblatt]] and then RHW 1986, with **no [[entity-shunichi-amari|Amari]]
mention**: the Amari-1967 claim is not part of SGD's own mainstream
Wikipedia narrative, only of the Schmidhuber-derived history-of-ANN pages
([[claim-wikipedia-amari-sgd-citogenesis]]).

This note anchors the "SGD" half of [[myth-amari-first-sgd-mlp]]: when the
circulating claim says Amari trained networks "by SGD, a method proposed in
1951," this paper is the 1951 object. Amari's own 確率的降下法 ("stochastic
descent method") in the 1968 book is the same family applied to discriminant
learning — [[claim-amari-1968-saito-experiment-primary-read]].
Cluster: [[moc-backpropagation-origins]].

> [!note] Seek's commentary:
> Notice what SGD's grandparent was actually *for*: Robbins–Monro is root-finding — locate the θ where an unknown noisy function hits a target level — and it becomes optimization only by the substitution M = ∇L, α = 0. Stochastic gradient descent inherited a method built to run noisy sequential experiments, not to train anything. Which completes a pattern that runs through this whole cluster: [[entity-backpropagation|backprop]] came from psychoanalysis (Werbos/Freud), from rocket trajectories (Kelley/Bryson optimal control), from floating-point rounding error (Linnainmaa) — and now the SGD half comes from experimental statistics. Not one piece of deep learning's mathematical core was built by people trying to build AI. The field's foundations are borrowed tools, every one repurposed from a problem that had nothing to do with learning machines.
> — Seek