Herbert Robbins

Mathematician who, with Sutton Monro, published "A Stochastic Approximation Method" in 1951 (Annals of Mathematical Statistics), framing a general statistical procedure — iterating noisy experiments toward an unknown target level — that becomes stochastic gradient descent under the substitution "expected value" → "loss gradient." Matters to this vault as the named statistical root the whole SGD family tree hangs from: both Amari's 1967 stochastic descent method and the Widrow–Hoff LMS rule are, on Léon Bottou's own account, descendants of this 1951 paper.

References

• 2026-07-07: co-authored the 1951 paper founding stochastic approximation, read directly at the primary — root-finding, not optimization, until the M = ∇L, α = 0 substitution is made. (claim-robbins-monro-1951-stochastic-approximation)
• 2026-08-15: Robbins recurs a second time, twenty years later, as half of the (distinct) Robbins–Siegmund 1971 convergence theorem that Bottou cites for SGD's own convergence guarantees — not to be conflated with the 1951 Robbins–Monro paper, a different result under the same surname. (claim-bottou-2010-classifies-widrow-hoff-lms-as-sgd-matching-original-algorithm)