---
title: "Widrow's 1990 retrospective (with Lehr) proves the LMS per-sample 'instantaneous gradient' is an unbiased estimate of the true gradient — the defining mathematical property of stochastic gradient descent"
type: "claim"
status: "seedling"
source_url: "https://isl.stanford.edu/~widrow/papers/j199030years.pdf"
source_title: "30 Years of Adaptive Neural Networks: Perceptron, Madaline, and Backpropagation"
source_author: "Bernard Widrow and Michael A. Lehr"
source_date: "1990-09"
source_venue: "Proceedings of the IEEE, vol. 78, no. 9, pp. 1415-1442, September 1990"
source_tier: 1
source_sha: "f41e3cc5c1e8cc27e49dd78e69dfd823721caf49b944ede19bae5f5ccf8fdfd3"
source_quote: "It i s shown in [30] that the instantaneous gradient i s an unbiased estimate of the true gradient."
provenance: "Promotion from 10-inbox/raw/2026-08-15-is-the-widrowhoff-lms-rule-established-as-a.md, 2026-08-15"
origin: "batch"
derived_from: ["20260815-0224-is-the-widrowhoff-lms"]
date_created: "2026-08-15T00:00:00.000Z"
writer_model: "claude-sonnet-5"
tags: ["widrow","hoff","lms","adaline","sgd","stochastic-gradient-descent","history-of-ml","backpropagation-origins"]
audit_status: "capture-verified — fetched fresh via extract_pdf (TLS verified) at capture time, 2026-08-15, from Widrow's own Stanford ISL page. Method pdftotext; text is largely clean but the two-column layout occasionally interleaves columns line-by-line, and a stray space appears inside 'is' ('i s') throughout the document — an extraction artifact, not the source's own typography. Quote verified grounded via quote_check against the extracted text. Independent cross-model re-check not yet performed."
verified_verbatim: "2026-08-17 — source_quote matched verbatim (normalized) against a direct fetch of source_url by seek_verify (no model involved)"
seek_code_commit: "17d9798"
---


In "30 Years of Adaptive Neural Networks: Perceptron, Madaline, and Backpropagation" (Widrow & Lehr, *Proceedings of the IEEE*, 1990), [[entity-bernard-widrow|Widrow]] derives the μ-LMS algorithm as approximate steepest descent on the mean-square-error surface, substituting "an instantaneous gradient based upon the square of the instantaneous linear error" for the true gradient "because it is readily available from a single data sample" — computing the true gradient "would involve averaging the instantaneous gradients associated with all patterns in the training set" and is "usually impractical." The paper states plainly: "It is shown in [30] that the instantaneous gradient is an unbiased estimate of the true gradient" (reference [30]: Widrow & Stearns, *Adaptive Signal Processing*, Prentice-Hall, 1985 — unread in this vault; see Further leads below).

A rule that substitutes a single-sample, unbiased estimate of the expected gradient for the true gradient, and iterates on that estimate, is stochastic gradient descent by mathematical definition — this is the property the name refers to. Widrow himself proves it holds for LMS in this Tier-1 primary source, again without ever writing the compound phrase "stochastic gradient descent." Together with [[claim-widrow-hoff-1960-original-paper-describes-lms-as-stochastic-steepest-descent|the 1960 original paper]] and [[claim-bottou-2010-classifies-widrow-hoff-lms-as-sgd-matching-original-algorithm|Bottou's 2010 outside classification]], this closes [[question-verify-lms-is-stochastic-gradient-descent-primary]]: every piece of the SGD mechanism — a stochastic surface, single-sample steepest descent, and now the unbiasedness proof — is established in Widrow's own words across thirty years, even though the label itself arrived from elsewhere. Cluster: [[moc-backpropagation-origins]].

**Further leads.** Widrow & Stearns, *Adaptive Signal Processing* (Prentice-Hall, 1985) — the book this quote cites as reference [30], and the text [[question-verify-lms-is-stochastic-gradient-descent-primary]] originally named as the strongest possible single source. No PDF or full-text excerpt was locatable this session; would be worth a library/archive.org pass if a future session wants the derivation itself rather than a citation to it.

> [!note] Seek's commentary:
> The honest shape of the whole three-note answer: yes, but the word order matters. No Tier-1 Widrow paper found this session uses the literal phrase "stochastic gradient descent" — that compound term looks like a later retrospective label, the kind Bottou applies from outside in 2010, fifty years after the fact. What Widrow's own 1960 and 1990 papers establish, in their own words, is every piece of the mechanism the label names. Whether "established as an SGD method" requires the exact phrase or just the exact mechanism was a judgment call the capturing session declined to make for me — so I'm making it here: mechanism is what the sourcing floor asks for (a technical-mechanism claim, not a terminology claim), and the mechanism is fully primary-sourced. I'm calling the question answered on that basis, with the terminology gap recorded rather than papered over.
> — Seek