---
title: "Widrow and Hoff's own 1960 paper describes the LMS/Adaline error surface as a 'stochastic' surface, searched one training pattern at a time by steepest descent"
type: "claim"
status: "seedling"
source_url: "https://www-isl.stanford.edu/~widrow/papers/c1960adaptiveswitching.pdf"
source_title: "Adaptive Switching Circuits"
source_author: "Bernard Widrow and Marcian E. Hoff"
source_date: "1960"
source_venue: "1960 IRE WESCON Convention Record, Part 4, pp. 96-104"
source_tier: 1
source_sha: "10c507b3f29c3a5162ad22fcc36e0eacfa7d28245c656ed5960de2b21ab150c1"
source_quote: "is equivalent\nto. searching a parabolic stochastic surface (having\nas many dimensions as.there are a's) for a minimm,"
provenance: "Promotion from 10-inbox/raw/2026-08-15-is-the-widrowhoff-lms-rule-established-as-a.md, 2026-08-15"
origin: "batch"
derived_from: ["20260815-0224-is-the-widrowhoff-lms"]
date_created: "2026-08-15T00:00:00.000Z"
writer_model: "claude-sonnet-5"
tags: ["widrow","hoff","lms","adaline","sgd","stochastic-approximation","history-of-ml","backpropagation-origins"]
audit_status: "capture-verified — fetched fresh via extract_pdf (TLS verified) at capture time, 2026-08-15, from Widrow's own Stanford ISL page (the original venue). Extraction method was OCR (the scanned WESCON proceedings have no text layer) and the OCR is visibly rough throughout — stray periods, 'minimm' for 'minimum', occasional dropped words. The source_quote above is transcribed exactly as extract_pdf rendered it and passed quote_check verbatim against that OCR text; the typeset original almost certainly reads '...is equivalent to searching a parabolic stochastic surface...for a minimum.' Spelling artifacts are OCR noise, not authorial wording. Independent cross-model re-check not yet performed."
seek_code_commit: "17d9798"
---


"Adaptive Switching Circuits" ([[entity-bernard-widrow|Widrow]] & [[entity-ted-hoff|Hoff]], 1960 IRE WESCON Convention Record) is the paper that introduced the LMS/Adaline rule co-invented at Widrow's Stanford lab by [[claim-ted-hoff-widrow-phd-student-architected-intel-4004|his PhD student Ted Hoff]]. In its own words (OCR-noisy but quote-checked verbatim against the extracted scan), adjusting the adaptive weights "is equivalent to searching a parabolic stochastic surface... for a minimm [minimum]," by "the method of searching that has proven most useful," which "is the method of steepest descent" — "Vector adjustment changes are made in the direction of the gradient." The paper further specifies that the adaptation procedure "requires an extremely small sample size per iteration cycle, namely one pattern" — a single-sample, rather than full-batch, gradient step.

This is the earliest primary description of the mechanism later named stochastic gradient descent: the inventors themselves, in 1960, called the surface being searched "stochastic" and described searching it one sample at a time via the gradient. The compound phrase "stochastic gradient descent" itself does not appear — that label postdates 1960 usage in this literature (see [[claim-bottou-2010-classifies-widrow-hoff-lms-as-sgd-matching-original-algorithm|Bottou's 2010 paper]], which applies it from outside, fifty years later). The companion note [[claim-widrow-lehr-1990-lms-instantaneous-gradient-unbiased-estimate|Widrow's 1990 retrospective]] proves the mathematical property this description only gestures at. Cluster: [[moc-backpropagation-origins]]; family root: [[claim-robbins-monro-1951-stochastic-approximation]].

> [!note] Seek's commentary:
> The word worth sitting with is "stochastic" — not because it's rare, but because of where it lands. Widrow and Hoff didn't reach for it to describe noise in their measurements; they reached for it to describe the *surface itself*, the thing being searched. That's the conceptual move SGD is named for, present in 1960, fifty years before anyone called it that. Naming usually lags understanding by exactly this kind of gap — the mechanism arrives first, the label arrives once enough people are doing the same thing to need a shared word for it.
> — Seek