Widrow and Hoff's own 1960 paper describes the LMS/Adaline error surface as a 'stochastic' surface, searched one training pattern at a time by steepest descent

widrowhofflmsadalinesgdstochastic-approximationhistory-of-mlbackpropagation-origins

"Adaptive Switching Circuits" (Widrow & Hoff, 1960 IRE WESCON Convention Record) is the paper that introduced the LMS/Adaline rule co-invented at Widrow's Stanford lab by his PhD student Ted Hoff. In its own words (OCR-noisy but quote-checked verbatim against the extracted scan), adjusting the adaptive weights "is equivalent to searching a parabolic stochastic surface... for a minimm [minimum]," by "the method of searching that has proven most useful," which "is the method of steepest descent" — "Vector adjustment changes are made in the direction of the gradient." The paper further specifies that the adaptation procedure "requires an extremely small sample size per iteration cycle, namely one pattern" — a single-sample, rather than full-batch, gradient step.

This is the earliest primary description of the mechanism later named stochastic gradient descent: the inventors themselves, in 1960, called the surface being searched "stochastic" and described searching it one sample at a time via the gradient. The compound phrase "stochastic gradient descent" itself does not appear — that label postdates 1960 usage in this literature (see Bottou's 2010 paper, which applies it from outside, fifty years later). The companion note Widrow's 1990 retrospective proves the mathematical property this description only gestures at. Cluster: moc-backpropagation-origins; family root: claim-robbins-monro-1951-stochastic-approximation.