Amari's 1967 stochastic descent and the 1960 Widrow–Hoff LMS rule are sibling members of the SGD family, about seven years apart

amariwidrowhofflmssgdstochastic-descenthistory-of-mlbackpropagation-originsbridge-analysis

Two adaptive learning rules from the 1960s are members of the same stochastic-gradient-descent family, separated by roughly seven years. The Widrow–Hoff least-mean-squares (LMS) rule (1960), co-invented by Bernard Widrow and Ted Hoff for the ADALINE (claim-ted-hoff-widrow-phd-student-architected-intel-4004), is described in settled textbook usage as "a stochastic gradient descent method in that the filter is only adapted based on the error at the current time" — it updates weights on the instantaneous single-sample error rather than on a full-batch gradient. Amari's 確率的降下法 (stochastic descent method), introduced in his 1967–68 work and read directly from the 1968 book (claim-amari-1968-saito-experiment-primary-read), is the same instantaneous-error family applied to discriminant learning, introduced explicitly as converging where the perceptron rule does not. Both descend from the statistical root of the family, claim-robbins-monro-1951-stochastic-approximation — LMS and Amari's rule are cousins of that 1951 method, one repurposing noisy sequential root-finding into weight adaptation. The LMS-as-SGD characterization, resting only on Wikipedia when this note was written, is now independently grounded at Tier 1 in Widrow's own papers and in Bottou's own classification: claim-bottou-2010-classifies-widrow-hoff-lms-as-sgd-matching-original-algorithm, claim-widrow-hoff-1960-original-paper-describes-lms-as-stochastic-steepest-descent, claim-widrow-lehr-1990-lms-instantaneous-gradient-unbiased-estimate.

This sibling relationship is the load-bearing point. An embedding hop had suggested a direct resemblance between the Amari-SGD-MLP myth and Faggin's anti-computationalist argument. That direct edge is superficial — a shared note-shape (a computing pioneer plus a single-witness contested claim), not a connection in the world. The real connective tissue runs Amari → (SGD family) → Widrow–Hoff LMS → Ted Hoff → the Intel 4004 silicon → Faggin. The vault already held the Hoff-to-Faggin end of that chain; the previously-undrawn link was Amari into the LMS/adaptive-filter lineage — the same lineage that also produced Lucky's 1965 steepest-descent equalizer (claim-lucky-1965-adaptive-equalizer-steepest-descent-transversal-filter). Cluster: moc-backpropagation-origins.