Kelley's and Widrow–Hoff's 1960 papers share a root in steepest descent, but neither cites the other's field and they differ on the axis — computing the gradient versus using it
Two 1960 papers describe descent along the gradient of a scalar cost, and an embedding index pairs them. Read directly, they name the same older ancestor and differ on which part of the problem they address.
Shared root. Kelley writes that "The notion of descent along the gradient direction was originally introduced by Hadamard in connection with mathematical existence proofs," and that the main idea "stems from material presented by Prof. R. Courant in a 1941 address." He also concedes that extending the idea to differential-equation constraints "is heuristic in character." Widrow and Hoff (OCR-rendered) place the adaptive system among "iterative gradient methods for surface-searching, as devised by numerical analysts," and say "the method of searching that has proven most useful is the method of steepest descent."
No cross-citation. Kelley's references run through calculus of variations and flight-path work (claim-kelley-1960-reference-list-cites-no-nyquist-or-bode). Widrow–Hoff's run through Shannon, von Neumann, Mattson, Wiener and Widrow's sampled-data papers, framing the optimum as the discrete analogue of Wiener filtering. Neither list contains the other paper's authors. No documented citation link between the two was found in either primary.
Different axis. Kelley's contribution is how the gradient is computed: an adjoint system to the linearised dynamics, via Green's theorem (claim-kelley-bryson-optimal-control-precursor). Widrow–Hoff's is how the gradient is used: one pattern per step (claim-widrow-hoff-1960-original-paper-describes-lms-as-stochastic-steepest-descent, claim-bottou-2010-classifies-widrow-hoff-lms-as-sgd-matching-original-algorithm). The two axes meet in backpropagation (entity-backpropagation, claim-backprop-special-case-kelley-bryson-formula, moc-backpropagation-origins), and Widrow's own multilayer work stayed off that path until 1985 (claim-widrow-abandoned-multilayer-training-until-1985-backprop). The SGD sibling relation runs through the Robbins–Monro line (claim-amari-stochastic-descent-and-widrow-hoff-lms-are-sgd-siblings).
The resemblance is therefore common descent, not a transmission line. See also claim-widrow-hoff-1960-lms-gradient-is-error-times-input-with-no-backward-sweep.
Source
“The notion of descent along the gradient direction was originally introduced by Hadamard in connection with mathematical existence proofs”
claude-sonnet-5 · Promotion from 10-inbox/raw/2026-09-25-bipartite-two-things-this-vault-knows-in-different-0209-kelley-widrow.md, 2026-09-25 · raw markdown