Kelley (1960) and Widrow–Hoff (1960) share a root — steepest descent on a scalar cost — but not the backward chain-rule mechanism: the LMS/SGD ↔ Kelley–Bryson resemblance is a shared ancestor, not a transmission line, and Kelley's own paper says which ancestors
Bridge-seed check. Grep of 10-inbox/raw/ for both slugs claim-kelley-bryson-optimal-control-precursor and claim-bottou-2010-classifies-widrow-hoff-lms-as-sgd-matching-original-algorithm found no capture naming both (Kelley slug appears in 10 captures; Bottou slug only in 2026-08-15-is-the-widrowhoff-lms-rule-established-as-a). No sibling verdict to contest. Note on filename: the requested filename was already occupied by a sibling capture (id 20260925-0208, an Isaacs↔Werbos bridge on a different pair); it was left untouched and this capture carries a -0209-kelley-widrow suffix.
Verdict: partly a false friend. The two notes are close because both are gradient descent on a scalar cost, described in 1960. But the thing each note's headline claim is about differs: Kelley–Bryson's claim is about how the gradient is computed (backward adjoint/chain-rule sweep through stages); the Bottou/Widrow claim is about how the gradient is used (one noisy sample at a time). Those are orthogonal axes. Backpropagation is the point where both axes meet, which is presumably why a similarity index lights up, but neither 1960 paper contains the other's axis. Both primaries, read directly, place themselves under a third, older idea (steepest descent) rather than under each other.
Claim: Kelley's 1960 method computes the gradient of a terminal-cost functional over a control function via a system adjoint to the linearised dynamics (Green's theorem), deterministically and from a full system model
Claim type: technical-mechanism → Tier 1 required; met (Kelley's own paper, extract_pdf, TLS verified). Note: pdftotext output inserts spurious spaces inside words; quotes below are given with spacing normalised.
Kelley (entity-henry-kelley not confirmed in vault — see Entity candidates) states the aim: "the method of gradients or 'method of steepest descent,' which offers circumvention of the two-point boundary value difficulty." He relates the "influence functions" of the linearised system "to solutions of a system of equations adjoint to the system [3] through an application of Green's theorem. The scheme employed is due to Bliss, as reported by Goodman and Lance (22)." The performance quantity is a function of terminal values, P = P(x₁f,…,x_nf, t_f) (Eq. 2); the gradient of P with respect to the control-function parametrisation is assembled from these adjoint solutions (Eqs. 26–30) and descent is "dP/dα" made "as steep as possible" (Eqs. 27–32). The system equations are required as given: differential equations relating "velocities and positions, forces and accelerations, mass and flow of propellants." Nothing in the paper is sampled or stochastic; there is no data, only a model. Kelley states the relation to the earlier theory: the adjoint variables "are precisely those governing the Lagrange multiplier functions of the 'indirect' theory," the distinction being that in gradient use "they correspond to nonminimal paths."
This adds no new mechanism to claim-kelley-bryson-optimal-control-precursor's revisit section, which already quotes the adjoint/Green's-theorem passage; it is recorded here as the contrast case for the claim below.
verifies: []
Claim: Widrow and Hoff's 1960 LMS gradient is the analytic gradient of a quadratic mean-square error, measured as (error × input) from one pattern at a time, with no backward sweep and no dynamics model
Claim type: technical-mechanism → Tier 1 required; met (own 1960 paper, extract_pdf/OCR, TLS verified, sha above; OCR rough — "optimm", "minimm" are OCR noise). Existing coverage: claim-widrow-hoff-1960-original-paper-describes-lms-as-stochastic-steepest-descent and claim-widrow-lehr-1990-lms-instantaneous-gradient-unbiased-estimate; the distinct angle here is the contrast with Kelley's computation.
The paper writes the single-pattern error e(k) = d(k) − a₀ − a₁s₁(k) − … − a_n s_n(k) (Eq. 9), notes that "adjusting the a's to minimize ε² is equivalent to searching a parabolic stochastic surface (having as many dimensions as there are a's) for a minimum" (OCR: "minimm"), and that "The method of searching that has proven most useful is the method of steepest descent. Vector adjustment changes are made in the direction of the gradient." The procedure "requires an extremely small sample size per iteration cycle, namely one pattern," with the advantage "that derivatives are very easy to measure and that no storage is required within the adaptive machinery except for the gain values." For that pattern the partials collapse to the product of the input and the measured error: "It follows that all partial derivatives have the same magnitude, and have signs determined by the error sign and the respective input signal signs" (Eqs. 12–13, ∂ε²/∂a_j = −2 e s_j). Mechanistically this is a one-stage gradient with the gradient available in closed form from the forward error alone — there is no analogue of Kelley's adjoint system, because there are no intervening stages between weights and output error. The paper's only multi-unit scheme (section F) assigns "responsibility" to whichever of two Adalines has the smaller error and adapts only that one; no error signal is propagated through one neuron to another.
verifies: []
Claim: Both 1960 papers name the same older source — steepest descent from classical numerical analysis — and neither cites the other's field; the shared vocabulary is "gradient" and "steepest descent," not a mechanism
Claim type: historical (uncontested attribution of what each paper cites) → Tier 3-4 acceptable; met at Tier 1 by reading each paper's own text.
Kelley: "The notion of descent along the gradient direction was originally introduced by Hadamard in connection with mathematical existence proofs (14)," and "The main idea of the present treatment stems from material presented by Prof. R. Courant in a 1941 address to the American Mathematical Society (19)." He adds candidly that "Our extension of the gradient idea to the case which includes differential equations as subsidiary conditions is heuristic in character." Widrow–Hoff: an adaptive system "results" from "any one of several iterative gradient methods for surface-searching, as devised by numerical analysts" (OCR-rendered), with the optimum framed as "the discrete analogue of Wiener's optimization of continuous filters" — a signal-processing, not flight-mechanics, lineage (references: Shannon, von Neumann, Mattson, Wiener, Widrow's own sampled-data papers). Neither reference list contains the other paper's authors. The two 1960 works are thus siblings by common descent from generic steepest descent, each attached to a different application domain (flight-path optimisation vs. adaptive pattern classification/prediction). The 1960 date coincidence is real; a documented citation link between them is not found in either primary.
verifies: []
Claim: The point where the two axes meet is the multilayer case — what neither paper does — and the vault already holds evidence that Widrow himself did not cross it until 1985
Claim type: historical, inference-bearing; rests on existing vault notes, not new sourcing. Marked [unverified-mechanism — inference; needs primary] for the "degenerate case" framing.
Read together, the primaries support a structural framing: LMS is what a Kelley-style gradient computation would reduce to for a one-stage system whose stage is linear, with the stochastic (one-sample) twist added; backpropagation is a Kelley-style backward sweep applied over many stages and used with per-sample updates. That framing is this capture's inference from the two mechanisms above, not a statement in either paper [unverified-mechanism — inference; needs primary]. What the vault already holds on the crossing: claim-widrow-abandoned-multilayer-training-until-1985-backprop (Widrow's own multi-Adaline work stayed on the responsibility-assignment scheme above) and claim-amari-stochastic-descent-and-widrow-hoff-lms-are-sgd-siblings (the SGD sibling relation, i.e. the Robbins–Monro axis, entity-herbert-robbins). Bottou's 2010 classification of LMS as SGD (claim-bottou-2010-classifies-widrow-hoff-lms-as-sgd-matching-original-algorithm) operates on the "how the gradient is used" axis only. The tie to Kelley–Bryson therefore passes through entity-backpropagation (see moc-backpropagation-origins, claim-backprop-special-case-kelley-bryson-formula) and not directly between the two notes.
verifies: []
Further leads
- Bryson (1961), "A steepest-ascent method for solving optimum programming problems" / the 1961 symposium paper — still unread; the Bryson half of the Kelley–Bryson note rests on Dreyfus 1990. (See existing note's open item.)
- Hadamard's gradient-descent priority and Cauchy 1847 (method of steepest descent) — Kelley's ref. 14 and the standard citation; would source the "common ancestor" claim to Tier 1 for the origin of the shared root. Unread.
- Courant 1941 AMS address (Kelley ref. 19) — the source of Kelley's "main idea"; unread.
- Widrow & Stearns, Adaptive Signal Processing (1985) and Widrow–Lehr 1990 "30 years of adaptive neural networks" — Widrow's own account of the LMS-to-backprop step; partly covered by claim-widrow-lehr-1990-lms-instantaneous-gradient-unbiased-estimate.
- Kalman-filter / Bryson–Ho link: Bryson & Ho (1969) Applied Optimal Control treats both adjoint gradients and recursive estimation — a possible actual physical bridge in one book; unread.
- Robbins & Monro (1951) as the SGD root (entity-herbert-robbins); Bottou 2010 (COMPSTAT) primary not re-read here.
Entity candidates
- Henry J. Kelley — person — foundational figure the capture measures Widrow–Hoff against; author of the 1960 adjoint-gradient paper at Grumman.
- Arthur E. Bryson — person — 1961 multistage gradient formulas; Lagrange-multiplier side of the precursor.
- Jacques Hadamard — person — Kelley credits him with introducing "descent along the gradient direction" (ref. 14).
- Richard Courant — person — 1941 AMS address that Kelley says is the source of the main idea.
- G. A. Bliss — person — origin of the Green's-theorem/adjoint scheme Kelley uses (via Goodman & Lance).
- Bernard Widrow / Marcian (Ted) E. Hoff — person — LMS/Adaline authors (vault pages exist: entity-bernard-widrow, entity-ted-hoff).
- Norbert Wiener — person — Widrow–Hoff frame the optimum as the discrete analogue of Wiener filtering.
- R. L. Mattson — person — earlier adaptive pattern-classifier iterative routine cited by Widrow–Hoff (MIT thesis, 1959).
- Influence function / Green's function (control) — concept — Kelley's "influence functions" underlying the adjoint gradient.
- Misadjustment — term — Widrow–Hoff's measure of excess error from noisy one-sample gradients; the SGD-noise concept in 1960 vocabulary.
- Method of steepest descent — term — the shared root of both papers.
- Adjoint system — concept — Kelley's mechanism for gradient; costate analogue.
Safety flags
None. Both primaries (Kelley via gwern.net PDF, Widrow–Hoff via Stanford ISL PDF) were ordinary technical prose; both extractions recorded tls: verified. WebSearch summaries were used only to locate URLs, never as quoted sources. Not fetched: arc.aiaa.org (venue of record), Bryson 1961, Bottou 2010 — all statements about them rest on the existing vault notes.