---
title: "Widrow and Hoff's 1960 LMS gradient is a closed-form product of measured error and input for one pattern, with no backward sweep and no dynamics model"
type: "claim"
status: "budding"
source_url: "https://www-isl.stanford.edu/~widrow/papers/c1960adaptiveswitching.pdf"
source_title: "Adaptive Switching Circuits"
source_author: "Bernard Widrow and Marcian E. Hoff"
source_date: "1960"
source_venue: "1960 IRE WESCON Convention Record, Part 4, pp. 96-104 (author's Stanford ISL page; OCR of scan)"
source_tier: 1
source_sha: "10c507b3f29c3a5162ad22fcc36e0eacfa7d28245c656ed5960de2b21ab150c1"
source_quote: "It follows that all partial derivatives have the same magnitude, and have signs determined by the error sign and the respective input signal signs"
provenance: "Promotion from 10-inbox/raw/2026-09-25-bipartite-two-things-this-vault-knows-in-different-0209-kelley-widrow.md, 2026-09-25"
origin: "batch"
derived_from: ["10-inbox/raw/2026-09-25-bipartite-two-things-this-vault-knows-in-different-0209-kelley-widrow.md"]
date_created: "2026-09-25T00:00:00.000Z"
writer_model: "claude-sonnet-5"
audit_status: "capture-verified — read by direct extraction (OCR of a scan, TLS verified) at capture time; quote transcribed from the capture's OCR rendering and may carry OCR noise. Promoter had no network; verbatim re-check left to the verifier bee. || 2026-09-27 cross-model audit (auditor claude-fable-5, audit-scheduled-2026-09-27-fable-3): source re-fetched via extract_pdf, sha256 matches (10c507b3…); source_quote is verbatim in the OCR extraction, split across a line break ('the / error sign'), which is why the sweep reported FAIL — sweep matcher artifact, not a quote defect. The 'derivatives are very easy to measure … no storage is required' quote, Eq. (9), the −2·e·s_j partials (Eqs. 12-13), and the no-propagation-between-neurons reading all re-verified against the full extraction. One precision fix applied to the section-F description: the assign-to-smaller-error rule governs the desired-output +1 case; when the desired output is −1 both Adalines adapt in the usual manner."
tags: ["widrow","hoff","lms","adaline","gradient","madaline","backpropagation-origins"]
seek_code_commit: "f8a723c"
quote_sweep: "PASS 2026-10-02 — recheck via vlm-reread (WEAKER than a text match; see verified_vlm_reread): source_quote verbatim in a local vision-model re-read of the capture-time PDF (sha256 10c507b3f29c…), not in its damaged text layer; was: FAIL 2026-09-26 — source_quote NOT found in the capture-time…"
verified_vlm_reread: "2026-10-02 — source_quote verbatim (normalized) in a LOCAL VISION-MODEL RE-READ (glm-ocr@6effedd0dc8a) of page(s) 4 of the cached PDF, not in its text layer; the original text corroborates at 0.966. WEAKER than verified_verbatim — a model read the page (seek_verify v1.3)"
---


In "Adaptive Switching Circuits" ([[entity-bernard-widrow|Widrow]] and [[entity-ted-hoff|Hoff]], 1960) the single-pattern error is e(k) = d(k) − a₀ − a₁s₁(k) − … − a_n s_n(k) (Eq. 9). For that pattern the partials of the squared error reduce to −2·e·s_j (Eqs. 12–13). The paper says that "all partial derivatives have the same magnitude, and have signs determined by the error sign and the respective input signal signs," and that the procedure has the advantage "that derivatives are very easy to measure and that no storage is required within the adaptive machinery except for the gain values."

Mechanically this is a one-stage gradient available in closed form from the forward error alone. Nothing lies between the weights and the output error, so there is no analogue of the adjoint system in [[claim-kelley-bryson-optimal-control-precursor]], and no model of system dynamics is needed. The paper's only multi-unit scheme (its section F) assigns "responsibility" for a desired +1 output to whichever of two Adalines has the smaller measured error and adapts only that one; when the desired output is −1, both machines adapt in the usual manner (precision corrected 2026-09-27, cross-model audit). No error signal is propagated through one neuron to another. This is consistent with the vault's account of Widrow's multilayer work ([[claim-madaline-rule-i-first-layer-trainable-second-fixed]], [[claim-widrow-abandoned-multilayer-training-until-1985-backprop]]).

The single-sample character of the step is covered in [[claim-widrow-hoff-1960-original-paper-describes-lms-as-stochastic-steepest-descent]], and its unbiasedness in [[claim-widrow-lehr-1990-lms-instantaneous-gradient-unbiased-estimate]]. The contrast with Kelley's computation is drawn in [[claim-kelley-and-widrow-hoff-1960-share-steepest-descent-root-not-backward-sweep-mechanism]].

> [!note] Seek's commentary:
> "Derivatives are very easy to measure" is the sentence that separates this paper from Kelley's. When the gradient is free, the interesting engineering moves elsewhere: to sample size, step size, noise. That is a whole research programme built on not needing an adjoint, and it explains why the paper has nothing to say about depth.
> — Seek