What duality argument does the Argmin "Mates of Costate" post make connecting adjoint variables/co-state equations to Bryson (1961) and Kalman (1960), and does it support a backprop lineage through optimal control?

Topic question (harvested from): 2026-06-30-what-role-did-kelley-1960-and-bryson-1961-optimal-control-work-play-in-the-backprop-lineage

Related notes: claim-kelley-bryson-optimal-control-precursor · 20260706-1505-did-kelley-1960-derive · 20260707-0208-did-bryson-hos-applied · 20260706-1508-what-formal-derivation-does · backpropagation · optimal-control · adjoint-method · Kalman-filter · Lagrangian-duality · Kelley-Bryson

The source is Ben Recht's 2016 blog post "Mates of Costate" on the arg min blog (the WordPress-era version, archived at the URL above; the blog now continues on Substack at argmin.net). Recht is a named, credentialed author (professor of EECS and statistics, UC Berkeley) writing on his own venue about his own technical argument — Tier 1 per the sources.md rubric. URL resolves and was fetched directly; quoted passages below were reproduced identically across four independent fetch passes and match an independent prior capture in this vault that fetched the same page separately, which is good evidence against fetch-model hallucination of the quotes.

Part 1 — The duality argument itself (technical-mechanism claim, Tier 1 required — met)

Claim type: Specific technical-mechanism claim → floor Tier 1–2 required. Source: Tier 1 (Recht's own derivation, own venue). ✓ clears floor.

The post sets up a discrete-time linear-quadratic optimal control problem — a linear dynamical system with cost function summed over a time horizon, subject to the dynamics as a constraint — and forms a Lagrangian with multipliers attached to the dynamics constraint at each time step. Recht's own exact words describing what those multipliers become:

"the Lagrange multipliers... become a trajectory of a related linear system called the adjoint or dual system"

The dynamics of that dual system are stated as running backward in time with the transpose of the forward system's state-transition matrix:

"The dynamics are linear in the costate p_t, with time running in reverse and the state transition matrix being the transpose (also known as the adjoint) of A."

Structurally, the argument is: taking the stationarity condition of the Lagrangian with respect to the multipliers p_t recovers the original forward dynamics constraint (the "forward pass," computing the state trajectory x_t going forward in time); taking the stationarity condition with respect to the state x_t yields a second, separate recursion for p_t that runs backward in time using the transpose/adjoint of the forward system matrix (the "backward pass," computing sensitivities). Recht states explicitly that this whole apparatus — deriving a backward adjoint recursion from Lagrangian stationarity conditions — is the method of adjoints, and that it is mathematically the same computation as neural-network backpropagation:

"A few years ago, Steve Wright introduced me to an older method from optimal control, called the method of adjoints, which is equivalent to backpropagation. It's also easier (at least for me) to derive. This is because the core of the method is Lagrangian duality, a topic at the foundation of everything we optimizers do."

So the duality argument, precisely: Lagrangian duality — treating the dynamics constraint as attached to the cost via multipliers turns one forward-only optimization problem into a primal/dual pair of linear recursions, one running forward (the original state dynamics) and one running backward (the adjoint/costate dynamics, using the transpose of the forward transition operator). Recht's claim is that this primal-forward/dual-backward pair is structurally identical to backprop's forward-activation-pass / backward-delta-pass — the "duality" is between the state trajectory and the costate trajectory, not a duality between optimal control and machine learning as fields per se; the ML connection is a claimed instance of the same underlying mathematical structure.

Part 2 — The Bryson (1961) connection

Claim type: Historical/attribution claim. Load-bearing (this is the priority claim the post rests on) → escalated to Tier 1–2 required, and Tier 1 is what's available (Recht's own words) — though note below on secondary attribution within the quote.

Exact quote:

"As I mentioned already, the method of adjoints originates in the study of controls. According to Dreyfus, this was first proposed by Bryson in a paper called 'A Gradient Method for Optimizing Multi-Stage Allocation Processes' that appeared in the Proceedings of the Harvard University Symposium on Digital Computers and Their Applications in 1961. I was unable to find this proceedings in our Engineering Library, but the Lagrangian derivation plays a prominent role in Bryson and Ho's 1968 book Applied Optimal Control."

Two things worth flagging precisely, because they affect how much weight this specific attribution can bear:

• Recht explicitly marks the "first proposed by Bryson... 1961" claim as attributed to Dreyfus ("According to Dreyfus"), not as something Recht verified against the original 1961 proceedings himself — he says outright he could not locate that proceedings. So within this post, the priority claim is Recht relaying a secondary source's claim, not independent archival confirmation. (A separate capture in this vault, 20260707-0208-did-bryson-hos-applied, independently checked Dreyfus's own 1990 retrospective paper and found it corroborates Bryson's 1961 priority in Dreyfus's own words — so the claim is cross-corroborated elsewhere in the vault, just not verified within this post by Recht himself.)
• Recht dates Bryson and Ho's book "1968" here; the standard bibliographic date for Applied Optimal Control is 1969 (also noted as a discrepancy in 20260707-0208-did-bryson-hos-applied). Preserved as-is since it's the primary quote; flagged rather than silently corrected. [minor discrepancy — not resolved, flagged]

What the book is cited as evidence of, in this post: a stand-in documentary source for the Lagrangian/adjoint derivation, used because Recht could not obtain the original 1961 conference paper — not a claim about curriculum or teaching (that framing is examined separately in 20260707-0208-did-bryson-hos-applied and found not to originate here).

Part 3 — The Kalman (1960) connection

Claim type: Historical claim (dating/sequencing) plus a technical-mechanism claim (Kalman filtering as an instance of the same cost-function framework). Both escalate to Tier 1–2 floor as load-bearing/technical. Source: Tier 1. ✓ clears floor.

Two distinct claims are made about Kalman:

(a) Chronological sequencing. Exact quote:

"Note that Bryson's paper appeared only a year after as Kalman's absurdly influential 'A New Approach to Linear Filtering and Prediction Problems'. This use of duality was very much at the birth of modern control theory."

This places Kalman's paper (1960, standardly A New Approach to Linear Filtering and Prediction Problems, ASME Journal of Basic Engineering) one year before Bryson's 1961 conference paper, and frames both as part of the same founding moment of modern control theory built on duality arguments.

(b) Structural/mechanism claim. The post argues Kalman filtering is a special case of the same LQ cost-function-plus-adjoint machinery just derived, not a separate topic:

"In the special case where the cost involves tracking an observation y_t, we arrive at the cost function of Kalman's Filter."

and

"One could solve the Kalman Filtering problem by performing gradient descent on the cost and computing the gradient via the method of adjoints."

So "this use of duality" in the Kalman-Bryson sentence refers to the same Lagrangian-duality/adjoint-system argument from Part 1 — Recht's point is that Kalman filtering and Bryson's gradient method are two applications, arriving within a year of each other, of one shared duality technique (primal state trajectory / dual costate trajectory) at the founding of modern control theory.

Part 4 — Does the post support a "backprop lineage through optimal control"? (the core question)

Verdict: Partially — and the distinction matters. The post makes a strong, explicit mathematical equivalence claim ("the method of adjoints... is equivalent to backpropagation") and situates that method's origin decades earlier, in Bryson (1961) and the surrounding control-theory literature (Kalman 1960, Bryson & Ho 1968/1969). That much is squarely present and quoted above.

What the post does not do, explicitly, is assert a causal historical-transmission lineage — i.e., it does not claim that Rumelhart, Hinton, Williams, or other neural-network-era inventors of backpropagation actually read or derived their algorithm from Bryson's or Kalman's work. The framing Recht uses for his own encounter with the connection is discovery/parallel, not transmission:

"A few years ago, Steve Wright introduced me to an older method from optimal control, called the method of adjoints, which is equivalent to backpropagation."

This is Recht describing his own re-discovery of an equivalence between two literatures, decades after both existed — not an argument that backprop's 1980s popularizers derived their method from control theory. The post's own explicit stated next step, at its close, is likewise about mathematical structure, not historical transmission:

"In the next post, I will show how the method of adjoints is intimately connected to the KKT conditions... And I'll also describe other algorithms that naturally arise when one views a cascade of function compositions as a dynamical system."

Additionally: the post does not mention Kelley (1960) at all — a check specifically run against the full text found no occurrence of "Kelley." This matters for the topic question's frame (which pairs "Kelley 1960 and Bryson 1961" as joint precursors, per the harvested topic-question title): this particular post's chain of attribution runs Kalman (1960) → Bryson (1961) → Bryson & Ho (1968/1969) only, omitting the Kelley side of the standardly-cited joint Kelley–Bryson priority (Kelley used adjoint equations and Green's theorem independently and contemporaneously — documented via Dreyfus's own words in 20260707-0208-did-bryson-hos-applied, not this post).

Summary answer: The post supports the mathematical half of a backprop-optimal-control lineage claim (an explicit, worked equivalence, Tier 1, grounded in Lagrangian duality) but does not itself argue the historical-transmission half (that backprop's NN-era inventors derived their method from, or were causally influenced by, this control-theory work). The historical-transmission question is [unverified in this source — the post argues equivalence and independent-parallel-discovery, not causal transmission; the transmission question is addressed by other sources in this vault, e.g. Dreyfus's own retrospective accounts, not by this post].

Further leads

• Two other Recht/argmin posts mention Bryson's adjoint method in passing without the Kalman/duality framing ("Fact, Fiction, and Forecast," https://www.argmin.net/p/fact-fiction-and-forecast, and "Reticulating Splines," https://www.argmin.net/p/reticulating-splines) — noted in 20260707-0208-did-bryson-hos-applied, not independently re-verified here.
• The post's own promised sequel (on the method of adjoints and KKT conditions) was not located/checked in this session — following the argmin blog forward from 2016-05-18 for that sequel post would round out the technical picture but is not needed to answer the core duality/lineage question here.
• Whether Recht elsewhere (post-2023 Substack era) revisits this argument with an explicit historical-transmission claim was not checked in this session.