Does Kelley-Bryson's discrete costate recursion reduce to Pontryagin's continuous adjoint equations in the small-step limit?

kelley-brysonpontryaginoptimal-controladjoint-methodcostatediscretizationneural-odesbackpropagationhistory-of-mlnumerical-analysis

Framing note: the vault already holds the "same algorithm or distant cousin" tension at the historical/mathematical level — claim-backprop-special-case-kelley-bryson-formula (Mizutani–Dreyfus–Nishio: backprop is a "simplified version"/"equivalent" to the Kelley-Bryson discrete gradient formula) versus claim-kelley-bryson-optimal-control-precursor (Schmidhuber: a precursor lacking backprop's efficiency) versus claim-neural-odes-cite-pontryagin-adjoint-method-as-backprop (Chen et al. 2018 name Pontryagin's continuous adjoint method, not Kelley-Bryson's discrete one, as the thing equal to backprop). This capture asks a narrower, checkable mathematical question underneath all three: does the discrete costate recursion (Kelley 1960, Bryson 1961, formalized by Recht's Lagrangian-duality argument) actually converge to Pontryagin's continuous-time adjoint ODE as the step size shrinks to zero — making them the same object under two discretizations — or is that convergence a vault assumption the control-theory and numerical-analysis literature does not actually support unconditionally?

Direct primaries for Kelley (1960) and Bryson (1961) were not re-read for this capture; it instead searches the control-theory and numerical-optimal-control literature that treats the discrete-vs-continuous maximum-principle question as its own research object, independent of the neural-network history.

Claim: The discretize-then-optimize recursion and the optimize-then-discretize continuous adjoint ODE are provably different computations, and their disagreement does not have to vanish as the step size shrinks

In neural-ODE training there are two ways to get a gradient through a numerically-discretized ODE layer: solve the continuous adjoint (costate) ODE backward in time and then discretize that ODE ("optimize-then-discretize," OTD — the move Chen et al. 2018 make, see claim-neural-odes-cite-pontryagin-adjoint-method-as-backprop); or differentiate the already-discretized forward recursion directly, which reproduces the Kelley-Bryson/backprop chain-rule recursion exactly ("discretize-then-optimize," DTO). Gholami, Keutzer & Biros (2019) prove these give literally different gradient expressions for an explicit Euler step (z1 = z0 + f(z0,θ)): DTO backpropagates ∂L/∂z1·(I + ∂f(z0,θ)/∂z0), while OTD's adjoint recursion computes α1·(I + ∂f(z1,θ)/∂z1) — "the results from DTO approach (Eq. 9), and OTD (Eq. 10) can be completely different. This is because in general ∂f(z1,θ)/∂z1 ≠ ∂f(z0,θ)/∂z0." They state the per-step discrepancy is formally small ("Except for rare cases [14], the error in OTD and DTO's gradient scales as O(dt)") but add immediately — in the paper's own words, kept verbatim rather than smoothed into the more intuitive-sounding "large step" reading — "Therefore, this error can become quite large for small time step sizes," and demonstrate empirically that for ReLU/non-ReLU activations and general convolution operators the backward-in-time (OTD-style) integration is numerically unstable, producing training that diverges outright rather than converging as resolution increases (Fig. 1, Fig. 3–5: RK45 integration of the OTD adjoint "leads to divergent training in the first epoch"). Their proposed fix, ANODE, is explicitly a DTO (discretize-then-optimize) method — i.e., it recovers the Kelley-Bryson-style discrete recursion rather than the continuous Pontryagin costate ODE — specifically because DTO "provides unconditionally accurate gradients" where OTD does not.

What kind of claim, and why this source clears the floor: specific technical-mechanism claim (how two gradient-computation procedures differ and under what conditions). Tier 1 — author's own arXiv preprint (Gholami, Keutzer, Biros; UC Berkeley / UT Austin), also published at IJCAI-19; read directly via extract_pdf, quote verified verbatim against the extracted text (quote_check: grounded).

Claim: A direct, unconditional discrete-time analogue of Pontryagin's full maximum principle is mathematically false in general — this was established by counterexample in classical control theory, independent of neural networks entirely

Before neural-ODE training ever raised the question, control theory already had this exact dispute and resolved it negatively in the general case. Kipka & Gupta (2017/2019, SIAM J. Control Optim.) summarize the state of the field: "the formulation of the maximum principle for discrete-time optimal control problems becomes quite tricky since the maximization condition cannot be expected to hold in general and in fact some of the early literature on this topic was mathematically incorrect... However, the maximization condition does hold under appropriate convexity assumptions on the dynamics and moreover an approximate maximization condition can be derived (under suitable assumptions) in the absence of such assumptions." The counterexamples this passage cites trace to Halkin's 1966 SIAM J. Control paper ("A Maximum Principle of the Pontryagin Type for Systems Described by Nonlinear Difference Equations," SIAM J. Control 4(1):90–111) and later work by Boltyanskii — this capture did not read Halkin or Boltyanskii directly, so the counterexamples themselves rest on Kipka & Gupta's secondary account, not a primary read. The point this establishes: it is specifically the maximization (Hamiltonian-optimality) condition of Pontryagin's apparatus — not the costate/adjoint equation alone — that fails to carry over to discrete time without extra convexity assumptions the continuous theory does not need. Butkovskiy is credited (per web secondary sources, not independently read) with first showing a direct extension of Pontryagin's principle to discrete systems is in general impossible.

What kind of claim, and why this source clears the floor: specific technical-mechanism claim about where exactly the discrete/continuous correspondence breaks (the maximization condition, under which assumptions). Tier 1 — Kipka & Gupta's own arXiv preprint (also published SIAM J. Control Optim. 57(4):2939–2961, 2019), read directly via extract_pdf, quote verified verbatim (quote_check: grounded). The attribution of the counterexamples specifically to Halkin (1966) and Boltyanskii is carried from this Tier-1 source's own citations, not independently verified against Halkin's or Boltyanskii's primary texts — flagged [unverified-mechanism — needs primary] for the Halkin/Boltyanskii attribution specifically, though the Tier-1-sourced general claim (no unconditional direct analogue exists) stands on its own.

Claim: Where smoothness, stability and the right discretization structure hold, convergence of the discrete costate recursion to the continuous adjoint IS a proven theorem — so the vault's intuition is right inside a narrower regime than it assumes

The positive case exists and is rigorous, not folklore. Ober-Blöbaum, Junge & Marsden (2008 preprint / 2011 ESAIM: COCV) prove, for Lagrangian optimal-control problems meeting a stated Smoothness assumption and a Coercivity assumption (ensuring a locally unique, Lipschitz-continuous control-from-costate map), that the discrete adjoint/costate recursion generated by a symplectic partitioned Runge-Kutta discretization of the state equations converges to the continuous costate system as the step size h goes to zero, with matching order: "If the symplectic partitioned Runge-Kutta discretization of the state system is of order κ and bi > 0 for each i, then the scheme for the adjoint system is again a symplectic partitioned Runge-Kutta scheme of the same order" (Theorem 3.15), building toward an explicit statement that "the purpose of this section is to establish the convergence of solutions of the discrete optimal control problem... to a solution of the Lagrangian optimal control problem... as the step size h goes to 0" (§3.6), via a general consistency-plus-stability convergence theorem credited to Hager (2001) and Dontchev, Hager & Veliov (2000).

What kind of claim, and why this source clears the floor: specific technical-mechanism claim (a stated convergence theorem and its hypotheses). Tier 1 — co-authored by Jerrold Marsden; read directly via extract_pdf from arXiv (TLS verified), quotes verified verbatim (quote_check: grounded). A near-identical typeset copy is also hosted on co-author Marsden's own Caltech CDS group wiki (cds.caltech.edu/~marsden/wiki/uploads/dmoc/papers/ObJuMa08.pdf) — confirmed to be the same paper (matching title/author/abstract header) but fetched over an unverified TLS connection, so per the safety spec this mirror is not used as the citation of record or a sole source for any claim here; the arXiv copy (verified TLS) is the read of record, with the published venue (ESAIM: Control, Optimisation and Calculus of Variations 17(2):322–352, 2011, doi:10.1051/cocv/2010012) recorded as source_venue_3.

Core question status

Partially resolved, with a precise shape rather than a flat yes/no. The vault's two notes (claim-backprop-special-case-kelley-bryson-formula and claim-neural-odes-cite-pontryagin-adjoint-method-as-backprop) are not simply in tension — they describe two ends of a genuine conditional: the discrete Kelley-Bryson costate recursion does converge to Pontryagin's continuous adjoint ODE as the step size shrinks, provably, under stated smoothness/coercivity/discretization-structure hypotheses (claim 3); but there is no unconditional version of that convergence, either in the general discrete-time Pontryagin maximum principle (claim 2, a classical-control-theory result predating neural networks) or in the specific neural-ODE setting with realistic nonsmooth activations and naive discretizations (claim 1, where the two computations can diverge outright rather than converge). "Same object under two time-discretizations" is correct as a limiting statement inside a regularity regime; "family resemblance" is the more accurate description once that regime's hypotheses are dropped — which is exactly what happens in the ReLU/non-convex settings neural networks actually use.

Further leads

• Halkin, H. (1966), "A Maximum Principle of the Pontryagin Type for Systems Described by Nonlinear Difference Equations," SIAM J. Control 4(1):90–111 — the original counterexample paper; not read directly (paywalled via SIAM/DOI 10.1137/0304009), worth a primary read in a future session.
• Hager, W.W. (2000), "Runge-Kutta methods in optimal control and the transformed adjoint system," Numerische Mathematik 87:247–282 — the result Ober-Blöbaum/Junge/Marsden build on for order-of-consistency between discrete and continuous adjoint systems; paywalled at Springer, not independently read; Hager's own faculty page (people.clas.ufl.edu/hager/optimal-control/) lists it without a direct PDF link.
• Butkovskiy is credited (via secondary web sources, not independently verified) as the first to show a direct discrete analogue of Pontryagin's principle is impossible in general — worth chasing the primary citation.
• Onken & Ruthotto, "Discretize-Optimize vs. Optimize-Discretize for Time-Series Regression and Continuous Normalizing Flows" (arXiv:2005.13420) — a second, independent ML paper on the same OTD/DTO distinction; not read for this capture.
• Holtzman is noted (secondary web source) to have generalized Halkin's maximum principle for difference equations by relaxing the convexity requirement to "directional convexity" — not independently verified.

Entity candidates

• Hubert Halkin — person — the foundational figure the modern discrete-time Pontryagin maximum principle literature measures itself against; his 1966 counterexample is the reason "does the discrete analogue hold" became its own research question rather than an assumed formality. Flagged first, per the known blind spot about under-catching the older figure a priority/ ancestry claim rests on.
• Lev Pontryagin — person — already present via other claim-notes (claim-pontryagin-worked-blind-via-mothers-spoken-symbol-glosses etc.) but worth a dedicated entity page given how load-bearing "Pontryagin's maximum principle" is across this whole cluster; no entity-pontryagin page currently exists in the vault.
• William W. Hager — person — control theorist whose Runge-Kutta/adjoint consistency results (Hager 2000, 2001) underlie the one rigorous convergence proof this capture found.
• Jerrold E. Marsden — person — co-author of the DMOC convergence paper; major figure in geometric mechanics/variational integrators, a different lineage than Kelley-Bryson-Dreyfus that arrives at the same discrete/continuous adjoint question from the mechanics side.
• Amir Gholami, Kurt Keutzer, George Biros — people — authors of ANODE; the paper that makes the OTD/DTO distinction concrete and empirically demonstrates divergence in realistic (ReLU/CNN) neural-ODE training.
• Sina Ober-Blöbaum, Oliver Junge — people — co-authors of the DMOC convergence analysis with Marsden.
• discretize-then-optimize / optimize-then-discretize — concept — the general numerical-analysis distinction this whole capture turns on; could anchor its own note and would usefully link claim-neural-odes-cite-pontryagin-adjoint-method-as-backprop (an OTD example) against claim-backprop-special-case-kelley-bryson-formula (a DTO example) directly.
• discrete-time Pontryagin maximum principle — concept — the classical-control-theory object; distinct from (and narrower than) "the costate/adjoint equation" alone, a distinction this capture found genuinely load-bearing and the vault does not yet separate explicitly.

Safety flags

None. All sources this session (arxiv.org PDFs, two independently fetched via extract_pdf with TLS verified; the Caltech CDS mirror fetched with TLS unverified but containing nothing more than a standard academic paper header, cross-checked against the verified arXiv copy rather than used as a citation) were ordinary academic preprints and journal articles. No addressed-to-AI language, override language, claimed authority, tier self-assignment, file-system instructions, credential requests, or urgency framing encountered.