The discrete Kelley-Bryson costate recursion and Pontryagin's continuous adjoint are the same object only inside a hypothesis-bound regime; outside it they are a family resemblance, not an identity

pontryaginkelley-brysoncostateadjoint-methodoptimal-controldiscretize-then-optimizebackpropagationhistory-of-mlnumerical-analysis

Three independently-sourced findings, read together, resolve a narrower question than they first appear to settle. Ober-Blöbaum, Junge & Marsden (2008/2011) prove that the discrete costate recursion generated by a structure-preserving (symplectic partitioned Runge-Kutta) discretization does converge to Pontryagin's continuous adjoint ODE as the step size shrinks to zero, at matching order — but only under stated Smoothness and Coercivity assumptions. Kipka & Gupta (2017/2019), working in classical control theory independent of neural networks, show there is no unconditional version of this correspondence for Pontryagin's full apparatus: the maximization condition specifically can fail without added convexity. And Gholami, Keutzer & Biros (2019) demonstrate, inside neural-ODE training with realistic ReLU/convolutional nonlinearities and naive (non-structure-preserving) discretization, that the two computations can diverge outright rather than converge.

"Same object under two time-discretizations" is therefore correct as a limiting statement inside a regularity regime — smoothness, coercivity/control-uniqueness, a structure-preserving discretization scheme — and "family resemblance" is the more accurate description once that regime's hypotheses are dropped, which is exactly what happens in the ReLU/non-convex settings neural networks actually use. This bears directly on the vault's standing tension between Mizutani, Dreyfus & Nishio's "special case" framing of backpropagation as the Kelley-Bryson discrete formula, Schmidhuber's "precursor lacking efficiency" framing, and Chen et al.'s citation of Pontryagin's continuous adjoint method as equivalent to backprop: the three framings are not simply in tension, they describe two ends of a genuine conditional, and which end a given training setup lands on is a mathematical fact about its smoothness and discretization, not a free choice of description.