---
title: "Discretize-then-optimize vs. optimize-then-discretize (DTO/OTD)"
type: "entity"
entity_kind: "concept"
status: "hub"
canonical_name: "Discretize-then-optimize vs. optimize-then-discretize"
aliases: ["DTO","OTD","discretize-optimize","optimize-discretize"]
first_seen: "2026-10-01T00:00:00.000Z"
writer_model: "claude-sonnet-5"
connects_to: ["adjoint method","neural ODEs","Pontryagin's maximum principle","backpropagation","numerical analysis (consistency and stability)"]
seek_code_commit: "c7427a5"
---


A standing distinction in numerical optimal control, predating and independent of neural networks: given a continuous-time dynamical system, one can either discretize the dynamics first and then differentiate the discrete recursion to get a gradient ("discretize-then-optimize," DTO), or derive the continuous-time adjoint/costate equations first and discretize *those* ("optimize-then-discretize," OTD). The two procedures are not guaranteed to commute, and in general do not.

Matters to this vault as the precise numerical-analysis name for the question underneath a cluster of history-of-backpropagation claims: differentiating a discretized neural-ODE forward pass (DTO) reproduces the [[entity-henry-kelley|Kelley]]-[[entity-arthur-bryson|Bryson]] discrete costate recursion exactly, while solving [[entity-lev-pontryagin|Pontryagin]]'s continuous adjoint ODE backward and then discretizing it (OTD) is the move modern neural-ODE papers make when they cite Pontryagin by name. [[claim-dto-and-otd-neural-ode-gradients-can-diverge]] shows these give provably different gradients, and can diverge outright for realistic nonlinearities; [[claim-ober-blobaum-junge-marsden-prove-discrete-adjoint-converges-to-continuous]] shows the two *do* converge under stated smoothness/coercivity/structure-preserving-discretization hypotheses. The synthesis sits at [[claim-kelley-bryson-pontryagin-costate-same-object-only-under-hypotheses]].

## References
- [[claim-dto-and-otd-neural-ode-gradients-can-diverge]]
- [[claim-ober-blobaum-junge-marsden-prove-discrete-adjoint-converges-to-continuous]]
- [[claim-kelley-bryson-pontryagin-costate-same-object-only-under-hypotheses]]
- [[claim-neural-odes-cite-pontryagin-adjoint-method-as-backprop]] — an OTD example
- [[claim-backprop-special-case-kelley-bryson-formula]] — a DTO example