--- id: "20260706-1508-what-formal-derivation-does" title: "What formal derivation does Mizutani & Dreyfus (2000) provide linking MLP backpropagation to the Kelley-Bryson optimal-control gradient formula?" type: "capture" status: "promoted" date_promoted: "2026-07-06T00:00:00.000Z" promoted_to: ["30-notes/claim-backprop-special-case-kelley-bryson-formula.md"] not_promoted: ["Terminology-correspondence table detail — summarized in the note; full table stays in capture","Hidden-node-teaching application — context only, not vault-relevant as a standalone claim"] origin: "batch" model: "claude-sonnet-5" date_created: "2026-07-06T00:00:00.000Z" provenance: "web-research batch run, 2026-07-06" derived_from: [] tags: ["backpropagation","optimal-control","Kelley-Bryson","dynamic-programming","Mizutani-Dreyfus","gradient-descent","chain-rule","history-of-AI","deep-learning-history"] source_urls: ["https://ieor.berkeley.edu/wp-content/uploads/2019/03/ijcnn2k.pdf","https://ieeexplore.ieee.org/document/857892/","https://www.semanticscholar.org/paper/On-derivation-of-MLP-backpropagation-from-the-and-Mizutani-Dreyfus/bdb4ba005dcedfd57e3110c12e56b9beeb012419"] source_authors: ["Eiji Mizutani","Stuart E. Dreyfus","Kenichi Nishio"] source_dates: ["2000"] source_tiers: ["Tier 1 — Mizutani, Dreyfus & Nishio (2000), 'On derivation of MLP backpropagation from the Kelley-Bryson optimal-control gradient formula and its application,' Proc. IEEE-INNS-ENNS IJCNN 2000. Primary source, hosted at co-author Dreyfus's own UC Berkeley IEOR department page; full text extracted directly via pdftotext, 6 pages, cleanly quotable.","Tier 3-4 — Semantic Scholar listing and IEEE Xplore document record (bibliographic metadata only: venue, page numbers, DOI; IEEE Xplore page itself returned no fetchable body text this session, corroborated via search-result snippet)."] --- **Topic question (harvested from):** [[2026-06-30-what-role-did-kelley-1960-and-bryson-1961-optimal-control-work-play-in-the-backprop-lineage]] — that capture's "Further leads" section flagged this paper (PDF at the same URL used here) as "compressed in this run" and worth extracting for a synthesis note. This capture resolves that lead: the PDF extracted cleanly via `extract_pdf` (pdftotext method) this run. **Related notes:** [[claim-kelley-bryson-optimal-control-precursor]] · [[20260706-1505-did-kelley-1960-derive]] · [[backpropagation-gap]] · [[dynamic-programming]] · [[chain-rule]] · [[adjoint-method]] --- ## Access note on the primary source The PDF at `https://ieor.berkeley.edu/wp-content/uploads/2019/03/ijcnn2k.pdf` is hosted on the University of California, Berkeley Industrial Engineering and Operations Research department's own site — the home department of co-author Stuart E. Dreyfus (author email `dreyfus@ieor.berkeley.edu` appears on the paper's own title page). This is the authors' own institutional venue, treated as Tier 1 primary. `extract_pdf` retrieved 6 pages of directly quotable text via `pdftotext`, resolving the prior run's "compressed" access problem. Bibliographic metadata (conference name IJCNN 2000, page range 167-172 vol.2, DOI 10.1109/ijcnn.2000.857892) was corroborated via a Semantic Scholar / IEEE Xplore search-result snippet; direct WebFetch of the IEEE Xplore page itself returned no body text (likely a bot gate), so that specific venue/page detail rests at Tier 3-4 — acceptable per the sourcing floor since it is uncontested bibliographic/historical detail, not a mechanism claim. --- ## Claim: The paper's central thesis is that standard MLP backpropagation is a simplified special case of the Kelley-Bryson optimal-control gradient formula, and the paper supplies an explicit re-derivation to demonstrate this **Claim type:** Specific technical-mechanism claim → floor Tier 1–2 required. Mizutani, Dreyfus & Nishio open the paper with this framing (exact quote, Abstract): > "The well-known backpropagation (BP) derivative computation process for multilayer perceptrons (MLP) learning can be viewed as a simplified version of the Kelley-Bryson gradient formula in the classical discrete-time optimal control theory [1]. We detail the derivation in the spirit of dynamic programming, showing how they can serve to implement more elaborate learning whereby teacher signals can be presented to any nodes at any hidden layers, as well as at the terminal output layer." And in the introduction (exact quote): > "The partial-derivative form of sensitivity is computed backward one stage after another from the terminal stage. In MLP terminology, this is the so-called generalized delta rule, which is equivalent to, what we call, the Kelley-Bryson formula, notably due to Kelley's adjoint equations [6] and Bryson's multiplier rule [7]." **Sourcing:** Tier 1 (primary source, direct-quoted text from the paper itself). ✓ clears floor for technical-mechanism claim. --- ## Claim: The formal derivation proceeds by casting MLP learning as a discrete-time optimal-control / dynamic-programming problem — defining a value function (cost-to-go), a recurrence relation, and a boundary condition — and differentiating the value function to obtain backprop's sensitivity (delta) recursion **Claim type:** Specific technical-mechanism claim → floor Tier 1–2 required. The paper sets up the objective cost as a multi-stage sum of per-stage costs plus a terminal cost (exact quote, Eq. 1 context): > "we consider the following objective cost function: E(θ) = Σ(s=1 to N-1) g^s(a^s, θ^s) + h(a^N), where a^s and θ^s signify the state vector and decision vector at stage s, and g^s(., .) and h(.) denote the immediate cost per stage s and the terminal cost, respectively." It then defines three dynamic-programming objects explicitly (exact quote, Section 2): > "Value function: T^s(a^s, θ^s) def= cost-to-go, starting at state a^s at stage s, using a guessed policy θ^s. ... Recurrence relation: T^s(a^s, θ^s) = T^{s+1}(a^{s+1}, θ^{s+1}) + g^s(a^s, θ^s) = T^{s+1}(f^{s+1}(a^s, θ^s), θ^{s+1}) + g^s(a^s, θ^s) ... Boundary condition: T^N(a^N, −) = h(a^N) = (1/2)Σ_k (t^N_k − a^N_k)^2." The sensitivity term δ (the "delta" of backprop's delta rule) is then derived as the partial derivative of the value function with respect to activation, and shown — via direct application of the chain rule to the recurrence relation — to reduce to the standard backprop recursion (exact quote, derivation in Section 2, terminal case and general recursion): > "δ^N_k def= ∂T^N/∂a^N_k = −(t^N_k − a^N_k). Departing from this terminal value at layer N, the sensitivity δ is backpropagated and computed backward one after another by the following recurrence relation, our version of the delta-rule: ... δ^s_j = Σ(k=1 to n^{s+1}) δ^{s+1}_k f'^{s+1}(net^{s+1}_k) θ^s_jk − (t^s_j − a^s_j)." And the weight-update rule follows the same differentiation logic (exact quote): > "∆θ^s_jk = −η ∂T^s/∂θ^s_jk = −η ∂T^{s+1}(a^{s+1}, θ^{s+1})/∂θ^s_jk (due to the recurrence relation and the fact that g^s is independent of θ^s) ... = −η δ^{s+1}_k f'^{s+1}(net^{s+1}_k) a^s_j." This is the paper's formal derivation: MLP backpropagation's forward activation pass and backward delta pass are shown to be, respectively, the state-transition recursion and the costate/adjoint recursion of a discrete-time optimal-control problem solved by dynamic programming, with the value function's partial derivatives being exactly backprop's sensitivities. **Sourcing:** Tier 1 (primary source, direct-quoted equations and text). ✓ clears floor for technical-mechanism claim. --- ## Claim: The paper gives an explicit terminology-correspondence table mapping optimal-control concepts one-to-one onto neural-network concepts **Claim type:** Specific technical-mechanism / definitional claim → floor Tier 1–2 required (mechanism mapping); source is Tier 1 primary. Exact quote (Section 2, the paper's own table, reproduced in row form): > "Notation θ — Optimal Control: decision variables — Neural Network: weight parameters. Notation a — Optimal Control: state variables — Neural Network: (neuron) activation. Notation s — Optimal Control: stage — Neural Network: layer. Notation δ — Optimal Control: adjoint variables / Lagrange multipliers / costates — Neural Network: delta." The paper adds an explicit caveat about a specific notational divergence from the most familiar BP formulation (exact quote): > "It should be noted that the best-known BP formulation due to Rumelhart et al. was made by choosing the net input but not the activation as the state variable; hence, it looks slightly different." **Sourcing:** Tier 1 (primary source, direct-quoted table and text). ✓ clears floor. --- ## Claim: The paper situates Dreyfus's own 1962 dynamic-programming-style gradient derivation as producing a recursion "almost identical" to the generalized delta rule, predating the neural-network formulation **Claim type:** Historical claim, load-bearing technical comparison → escalated to Tier 1–2 per the "surprising/contested/load-bearing" clause of the historical-claims floor rule (this is the specific technical-lineage point the topic exists to chase). Exact quote (Introduction): > "Also, in 1962, Dreyfus derived a dynamic-programming-like recursive gradient formulation [8, 9], which is almost identical to the generalized delta rule, for solving the multi-stage Mayer-type optimal control problem (see Eq. (5.4) in ref. [8])." Reference [8] is cited in the paper's bibliography as: "Stuart E. Dreyfus. The numerical solution of variational problems. Journal of Mathematical Analysis and Applications, 5(1):30–45, 1962." Reference [9] is "Stuart E. Dreyfus and Averill M. Law. The Art and Theory of Dynamic Programming... Academic Press Inc., 1977." This capture did not retrieve the 1962 *J. Math. Anal. Appl.* paper itself to check Eq. (5.4) directly — the claim here rests on Mizutani, Dreyfus & Nishio's own (Tier 1) characterization of that earlier paper, made by a co-author (Dreyfus) about his own prior work, which is about as strong as secondary characterization gets but is not yet cross-checked against the 1962 primary text itself. **Sourcing:** Tier 1 (primary source; author characterizing his own earlier work). ✓ clears floor. Flagged for completeness: the 1962 paper's Eq. (5.4) itself is [unverified-mechanism — needs primary] pending direct retrieval of Dreyfus (1962), *J. Math. Anal. Appl.*, 5(1):30–45. --- ## Claim: The 2000 paper's own novel contribution — extending the Kelley-Bryson-derived formula to permit teacher signals at hidden layers ("hidden-node teaching") — was demonstrated on a small industrial color-characterization problem **Claim type:** Quantitative claim (specific numeric results) → floor Tier 1–2 required. Included as context for what the derivation was built to support, not as a claim about the Kelley-Bryson lineage itself. Exact quote (Discussion): > "At the epoch of 3,000, the training color difference (associated with the final outputs XYZ) obtained by the MLP with hidden-node teaching was 7.7008, which is much smaller than 24.5868, the color difference obtained by the MLP without hidden-node teaching." Experimental setup (exact quote, Section 3): "a 3 × 7 × 3 MLP," trained on "25 color samples (24 color samples from Machbeth ColorChecker plus the 'perfect black' color sample)" with "51 checking data," using "the steepest descent-type pattern-by-pattern learning method." **Sourcing:** Tier 1 (primary source, direct-quoted numbers). ✓ clears floor. Recorded for completeness only — this application result is downstream of the derivation and not itself part of the Kelley-Bryson lineage question. --- ## Overall verdict on the core question **Core question — "What formal derivation does Mizutani & Dreyfus (2000) provide linking MLP backpropagation to the Kelley-Bryson optimal-control gradient formula?"** — **Confirmed and detailed** against the primary source. The paper: (1) explicitly frames standard MLP backpropagation as a simplified special case of the Kelley-Bryson discrete-time optimal-control gradient formula; (2) supplies a step-by-step dynamic-programming derivation (value function → recurrence relation → boundary condition → differentiate for sensitivity δ → differentiate again for the weight-update rule) that reproduces the standard backprop delta-rule and weight-update equations from first principles; (3) supplies an explicit notation-correspondence table (decision variables/weights, state variables/activations, stage/layer, adjoint-variables-Lagrange-multipliers-costates/delta); and (4) notes precisely where the most familiar (Rumelhart et al.) BP formulation diverges notationally (net input vs. activation as the state variable). The paper then uses this re-derivation as the springboard for its own novel contribution (teacher signals at hidden layers), which is not part of the lineage question itself but is the paper's stated reason for existing. > [!note] Seek's commentary: > The satisfying piece here is the correspondence table itself — it's the paper doing, in one small block, exactly the kind of "this is the same idea wearing different clothes" work that the whole Kelley→Bryson→Dreyfus→backprop chain has been circling. Costate variables and Lagrange multipliers and "delta" are one concept under three names from three different fields' habits of speech, and Mizutani & Dreyfus just put the three names next to each other and let the reader see it. --- ## Further leads - Dreyfus (1962), "The numerical solution of variational problems," *Journal of Mathematical Analysis and Applications*, 5(1):30–45 — the primary source for Eq. (5.4), the "almost identical to the generalized delta rule" recursion referenced above. Not yet retrieved in full text in any run; would let the [unverified-mechanism] flag above be resolved directly rather than through Mizutani/Dreyfus's own characterization of it. - Dreyfus & Law (1977), *The Art and Theory of Dynamic Programming* — cited alongside the 1962 paper as ref. [9]; likely contains a fuller worked version of the same dynamic-programming gradient recursion in textbook form. - The paper's reference [4], Yann LeCun's "A theoretical framework for back-propagation" (1988 Connectionist Models Summer School) — cited by Mizutani, Dreyfus & Nishio as one of the works that "clearly" explains BP "in light of the classical optimal control theory," alongside Dreyfus (1990) and Bertsekas's *Nonlinear Programming* (1996). Worth a parallel capture to see whether LeCun's derivation matches or differs from this one.