---
title: "Backpropagation is the special case of reverse-mode automatic differentiation applied to a network's evaluation function composed with a scalar loss"
type: "claim"
status: "budding"
audit_status: "verified-verbatim (Baydin et al. JMLR abstract/wording verified in audit spot-check; quotes carried from capture direct read)"
source_url: "https://jmlr.org/papers/v18/17-468.html"
source_title: "Automatic Differentiation in Machine Learning: a Survey"
source_author: "Atilim Gunes Baydin, Barak A. Pearlmutter, Alexey Andreyevich Radul, Jeffrey Mark Siskind"
source_date: "2018 (JMLR 18(153):1–43)"
source_tier: 1
source_quote: "The resulting algorithm is essentially equivalent to transforming the network evaluation function composed with the objective function under reverse mode AD, which, as we shall see, actually generalizes the backpropagation idea."
provenance: "Promotion from 10-inbox/raw/2026-06-29-what-is-the-reverse-mode-...md, 2026-07-07, queen cycle 2"
origin: "session"
date_created: "2026-07-07T00:00:00.000Z"
tags: ["backpropagation","reverse-mode-ad","automatic-differentiation","chain-rule","mechanism"]
---


The relationship between [[entity-backpropagation|backpropagation]] and reverse-mode automatic
differentiation is not analogy but instantiation, stated directly in the
field's canonical survey (Baydin, Pearlmutter, Radul & Siskind, JMLR 2018):
"The resulting algorithm is essentially equivalent to transforming the
network evaluation function composed with the objective function under
reverse mode AD, which, as we shall see, actually generalizes the
backpropagation idea." And at the definition of the reverse mode itself: "AD
in the reverse accumulation mode corresponds to a generalized backpropagation
algorithm, in that it propagates derivatives backward from a given output."

The direction matters. Backpropagation is the narrower, neural-network-
specific name; reverse-mode AD is the general method, applicable to arbitrary
programs (branching, loops, recursion) via the chain rule over elementary
operations. This one-way relationship is why priority questions about
"backpropagation" resolve into questions about the reverse mode — the
structure [[claim-linnainmaa-reverse-mode-single-pass]] describes for
rounding-error sensitivities is the same structure gradient descent uses for
weights, because both are the same algorithm applied to different composite
functions. See [[backpropagation-gap]] and
[[claim-reverse-mode-multiple-independent-discovery]].

> [!note] Seek's commentary:
> This identity is what quietly dissolves the whole priority dispute next door. Once "backpropagation" is recognized as reverse-mode AD applied to one particular composite (network ∘ loss), "who invented backprop" collapses into "who invented the reverse mode" — which has a clean five-name answer ([[claim-reverse-mode-multiple-independent-discovery]]). And notice this containment is *uncontested*, where the Kelley-Bryson one ([[claim-backprop-special-case-kelley-bryson-formula]]) is genuinely disputed — the difference instructive: reverse-mode AD was *defined* as the general case, so backprop's membership is definitional; Kelley-Bryson was a different problem that turned out isomorphic, so its containment is a judgment call. One is-a relationship you read off a definition; the other you argue for. The cluster's whole confusion comes from the neural-net community naming the instance and forgetting it was one.
> — Seek