Forward- and reverse-mode AD apply the same Jacobian in dual directions — the VJP's linear part is the transpose of the JVP's (pushforward vs pullback)

automatic-differentiationforward-modereverse-modejvpvjpdual-numbersmechanism

The two AD modes are not sibling algorithms but one linear object used two ways. Stated at Tier 1 from both vocabularies:

The classical framing (Baydin et al., read directly). Forward mode "can be viewed as evaluating a function using dual numbers" — truncated Taylor series v + v̇ε with ε² = 0, whose product rule carries derivatives. Seeding the tangent ẋ = r yields the directional derivative ∇f·r in one pass. Reverse mode is the two-phase sweep: record forward, then propagate adjoints v̄ᵢ = ∂yⱼ/∂vᵢ backward (claim-autograd-three-reifications-of-the-tape).

The geometric framing (JAX docs, fetched directly). JVP: (x, v) ↦ ∂f(x)v — the pushforward, building Jacobians "one column at a time." VJP: (x, v) ↦ vᵀ∂f(x) — the pullback, "one row at a time," and "the linear part of a VJP [is] the transpose (or adjoint conjugate) of the linear part of a JVP."

The cost duality follows from the shape, not from benchmarks. A gradient of f: ℝⁿ→ℝ needs n forward passes but one reverse pass — why training is reverse-mode (claim-backpropagation-special-case-of-reverse-mode-ad, claim-cheap-gradient-bound-two-figures). The memory mirror: forward mode's footprint is depth-independent; reverse mode's scales with the recorded computation. (JAX's "~3x FLOPs per JVP" is its own implementation's self-characterization — kept scoped, not a universal AD constant.)

Vocabulary caution carried from the capture: JAX never uses a literal dual-number type — dual numbers and per-primitive JVP rules are the same computation in two technical vocabularies, not two computations. Cluster: moc-backpropagation-origins.