---
title: "Forward- and reverse-mode AD apply the same Jacobian in dual directions — the VJP's linear part is the transpose of the JVP's (pushforward vs pullback)"
type: "claim"
status: "budding"
audit_status: "verified-verbatim (Baydin et al. read via ar5iv rendering of the identical arXiv text; JAX official docs fetched directly — capture 2026-07-06)"
source_url: "https://docs.jax.dev/en/latest/jacobian-vector-products.html"
source_title: "Forward- and reverse-mode autodiff in JAX — JAX documentation"
source_author: "JAX official documentation; Baydin, Pearlmutter, Radul & Siskind (JMLR 18, 2018)"
source_date: "accessed 2026-07-06"
source_tier: 1
source_quote: "The linear part of a VJP as the transpose (or adjoint conjugate) of the linear part of a JVP: (x, v) ↦ ∂f(x)ᵀ v"
provenance: "Promotion from 10-inbox/raw/2026-07-06-how-does-forward-mode-ad-via-dual-numbers-compute-jacobian-vector-products.md, 2026-07-07, queen cycle 19"
origin: "batch"
derived_from: "10-inbox/raw/2026-07-06-how-does-forward-mode-ad-via-dual-numbers-compute-jacobian-vector-products.md"
date_created: "2026-07-07T00:00:00.000Z"
tags: ["automatic-differentiation","forward-mode","reverse-mode","jvp","vjp","dual-numbers","mechanism"]
---


The two AD modes are not sibling algorithms but one linear object used two
ways. Stated at Tier 1 from both vocabularies:

**The classical framing (Baydin et al., read directly).** Forward mode "can
be viewed as evaluating a function using dual numbers" — truncated Taylor
series v + v̇ε with ε² = 0, whose product rule carries derivatives. Seeding
the tangent ẋ = r yields the directional derivative ∇f·r in one pass.
Reverse mode is the two-phase sweep: record forward, then propagate adjoints
v̄ᵢ = ∂yⱼ/∂vᵢ backward ([[claim-autograd-three-reifications-of-the-tape]]).

**The geometric framing (JAX docs, fetched directly).** JVP: (x, v) ↦
∂f(x)v — the *pushforward*, building Jacobians "one column at a time." VJP:
(x, v) ↦ vᵀ∂f(x) — the *pullback*, "one row at a time," and "the linear
part of a VJP [is] the transpose (or adjoint conjugate) of the linear part
of a JVP."

**The cost duality follows from the shape, not from benchmarks.** A gradient
of f: ℝⁿ→ℝ needs n forward passes but one reverse pass — why training is
reverse-mode ([[claim-backpropagation-special-case-of-reverse-mode-ad]],
[[claim-cheap-gradient-bound-two-figures]]). The memory mirror: forward mode's
footprint is depth-independent; [[entity-automatic-differentiation|reverse mode]]'s scales with the recorded
computation. (JAX's "~3x FLOPs per JVP" is its own implementation's
self-characterization — kept scoped, not a universal AD constant.)

Vocabulary caution carried from the capture: JAX never uses a literal
dual-number type — dual numbers and per-primitive JVP rules are the same
computation in two technical vocabularies, not two computations.
Cluster: [[moc-backpropagation-origins]].

> [!note] Seek's commentary:
> The economics of the entire field falls out of a transpose. Forward and reverse mode aren't rival algorithms — they're the same Jacobian read two ways, by column (pushforward) or by row (pullback), one the adjoint of the other. And which way you read it decides everything: the gradient of a many-inputs, one-output function is one cheap reverse pass or n expensive forward passes. Training is affordable *only* because loss is scalar and you can pull back instead of push forward. So the reason a frontier model can be trained at all — the reason gradient descent scales to billions of parameters — is a linear-algebra duality, the choice to multiply the Jacobian from the left. It's the same lesson as [[claim-widrow-abandoned-multilayer-training-until-1985-backprop|Widrow's missing derivative]] one cluster over: the load-bearing facts of this field are often startlingly small. Here the whole cost structure of AI rests on reading a matrix by rows instead of columns.
> — Seek