Training pays for backpropagation's backward pass; inference runs the forward pass alone — the backward pass costs roughly 2x the forward, making training tokens ~3x inference tokens
The bridge between the vault's two largest clusters is a mechanism, stated
in FLOPs terms by Epoch AI: "When training a model, we need to perform both
forward and backward passes, and backward passes are generally twice as
expensive to do as forward passes" — hence "tokens seen during training are
around three times more expensive compared to tokens seen or generated
during inference" (Erdil 2024, the source_url). The scale of the
asymmetry, illustrated on GPT-3 in Epoch's earlier piece (Villalobos &
Atkinson 2023, source_supporting_url, footnote 10): "the cost of training
was 3e23 FLOP, whereas the cost of a single inference is 3e11" — one
training run ≈ a trillion inferences, with single-inference compute
"usually close to the square root of the training compute" (a heuristic
with stated variability, not a law). (2026-09-11 audit: the two Epoch
pieces do not agree on the backward-pass multiplier — the 2023 piece's
footnote 15 says training "requires 3.5 trillion backward passes, each of
which is 3x as expensive as a forward pass", against Erdil's "twice as
expensive"; the headline's ~2x/~3x rests on Erdil. The order-of-magnitude
point survives either reading.)
The backward pass is backpropagation's own computational content — the thing claim-linnainmaa-reverse-mode-single-pass describes and claim-reverse-mode-multiple-independent-discovery traces. Inference (claim-ai-inference-means-running-a-model, claim-llm-inference-prefill-decode) is the phase that never runs it. That is the load-bearing link between the origins cluster and the economics cluster.
What this note deliberately does NOT claim: that inference therefore dominates industry compute. The magnitude story is separate and unsettled — Epoch's theoretical allocation argument predicts comparable spending; the one real datapoint (OpenAI 2024, Tier 2 via Epoch's curation of press-derived figures) shows training spend above inference spend ($3B vs $1.8B). The circulating "two-thirds" figure is ledgered at myth-inference-two-thirds-of-compute, and claim-inference-dominant-ai-compute-2026 keeps its [unverified-quant] flag. See moc-inference-economics, moc-backpropagation-origins.
Source
“When training a model, we need to perform both forward and backward passes, and backward passes are generally twice as expensive to do as forward passes.”