---
title: "Training pays for backpropagation's backward pass; inference runs the forward pass alone — the backward pass costs roughly 2x the forward, making training tokens ~3x inference tokens"
type: "claim"
status: "budding"
audit_status: "capture-verified (Epoch AI pieces fetched directly at capture level, quotes verbatim per capture provenance; queen re-check not yet run)"
source_url: "https://epoch.ai/blog/optimally-allocating-compute-between-inference-and-training"
source_title: "Optimally allocating compute between inference and training"
source_author: "Ege Erdil (Epoch AI); supporting: Pablo Villalobos & David Atkinson (Epoch AI)"
source_date: "2024-03-29T00:00:00.000Z"
source_tier: 1
source_quote: "When training a model, we need to perform both forward and backward passes, and backward passes are generally twice as expensive to do as forward passes."
provenance: "Promotion from 10-inbox/raw/2026-07-06-is-training-vs-inference-compute-asymmetry-...md, 2026-07-07, queen cycle 4 — answering 50-questions/question-bridge-training-inference-compute.md (opened cycle 1, researched by the bee the same night)"
origin: "session"
date_created: "2026-07-07T00:00:00.000Z"
tags: ["training","inference","backward-pass","compute-economics","bridge-note","epoch-ai"]
drafted_in: ["2026-07-09-inference-inverted","inference-inverted"]
---


The bridge between the vault's two largest clusters is a mechanism, stated
in FLOPs terms by Epoch AI: "When training a model, we need to perform both
forward and backward passes, and backward passes are generally twice as
expensive to do as forward passes" — hence "tokens seen during training are
around three times more expensive compared to tokens seen or generated
during inference." The scale of the asymmetry, illustrated on GPT-3: "the
cost of training was 3e23 FLOP, whereas the cost of a single inference is
3e11" — one training run ≈ a trillion inferences, with single-inference
compute "usually close to the square root of the training compute" (a
heuristic with stated variability, not a law).

The backward pass is backpropagation's own computational content — the thing
[[claim-linnainmaa-reverse-mode-single-pass]] describes and
[[claim-reverse-mode-multiple-independent-discovery]] traces. Inference
([[claim-ai-inference-means-running-a-model]],
[[claim-llm-inference-prefill-decode]]) is the phase that never runs it.
That is the load-bearing link between the origins cluster and the economics
cluster.

**What this note deliberately does NOT claim:** that inference therefore
dominates industry compute. The magnitude story is separate and unsettled —
Epoch's theoretical allocation argument predicts *comparable* spending; the
one real datapoint (OpenAI 2024, Tier 2 via Epoch's curation of press-derived
figures) shows training spend *above* inference spend ($3B vs $1.8B). The
circulating "two-thirds" figure is ledgered at
[[myth-inference-two-thirds-of-compute]], and
[[claim-inference-dominant-ai-compute-2026]] keeps its [unverified-quant]
flag. See [[moc-inference-economics]], [[moc-backpropagation-origins]].