Capture: Is "training-vs-inference compute asymmetry" the missing bridge note between the vault's two biggest clusters?

Answering question-bridge-training-inference-compute, which asked whether the compute asymmetry between training and inference is the load-bearing bridge missing between the vault's two largest clusters — backpropagation-origins (~10 notes, spine claim backpropagation-gap) and the inference-economics cluster (~11 notes, spine claim claim-inference-dominant-ai-compute-2026).

Short answer, established claim by claim below: the mechanism half of the candidate bridge — training requires backpropagation's forward-and-backward pass, inference runs the forward pass alone, and that structural difference is what produces the compute/cost asymmetry — is now solidly Tier 1-sourced and does connect the two clusters. The magnitude half — the vault's current headline claim that inference is "roughly two-thirds" of all AI compute in 2026 — is not resolved by this research; if anything, tracing its citation chain back to its root makes it look weaker than before. The bridge note is real at the mechanism level and still ungrounded at the economic-dominance-statistic level. Both halves are recorded below so promotion sees exactly where the seam is solid and where it isn't.

Claim 1: The FLOPs cost of a full training run and the FLOPs cost of a single inference differ by roughly twelve orders of magnitude for a concrete example model (GPT-3)

Claim type: quantitative Sourcing floor: Tier 1–2 required; achieved Tier 1

"For GPT-3, the cost of training was 3e23 FLOP, whereas the cost of a single inference is 3e11."

Pablo Villalobos and David Atkinson, Epoch AI, "Trading off compute in training and inference," 2023-07-28. 3e23 ÷ 3e11 = 1e12 — the training run is equivalent in FLOPs to roughly one trillion individual inferences.

Provenance: https://epoch.ai/blog/trading-off-compute-in-training-and-inference — Tier 1 (Epoch AI's own original compute-cost estimation, published on its own research venue, named authors).

Claim 2: A commonly-used rule of thumb: single-inference compute is close to the square root of total training compute

Claim type: quantitative Sourcing floor: Tier 1–2 required; achieved Tier 1

"The single-inference compute is usually close to the square root of the training compute."

Same source and authors as Claim 1. The piece separately notes "significant variability" around this rule, so it is recorded as a heuristic, not a law.

Provenance: https://epoch.ai/blog/trading-off-compute-in-training-and-inference — Tier 1.

Claim 3: The structural reason training is more expensive than inference, per token: training requires both a forward and a backward pass; inference requires only the forward pass; and the backward pass is roughly twice as expensive as the forward pass alone

Claim type: technical-mechanism Sourcing floor: Tier 1–2 required; achieved Tier 1

"When training a model, we need to perform both forward and backward passes, and backward passes are generally twice as expensive to do as forward passes."

"tokens seen during training are around three times more expensive compared to tokens seen or generated during inference"

Ege Erdil, Epoch AI, "Optimally allocating compute between inference and training," 2024-03-29.

This is the precise mechanistic link the parent question asked for. The backpropagation-origins cluster is, at root, about the algorithm that computes the backward pass — Linnainmaa's reverse-mode automatic differentiation, later rediscovered as the neural-network training algorithm. The inference-economics cluster is about the phase of a model's life that runs without that backward pass. Erdil's piece states directly, in FLOPs terms, why the second phase is cheaper than the first: the backward pass — backpropagation's own computational content — is the extra cost that only training pays.

Provenance: https://epoch.ai/blog/optimally-allocating-compute-between-inference-and-training — Tier 1 (Epoch AI's own derivation, named author, own venue).

Claim 4: Under this framework, compute-optimal AI labs should in principle spend comparable total resources on training and on running inference, given flexible trade-offs between the two

Claim type: technical-mechanism / theoretical-economic claim (an allocation argument built from Claims 1–3, not an empirical measurement) Sourcing floor: treated as Tier 1–2 given its mechanism-adjacent nature; achieved Tier 1

"AI labs should spend comparable resources on training and running inference, assuming they can flexibly balance compute between these tasks to maintain model performance."

Same source as Claim 3. This is a normative/theoretical claim about optimal allocation, not a report of what labs actually spend (see Claim 5 for an empirical figure) — recorded as such, not conflated with observed spending.

Provenance: https://epoch.ai/blog/optimally-allocating-compute-between-inference-and-training — Tier 1.

Claim 5: At one frontier lab in one year, actual compute spending was split roughly $3B training / $1.8B inference / ~$1–2B research, out of about $7B total cloud compute cost

Claim type: quantitative Sourcing floor: Tier 1–2 required; achieved Tier 2, with an explicit caveat on the chain of custody

"OpenAI spent $3 billion on training compute, $1.8 billion on inference compute, and $1 billion on research compute amortized over 'multiple years'."

Josh You, Epoch AI, "Most of OpenAI's 2024 compute went to experiments," 2025-10-10.

Caveat, recorded rather than smoothed over: the article itself states its dollar figures for OpenAI's overall spend come from "Epoch's AI Companies dataset, ultimately via reports in The Information and The New York Times" — i.e. Epoch is curating and modeling on top of numbers whose ultimate origin is journalism about a private company's unpublished finances, not OpenAI's own disclosure. Epoch's own methodology (stated assumptions, named analyst, transparent modeling) earns this a Tier 2 placement rather than the Tier 3 the raw press reports alone would carry — but it is not treated as a clean Tier 1 primary figure, and the number should not be read as more solid than "best current outside estimate."

Provenance: https://epoch.ai/data-insights/openai-compute-spend — Tier 2 (see caveat above).

Claim 6 (re-audit of an existing vault claim): the "inference is roughly two-thirds of all AI compute in 2026, up from a third in 2023 and half in 2025" figure — already flagged [unverified-quant — needs primary] on claim-inference-dominant-ai-compute-2026 — traces to a citation chain that does not clear the sourcing floor, and this session's research reinforces the existing flag rather than resolving it

Claim type: quantitative Sourcing floor: Tier 1–2 required; not achieved — remains below floor, now with the citation chain identified

The identical "two-thirds / half / a third" figure appears verbatim or near-verbatim across multiple secondary venues (Telnyx, Vast.ai, Introl, and Deloitte). Tracing it to its earliest and most-cited instance:

"Inference workloads will indeed be the hot new thing in 2026, accounting for roughly two-thirds of all compute (up from a third in 2023 and half in 2025)"

Deloitte Insights, "More compute for AI, not less" (2026 Technology, Media & Telecommunications Predictions), 2026. Tier 3 (named research team, real venue, but a predictions/trend report, not a primary compute measurement).

Deloitte's own endnote for this statistic cites: Rodrigo Liang, "Scaling AI without breaking the grid: The path to sustainable innovation," World Economic Forum, 2025-01-03. This WEF piece could not be fetched directly this session (403 Forbidden on direct request), so its exact wording is [unsourced — needs verification]; but its authorship is independently confirmed via search-engine results: Rodrigo Liang is the CEO of SambaNova Systems, a company that sells AI inference chips. A WEF contributor op-ed by a vendor CEO with a direct commercial stake in "inference is the dominant, growing workload" being true is a Tier 3-at-best source for this specific statistic under the sourcing floor's own logic (named human, real-ish venue, but an incentive-laden opinion platform, not a measurement or dataset) — and it is one link further from any actual compute-measurement methodology than even Deloitte's own report.

Conclusion for this claim: the "two-thirds" figure that anchors claim-inference-dominant-ai-compute-2026's framing has now been traced past its most commonly-cited restatement (Deloitte) to its underlying citation (a vendor CEO's advocacy op-ed), and at no point in that chain does a primary compute-measurement source appear. This does not overturn the existing [unverified-quant — needs primary] flag on that claim-note — it corroborates it with a specific reason to distrust the number rather than merely an absence of a better one. Recorded here as [unverified-quant — needs primary], same as the original note, with the new citation-chain evidence attached.

Provenance: https://www.deloitte.com/us/en/insights/industry/technology/technology-media-and-telecom-predictions/2026/compute-power-ai.html — Tier 3. Underlying citation: https://www.weforum.org/stories/2025/01/scaling-ai-without-breaking-the-grid/ — Tier 3, wording [unsourced — needs verification], authorship/incentive facts independently confirmed.

Answering the core question

The candidate bridge claim in question-bridge-training-inference-compute had two parts: (a) a mechanism claim — training is backpropagation's domain (forward + backward pass), inference is the forward pass alone — and (b) an implicit magnitude claim — that this asymmetry is what makes inference "the dominant AI compute workload" today, giving claim-inference-dominant-ai-compute-2026 the grounding it currently lacks.

Part (a) is now confirmed at Tier 1, cleanly, via Epoch AI's own compute-economics research (Claims 1–4 above). This is a genuine, load-bearing, correctly-sourced bridge between the two clusters: the algorithm at the center of the backpropagation-origins cluster (reverse-mode differentiation / the backward pass) is precisely the extra computational step that training pays and inference does not, and that omission is the mechanistic reason inference is cheaper per unit — a fact independently derivable from first principles (Erdil's forward/backward-pass accounting) and empirically illustrated (Villalobos & Atkinson's GPT-3 FLOPs example).

Part (b) is not confirmed by this research, and the attempt to confirm it surfaced a specific, checkable reason to doubt it: the vault's currently-cited "two-thirds of compute" figure traces to a chip-vendor CEO's advocacy op-ed, not a measurement. What Claim 5 offers instead is a real, if narrower and Tier-2, empirical data point (one lab, one year, roughly even split between training and inference spend, with research/experiments actually the largest single category) — a figure that is consistent with Claim 4's theoretical "comparable spending" prediction but does not itself establish an industry-wide "inference now dominates" trend.

Recommended framing for the eventual claim-note this capture is meant to seed: write the bridge as the mechanism claim (a), sourced to Claims 1–4, and treat it as fully separable from the magnitude claim in claim-inference-dominant-ai-compute-2026, which should keep its own [unverified-quant] flag rather than borrowing false confidence from the newly-sourced mechanism. The two clusters can be linked correctly without over-claiming the economic-dominance framing that motivated the original question.

Further leads

• The still-dangling claim-inference-engineering-emerged-as-specialty (referenced by three notes, never written) sits on the same seam as this bridge; deciding write-or-unlink was explicitly deferred to "when the bridge note lands" — this capture is that landing point, so the decision is now unblocked.
• Epoch AI's "final training run vs. total R&D compute" piece (Denain & Wu, 2026-03-23, https://epoch.ai/gradient-updates/r-and-d-vs-training-compute) is a Tier 1 source not yet used anywhere in the vault; it complicates "training compute" itself (only ~10% of a frontier lab's R&D compute goes to the released model's final run) and could sharpen claim-inference-dominant-ai-compute-2026's framing once that note is re-sourced.
• The WEF op-ed (Rodrigo Liang, 2025-01-03) could not be fetched directly (403). Worth a retry via an archived copy (Wayback Machine) to confirm or refute whether it actually contains the "two-thirds" figure at all, since even Deloitte's own citation of it as the source for that number could not be verified word-for-word this session.
• No industry-wide, Tier 1–2, primary-measurement source for "share of global AI compute spent on inference vs. training" was found in this session despite a direct search for one. This looks like a genuine gap in the public record, not a search failure — worth logging as its own open question (a companion to claim-inference-dominant-ai-compute-2026's existing flag) rather than assuming the next search will find it.