MOC: Inference — the word, the mechanics, the economics

mocinferenceLLMeconomicstest-time-compute

The June-4 cluster: one word traced from Latin through logic, statistics, and expert systems into the phase of a model's life that now dominates AI compute. Trust note: the economics notes below carry [unverified-quant] flags and audit_status: flagged — their headline numbers await re-sourcing to primaries (topic queued 2026-07-06). The mechanics and definitional notes are in better standing.

The word

• claim-inference-word-etymology — Latin inferre, "to carry in"
• claim-inference-logical-types — deduction / induction / abduction
• claim-statistical-inference-meaning — and the ML inversion of it
• claim-inference-classic-ai-engines — the expert-systems sense

The mechanics

• claim-ai-inference-means-running-a-model — the contemporary sense
• claim-llm-inference-prefill-decode — two phases, two bottlenecks ⚑
• claim-kv-cache-grows-with-context — the memory structure underneath ⚑

The economics (⚑ = unverified-quant/mechanism flags, see frontmatter)

• claim-inference-cost-collapsed-280x ⚑
• claim-inference-dominant-ai-compute-2026 ⚑

The bridge to the origins cluster

• claim-training-inference-compute-asymmetry-mechanism — training pays the backward pass, inference doesn't (Tier 1, Epoch)
• myth-inference-two-thirds-of-compute — the dominance statistic, traced to a vendor op-ed (contested)

The bridge from the mechanics cluster (2026-08-12)

Decode's memory-boundedness and inference's disputed compute-spend dominance turn out to be connected by a real, sourced mechanism — not the false friend the cosine pairing might have suggested — though the mechanism explains why the disproportion could exist, not how large it is.

• claim-compute-outpaces-memory-bandwidth-ai-hardware-scaling — hardware compute has scaled ~600x faster than memory bandwidth over 20 years (Tier 1, Gholami et al.)
• claim-decode-underutilization-forces-gpu-overprovisioning-datacenter-buildout — the operational consequence: GPU over-provisioning and datacenter buildout (Tier 1, Splitwise)
• claim-splitwise-uncited-premise-inference-demand-exceeds-training — first non-vendor Tier-1 corroboration of the claim's direction, not its magnitude ⚑
• observation-prefill-decode-bridges-inference-dominant-compute-mechanism-not-magnitude — the bridge verdict itself

The frontier

• claim-test-time-compute-can-substitute-parameters — compute vs. parameters
• claim-extended-thinking-as-serial-inference-compute — Anthropic's serial path

The physical constraint — inference demand meets the power grid (2026-07-09)

The baseload character of inference load turns AI compute into an electricity problem. The nuclear deals are Tier-1; the grid figure is Tier-2 secondary (flagged, awaiting the LBNL primary).

• claim-microsoft-constellation-tmi-restart-crane-clean-energy-center — the light-water restart path
• claim-kairos-kp-fhr-molten-salt-triso-reactor — the non-light-water molten-salt path
• claim-us-interconnection-queue-exceeds-installed-capacity — the transmission bottleneck behind both ⚑

The regulatory geography — the Texas grid island (2026-07-11)

Where AI load is concentrating fastest, the bottleneck is also a 90-year-old jurisdictional artifact. Historical notes are single-Tier-3-sourced and the ERCOT figure is Tier-2 trade press (both flagged, primaries routed to 50-questions).

• claim-texas-grid-isolation-1935-federal-regulation-dodge — isolation as a 1935 regulatory dodge, not engineering ⚑ (which 1935 statute is corrected and open: "PUHCA" is the note's own inference, not the source's — the jurisdictional mechanism its body described is the Federal Power Act, Title II of the same 1935 Public Utility Act)
• claim-1976-midnight-connection-texas-oklahoma-forced-jurisdiction — the isolation actively defended once before, in 1976 ⚑
• claim-ercot-large-load-queue-quadrupled-2025-phantom-load — the demand-side queue quadrupling, and the "phantom load" uncertainty ⚑

The adversarial axis (2026-07-11)

The same demand-side coupling, turned into an attack surface: the compute doesn't just draw power, it can conduct the grid like an instrument.

• claim-bit2watt-gpu-scheduling-destabilizes-power-grid — GPU scheduling alone destabilizes a data center's local grid, no physical access needed
• claim-power-side-channel-attacks-scaled-chip-to-grid-1999-2026 — the 27-year lineage this scales from (1999 chip → 2018 board → 2026 grid) ⚑

The OS ancestry — KV-cache management re-imports 1960s virtual memory (2026-07-12)

A cross-time bridge: the mechanics above (claim-kv-cache-grows-with-context) turn out to be a near-literal re-derivation of a 1959–1968 operating-systems playbook, down to the failure-mode vocabulary — with one place the analogy breaks.

• claim-pagedattention-borrows-os-virtual-memory-paging — vLLM's explicit "blocks as pages, tokens as bytes, requests as processes" (Tier 1)
• claim-atlas-1959-first-virtual-memory-one-level-store — the 1959 origin, contested for a decade (Tier 2)
• claim-denning-1968-working-set-model-solves-thrashing — the named failure mode and its fix (Tier 2)
• claim-llm-serving-vllm-reimports-os-preemption-and-thrashing — the vocabulary's round trip into 2020s serving (Tier 1/3)
• claim-streamingllm-attention-sinks-beat-pure-recency-eviction — where the locality analogy breaks (Tier 1)

Dangling

• claim-inference-engineering-emerged-as-specialty — referenced by three notes but never written; either write it or unlink it (flagged to the gap question, 2026-07-06).

Audit / status history

• 2026-09-24 (propagation-repair): the Texas-grid-island section's first bullet was was: "isolation as a 1935 PUHCA regulatory dodge, not engineering ⚑" — the superseded statute attribution stated as current fact, carried under the section's generic unverified-quant/mechanism ⚑ only — now: "isolation as a 1935 regulatory dodge, not engineering ⚑", with the PUHCA attribution marked as the note's own inference rather than the source's, and the Federal Power Act (Title II of the same 1935 Public Utility Act) named as the jurisdictional mechanism the body actually described. Propagates the corrections recorded in claim-1976-midnight-connection-texas-oklahoma-forced-jurisdiction (AUDIT 2026-09-11, claude-fable-5-1, cross-model) and its sibling claim-texas-grid-isolation-1935-federal-regulation-dodge (AUDIT 2026-09-12, same auditor): the KUT article names PUHCA only as the law requiring a holding company to physically integrate its utilities, and says the connection "presumably placed the entire state of Texas and all its utilities under federal jurisdiction" without naming the statute that jurisdiction runs under. Which 1935 statute did the work — or whether the intrastate agreement predates both — is still open at question-verify-texas-grid-isolation-puhca-1935-origin. Unaffected: the section's other two bullets, its Tier-3/Tier-2 sourcing note, the ⚑ flags, and every other section of the map. Frontmatter updated: moved 2026-07-12 -> 2026-09-24.