---
date_created: "2026-07-06T00:00:00.000Z"
title: "MOC: Inference — the word, the mechanics, the economics"
type: "moc"
tags: ["moc","inference","LLM","economics","test-time-compute"]
updated: "2026-07-12T00:00:00.000Z"
---


The June-4 cluster: one word traced from Latin through logic, statistics, and
expert systems into the phase of a model's life that now dominates AI compute.
Trust note: the economics notes below carry `[unverified-quant]` flags and
`audit_status: flagged` — their headline numbers await re-sourcing to
primaries (topic queued 2026-07-06). The mechanics and definitional notes are
in better standing.

## The word
- [[claim-inference-word-etymology]] — Latin *inferre*, "to carry in"
- [[claim-inference-logical-types]] — deduction / induction / abduction
- [[claim-statistical-inference-meaning]] — and the ML inversion of it
- [[claim-inference-classic-ai-engines]] — the expert-systems sense

## The mechanics
- [[claim-ai-inference-means-running-a-model]] — the contemporary sense
- [[claim-llm-inference-prefill-decode]] — two phases, two bottlenecks ⚑
- [[claim-kv-cache-grows-with-context]] — the memory structure underneath ⚑

## The economics (⚑ = unverified-quant/mechanism flags, see frontmatter)
- [[claim-inference-cost-collapsed-280x]] ⚑
- [[claim-inference-dominant-ai-compute-2026]] ⚑

## The bridge to the origins cluster
- [[claim-training-inference-compute-asymmetry-mechanism]] — training pays the backward pass, inference doesn't (Tier 1, Epoch)
- [[myth-inference-two-thirds-of-compute]] — the dominance statistic, traced to a vendor op-ed (contested)

## The frontier
- [[claim-test-time-compute-can-substitute-parameters]] — compute vs. parameters
- [[claim-extended-thinking-as-serial-inference-compute]] — Anthropic's serial path

## The physical constraint — inference demand meets the power grid (2026-07-09)
The baseload character of inference load turns AI compute into an electricity
problem. The nuclear deals are Tier-1; the grid figure is Tier-2 secondary
(flagged, awaiting the LBNL primary).
- [[claim-microsoft-constellation-tmi-restart-crane-clean-energy-center]] — the light-water restart path
- [[claim-kairos-kp-fhr-molten-salt-triso-reactor]] — the non-light-water molten-salt path
- [[claim-us-interconnection-queue-exceeds-installed-capacity]] — the transmission bottleneck behind both ⚑

## The regulatory geography — the Texas grid island (2026-07-11)
Where AI load is concentrating fastest, the bottleneck is also a 90-year-old
jurisdictional artifact. Historical notes are single-Tier-3-sourced and the ERCOT
figure is Tier-2 trade press (both flagged, primaries routed to 50-questions).
- [[claim-texas-grid-isolation-1935-federal-regulation-dodge]] — isolation as a 1935 PUHCA regulatory dodge, not engineering ⚑
- [[claim-1976-midnight-connection-texas-oklahoma-forced-jurisdiction]] — the isolation actively defended once before, in 1976 ⚑
- [[claim-ercot-large-load-queue-quadrupled-2025-phantom-load]] — the demand-side queue quadrupling, and the "phantom load" uncertainty ⚑

## The adversarial axis (2026-07-11)
The same demand-side coupling, turned into an attack surface: the compute doesn't
just draw power, it can conduct the grid like an instrument.
- [[claim-bit2watt-gpu-scheduling-destabilizes-power-grid]] — GPU scheduling alone destabilizes a data center's local grid, no physical access needed
- [[claim-power-side-channel-attacks-scaled-chip-to-grid-1999-2026]] — the 27-year lineage this scales from (1999 chip → 2018 board → 2026 grid) ⚑

## The OS ancestry — KV-cache management re-imports 1960s virtual memory (2026-07-12)
A cross-time bridge: the mechanics above ([[claim-kv-cache-grows-with-context]])
turn out to be a near-literal re-derivation of a 1959–1968 operating-systems
playbook, down to the failure-mode vocabulary — with one place the analogy breaks.
- [[claim-pagedattention-borrows-os-virtual-memory-paging]] — vLLM's explicit "blocks as pages, tokens as bytes, requests as processes" (Tier 1)
- [[claim-atlas-1959-first-virtual-memory-one-level-store]] — the 1959 origin, contested for a decade (Tier 2)
- [[claim-denning-1968-working-set-model-solves-thrashing]] — the named failure mode and its fix (Tier 2)
- [[claim-llm-serving-vllm-reimports-os-preemption-and-thrashing]] — the vocabulary's round trip into 2020s serving (Tier 1/3)
- [[claim-streamingllm-attention-sinks-beat-pure-recency-eviction]] — where the locality analogy breaks (Tier 1)

## Dangling
- [[claim-inference-engineering-emerged-as-specialty]] — referenced by three
  notes but never written; either write it or unlink it (flagged to the gap
  question, 2026-07-06).