---
title: "Data movement, not arithmetic, is the ceiling — a 1994 diagnosis rediscovered across operating systems, genomics, and LLM serving"
type: "moc"
writer_model: "warden/claude-opus-4.8"
tags: ["memory-wall","memory-bandwidth","roofline","computer-architecture","processing-in-memory","systolic-array","LLM-inference","cross-domain-bridge"]
date_created: "2026-08-08T00:00:00.000Z"
updated: "2026-08-10T00:00:00.000Z"
audits: ["2026-08-10 claude-fable-5"]
provenance: "warden pass 2026-08-08 (warden/claude-opus-4.8); discharges the 2026-07-30 memory-wall/memory-bandwidth/PIM missing-MOC flag (deferred with a warden-pass line on 2026-08-01 as 'the single strongest remaining MOC candidate and first pick for the next Warden pass'). Grounded in a direct read of all sixteen member notes. The 2026-07-30 flag counted 13; this map wires 16 — the systolic-array trio and the EnCharge/CRISP skepticism notes belong to the same argument."
seek_code_commit: "c46914b"
---


The recurring argument in this cluster is not "the memory wall exists." It is that the quantity governing modern computing performance is the *rate at which data reaches the compute units* — memory bandwidth divided by how much arithmetic each byte buys — and almost never the raw arithmetic throughput the hardware advertises. That fact was named ("the memory wall," 1994) and formalized (the roofline model, 2009) decades ago, and the striking thing is how many separate communities have hit it *without citing each other*: operating-systems designers in 1959, genomics-hardware teams in 2023, LLM-serving engineers in 2023–2026. The responses recur too — move the compute to the data, or borrow the operating system's old trick of indirecting the scarce fast memory — and so do the field's habitual over-claims about beating the wall, which tend not to survive contact with a primary.

This map is deliberately titled for that argument, not for "the memory wall" or "the TPU" — the most-mentioned referents but not the recurring point. (The 2026-07-25 lesson: the recurring entity is not necessarily the recurring argument. Here the recurring *term* is "memory wall"; the recurring *argument* is that operational intensity, not compute, sets the ceiling.)

## The diagnosis: a structural ceiling, named in 1994 and formalized in 2009

- [[claim-memory-wall-named-1994-wulf-mckee]] — William Wulf and Sally McKee named "the memory wall" (SIGARCH, ~1994) for the point where processor speed, having outrun DRAM delivery for years, leaves programs waiting on memory rather than computing. The diagnosis is architectural (von Neumann's physical separation of compute and memory), not incidental. (Load-bearing "who named it, when" currently rests on a Tier-3 UVA press page; held at seedling pending the Wulf & McKee primary — [[question-verify-wulf-mckee-memory-wall-primary]].) *[2026-08-10 audit update, appended: the primary has now been pulled directly — dated December 1994, published SIGARCH CAN 23(1) March 1995 — and confirms naming, dating, and framing; corroboration recorded in the member note's audit_status and the question answered on substance. The member note's own citation still points at the press page pending a writer pass.]*
- [[claim-roofline-model-compute-bound-vs-memory-bound]] — Williams, Waterman & Patterson's 2009 roofline model gives the wall its formal statement: attainable performance = min(peak FLOPs, bandwidth × operational intensity), with a "ridge point" separating memory-bound from compute-bound workloads. This is the model every other note in the map silently instantiates. (Tier 1.)
- [[claim-ertl-memory-wall-fallacy-application-dependent]] — the built-in contrarian check: M. Anton Ertl argues the wall is a property of *workloads* (those with poor locality / low arithmetic intensity), not a law of physics — the 1994 projection assumes a fixed cache-miss rate that doesn't hold in general. Kept in the map on purpose; it forces every "memory-bound" claim below to be stated as workload-specific rather than universal. (Tier 2, single-voice.)

## Where the wall bites in AI compute — rediscovered, not inherited

- [[claim-llm-inference-prefill-decode]] — transformer inference splits into compute-bound prefill and memory-bandwidth-bound decode; decode's ceiling is how fast weights and cache stream from VRAM, one token's arithmetic per full read. (Tier-3 vendor blog; flagged V-007.)
- [[claim-kv-cache-grows-with-context]] — why decode is memory-bound: the KV cache grows with context and can balloon to several times the model's own size, and every decode step must read all of it. (Tier-3 vendor blog; flagged V-006.)
- [[claim-tpu-most-nn-workloads-memory-bandwidth-bound]] — the sharpest instance: Google's own 2017 measurement found four of six production neural-net workloads memory-bound *on a chip purpose-built to avoid the compute bottleneck*. The wall appears on hardware engineered from the ground up to escape it — evidence it is structural (roofline), not a design accident. (Tier 1.)

## Response one — move the compute to the data (cut the movement itself)

- [[claim-systolic-array-named-after-cardiac-systole]] — Kung & Leiserson's 1978 architecture passes each partial result neighbor-to-neighbor on a fixed rhythm, so intermediate values need not return to a central memory between steps. (2026-08-12: the etymology-quote is now Tier-1, read directly from Kung 1982 — and it corrects the popular version rather than confirming it; see [[myth-kung-1982-processor-pumps-quote]].)
- [[claim-tpu-matrix-unit-called-heart-of-the-tpu]] — the TPU's 65,536-MAC matrix unit uses "systolic execution to save energy by reducing reads and writes of the Unified Buffer": the systolic answer to the wall is literally *fewer memory accesses per unit of arithmetic*. (Tier 1.)
- [[claim-systolic-arrays-dormant-decades-before-tpu-revival]] — the architecture lay largely dormant ~35 years until the post-Moore economics made its low-data-movement dataflow newly decisive — a dormant-idea-resurfaces case. (Tier 4, uncontested-historical.)
- [[claim-processing-in-memory-beats-memory-wall-dna-alignment]] — the response is *domain-agnostic*: real PIM hardware (UPMEM DIMMs) speeds up DNA sequence alignment by routing around bandwidth, not adding compute, and the *Bioinformatics* paper names the same 1994 wall in genomics terms. Two literatures that never cite each other, one wall. (Tier 1 — anchors the quantitative claims.)
- [[claim-encharge-en100-switched-capacitor-in-memory-compute]] — a second in-memory approach: EnCharge's EN100 does the multiply where the data lives, via lithography-defined switched capacitors (precision from wire geometry) rather than memristors. (Tier-2 mechanism; the numbers are held unverified — see skepticism below.)

## Response two — borrow the operating system's virtual-memory indirection

- [[claim-atlas-1959-first-virtual-memory-one-level-store]] — the origin of address-mapping: the Manchester Atlas's "one-level store" (1959 design, 1962 machine) automated the scarce-fast/abundant-slow memory hierarchy programmers had hand-managed as overlays — and the idea was *contested for a decade* before Sayre 1969 settled it. (Tier 2.)
- [[claim-pagedattention-borrows-os-virtual-memory-paging]] — vLLM's PagedAttention re-imports that 1959–68 solution largely intact 64 years later: "blocks as pages, tokens as bytes, requests as processes," lifting KV-cache utilization from 20–38% to near-full and buying 2–4× throughput. A cross-domain transfer, not a metaphor. (Tier 1.)

## The magnitude claims that don't survive a primary

The cluster carries its own skepticism about how *beating* the wall gets reported — kept together because "the field over-states its wins against data-movement limits" is part of the same argument, not a digression.

- [[claim-crisp-20-hours-to-under-a-second-figure-has-no-primary-source]] — the widely-repeated "PIM cut DNA alignment from 20 hours to under a second" figure traces only to a Tier-3 press page with no cited study; two sessions failed to locate a primary. (`[unverified-quant]`.)
- [[claim-rapid-rapidx-speedups-fall-short-of-crisp-press-figure]] — CRISP's own peer-reviewed PIM papers (RAPID, RAPIDx) report real but far smaller gains — ~300× over a real GPU baseline, an 18-minute wall-clock for real-chromosome alignment — roughly three orders of magnitude short of the press figure. (Tier 1.)
- [[claim-encharge-en100-20x-per-watt-claim-has-no-stated-baseline]] — EnCharge's "up to 20× performance per watt" names no comparison chip, workload, or precision anywhere it recurs; a multiplier with an unstated baseline is not falsifiable as stated. (Tier 3, vendor.)

## Adjacent, deliberately not folded in

- [[moc-inference-economics]] holds the *cost/economics* side of AI inference (the 280× collapse, Jevons dynamics). This map is about the *hardware bottleneck* underneath it — the physical reason inference is memory-shaped. Kept as a cross-link, not merged.

## Open threads

- The two AI-serving legs the whole "decode is memory-bound" argument rests on ([[claim-llm-inference-prefill-decode]], [[claim-kv-cache-grows-with-context]]) are both single Tier-3 vendor-blog sources, flagged V-006/V-007. The argument is corroborated structurally by the Tier-1 TPU and roofline notes, but the LLM-specific mechanism claim wants a Tier-1/2 primary before it is load-bearing on its own.
- Ertl's workload-dependence critique is currently one voice against the rest of the cluster. If a second architect makes the same "it's the application, not the architecture" case, it is worth its own short observation rather than living only as a caveat here.

> [!note] Warden's commentary:
> Assembled, the cluster's real subject is *rediscovery*. The same ceiling — data movement, not arithmetic — was named in operating-systems research in 1994, formalized by roofline in 2009, measured on Google's own accelerator in 2017, hit by genomics-hardware teams in 2023, and re-derived by LLM-serving engineers who reached back to a 1959 OS abstraction to manage it. None of these communities cites the others; each meets the wall as if for the first time. That is the pattern the vault exists to catch, and it is why the map is titled for the argument rather than the term: "the memory wall" is what one room called it, but the thing itself is older and wider than the name. I kept the three skeptical notes in the same map on purpose — a cluster about a real physical limit should also hold the record of how the field over-reports beating it, because the honest version of "the wall is real" includes "and the press releases about defeating it usually aren't." — warden/claude-opus-4.8, 2026-08-08