The 'memory wall' was named in 1994 by William Wulf and Sally McKee to describe processors outrunning memory delivery

The memory wall is the name given to a structural mismatch in computer architecture: processor speed improved far faster than the rate at which memory could deliver data, so an increasing share of a program's time is spent waiting on memory rather than computing. The term was coined by William Wulf and his then–University of Virginia graduate student Sally McKee around 1994, in a short paper published in ACM SIGARCH Computer Architecture News.

Per a University of Virginia retrospective: "The memory wall results from two issues: outdated computing architecture, with a physical separation between computer processors and memory; and the fact that a processor can run much faster than the speed at which memory chips can provide data." (UVA Today)

The diagnosis is architectural rather than incidental. In the conventional von Neumann arrangement, compute and memory are physically separated, and data must be shipped across a comparatively slow interface for every operation. Because CPU performance historically tracked Moore's-Law-pace gains while DRAM latency and bandwidth improved only single-digit percentages per year, the gap widened structurally — the "wall" is the point at which memory delivery, not arithmetic, sets the ceiling on performance.

The same bottleneck is what modern LLM-inference engineering calls memory-bandwidth-bound: the decode phase of transformer inference is limited by how fast the KV cache can be streamed from memory, not by compute. The memory wall is thus the 30-year-old named ancestor of a bottleneck the LLM-serving literature rediscovered independently. Two architectural responses to the same wall run through the vault: processing-in-memory hardware (put compute inside the memory) and the counter-argument that the wall is application-dependent rather than a law of physics.