PagedAttention manages the LLM KV cache with a literal port of OS virtual-memory paging — blocks as pages, tokens as bytes, requests as processes

vLLM's PagedAttention (Kwon et al., SOSP 2023, arXiv 2309.06180) is not a loose analogy to operating-systems memory management — the paper states the borrow directly: it is "an attention algorithm inspired by the operating system's (OS) solution to memory fragmentation and sharing: virtual memory with paging." The correspondence is stated as a literal mapping: "one can think of blocks as pages, tokens as bytes, and requests as processes." The KV cache is split into fixed-size, non-contiguous blocks (pages) rather than requiring one contiguous memory allocation per request, which is what conventional serving systems did.

The problem this solved was severe: prior systems used only "20.4%–38.2% of the KV cache memory... to store the actual token states," the rest lost to internal and external fragmentation from over-provisioned contiguous allocations. Eliminating that waste, alongside memory sharing across requests (e.g. shared prompt prefixes), is credited with a 2–4× throughput improvement over prior serving systems.

This is a cross-domain, not merely metaphorical, transfer: a 1959–1968 operating-systems solution to a scarce-fast/abundant-slow memory hierarchy was re-derived, largely intact, for GPU-bound transformer inference in 2023. See claim-llm-serving-vllm-reimports-os-preemption-and-thrashing for how the borrow extends past structure into OS failure-mode vocabulary.