---
title: "PagedAttention manages the LLM KV cache with a literal port of OS virtual-memory paging — blocks as pages, tokens as bytes, requests as processes"
type: "claim"
status: "seedling"
audit_status: "capture-verified"
writer_model: "claude-sonnet-5"
source_url: "https://arxiv.org/abs/2309.06180"
source_title: "Efficient Memory Management for Large Language Model Serving with PagedAttention"
source_author: "Kwon, Li, Zhuang, Sheng, Zheng, Yu, Gonzalez, Zhang, Stoica"
source_date: "2023-09-12"
source_quote: "PagedAttention, an attention algorithm inspired by the operating system's (OS) solution to memory fragmentation and sharing: virtual memory with paging"
source_tier: 1
provenance: "Promotion from 10-inbox/raw/2026-07-11-hop-os-virtual-memory-in-llm-serving.md, 2026-07-12"
origin: "batch"
derived_from: "10-inbox/raw/2026-07-11-hop-os-virtual-memory-in-llm-serving.md"
date_created: "2026-07-12T00:00:00.000Z"
tags: ["LLM","inference","KV-cache","PagedAttention","vLLM","virtual-memory","operating-systems","memory-bandwidth"]
related_notes: ["claim-kv-cache-grows-with-context","claim-llm-inference-prefill-decode","claim-memory-wall-named-1994-wulf-mckee","claim-atlas-1959-first-virtual-memory-one-level-store","claim-llm-serving-vllm-reimports-os-preemption-and-thrashing"]
drafted_in: ["1968-had-no-word","2026-07-13-1968-had-no-word"]
---


vLLM's **PagedAttention** (Kwon et al., SOSP 2023, [arXiv 2309.06180](https://arxiv.org/abs/2309.06180)) is not a loose analogy to operating-systems memory management — the paper states the borrow directly: it is "an attention algorithm inspired by the operating system's (OS) solution to memory fragmentation and sharing: virtual memory with paging." The correspondence is stated as a literal mapping: "one can think of blocks as pages, tokens as bytes, and requests as processes." The [[claim-kv-cache-grows-with-context|KV cache]] is split into fixed-size, non-contiguous blocks (pages) rather than requiring one contiguous memory allocation per request, which is what conventional serving systems did.

The problem this solved was severe: prior systems used only "20.4%–38.2% of the KV cache memory... to store the actual token states," the rest lost to internal and external fragmentation from over-provisioned contiguous allocations. Eliminating that waste, alongside memory sharing across requests (e.g. shared prompt prefixes), is credited with a 2–4× throughput improvement over prior serving systems.

This is a cross-domain, not merely metaphorical, transfer: a 1959–1968 [[claim-atlas-1959-first-virtual-memory-one-level-store|operating-systems solution to a scarce-fast/abundant-slow memory hierarchy]] was re-derived, largely intact, for GPU-bound transformer inference in 2023. See [[claim-llm-serving-vllm-reimports-os-preemption-and-thrashing]] for how the borrow extends past structure into OS *failure-mode* vocabulary.

> [!note] Seek's commentary:
> The tell that this is a real borrow rather than a coincidental naming choice is the fragmentation number — 20–38% utilization is exactly the kind of waste virtual memory with paging was built to eliminate in 1959–1968, and vLLM's fix has the same shape (fixed-size blocks, indirection, no contiguity requirement).
> — Seek