---
title: "Modern LLM-serving systems reimport OS memory-pressure vocabulary intact — vLLM preempts and recomputes KV cache, and llm-d names cache-eviction churn 'cache thrashing'"
type: "claim"
status: "seedling"
audit_status: "capture-verified"
writer_model: "claude-sonnet-5"
source_url: "https://arxiv.org/abs/2309.06180"
source_title: "Efficient Memory Management for Large Language Model Serving with PagedAttention"
source_author: "Kwon, Li, Zhuang, Sheng, Zheng, Yu, Gonzalez, Zhang, Stoica"
source_date: "2023-09-12"
source_quote: "can preempt requests… recomputed when sufficient KV cache space becomes available"
source_tier: 1
source_url_2: "https://llm-d.ai/blog/kvcache-wins-you-can-see"
source_tier_2: 3
source_quote_2: "This instability is caused by 'cache thrashing.'"
provenance: "Promotion from 10-inbox/raw/2026-07-11-hop-os-virtual-memory-in-llm-serving.md, 2026-07-12"
origin: "batch"
derived_from: "10-inbox/raw/2026-07-11-hop-os-virtual-memory-in-llm-serving.md"
date_created: "2026-07-12T00:00:00.000Z"
tags: ["LLM","inference","KV-cache","vLLM","llm-d","thrashing","virtual-memory","operating-systems"]
related_notes: ["claim-pagedattention-borrows-os-virtual-memory-paging","claim-denning-1968-working-set-model-solves-thrashing","claim-atlas-1959-first-virtual-memory-one-level-store","claim-streamingllm-attention-sinks-beat-pure-recency-eviction"]
drafted_in: ["1968-had-no-word","2026-07-13-1968-had-no-word"]
---


The borrow from operating-systems virtual memory into LLM serving (see [[claim-pagedattention-borrows-os-virtual-memory-paging]]) did not stop at data-structure design — it carried the OS's *failure-mode vocabulary* along with it. Under memory pressure, vLLM "can preempt requests… recomputed when sufficient KV cache space becomes available" — the same preemption-and-recomputation strategy operating systems use to survive over-multiprogramming beyond what [[claim-denning-1968-working-set-model-solves-thrashing|the working-set model]] budgets for.

The vocabulary returned by name, not just by mechanism: the LLM-serving platform **llm-d** describes cache-blind eviction churn directly as "cache thrashing" — "This instability is caused by 'cache thrashing.'" That is [[claim-denning-1968-working-set-model-solves-thrashing|Denning's 1968 term]], applied to a 2020s GPU memory hierarchy, by engineers who were solving a scarce-fast/abundant-slow memory problem structurally identical to the one multiprogrammed mainframes hit in the 1960s.

The vLLM preemption mechanism is Tier 1 (the PagedAttention system paper); the specific "cache thrashing" naming is sourced to a Tier-3 vendor blog and should be read as evidence of *industry usage* of the term rather than an independently verified causal claim about llm-d's internals.

> [!note] Seek's commentary:
> This is the part of the capture I find most convincing as a genuine cross-time bridge rather than a cute coincidence: it's not enough that the *data structure* (pages/blocks) got reused — the *failure mode got a name transplant too*. That's a stronger signal of real structural identity than shared vocabulary alone would be.
> — Seek