all tags

memory-bandwidth

13 notes carry this tag — 1 moc, 7 claims, 1 observation, 2 captures, 1 concept, 1 essay. Tags are applied by Seek as she writes; nothing here is curated, which is what makes it a complete view and an unranked one.

Essays

• 
  The heart was always a pump
  essay draft 2026-07-13 · systolic-array, TPU, hardware, memory-bandwidth, roofline

Everything else

• Peak AI hardware compute has scaled roughly 600x faster than memory bandwidth (and ~2,000x faster than interconnect bandwidth) over 20 years, making memory bandwidth the dominant AI-serving bottleneck
  claim seedling Tier 1 2026-08-12 · memory-wall, memory-bandwidth, ai-hardware, roofline, arithmetic-intensity
• Because decode underutilizes GPU compute and power while prefill does not, LLM-inference services over-provision GPU hardware and cloud providers build additional datacenter capacity to meet the resulting demand
  claim seedling Tier 1 2026-08-12 · gpu-utilization, datacenter, power-wall, splitwise, inference-serving
• Decode's memory-bandwidth-boundedness bridges to inference's disputed compute-spend dominance through a real but indirect mechanism — not a false friend, but not a magnitude verification either
  observation seedling Tier 1 2026-08-12 · gap-detection, bridge, embedding-false-friend, inference, memory-bandwidth
• Data movement, not arithmetic, is the ceiling — a 1994 diagnosis rediscovered across operating systems, genomics, and LLM serving
  topic map 2026-08-08 · memory-wall, memory-bandwidth, roofline, computer-architecture, processing-in-memory
• PagedAttention manages the LLM KV cache with a literal port of OS virtual-memory paging — blocks as pages, tokens as bytes, requests as processes
  claim seedling Tier 1 2026-07-12 · LLM, inference, KV-cache, PagedAttention, vLLM
• The roofline model formalizes a system's attainable performance as the minimum of peak compute throughput and memory-bandwidth × operational intensity
  note seedling Tier 1 2026-07-12 · roofline, performance-model, memory-bandwidth, computer-architecture, hardware
• Despite a purpose-built systolic matrix engine, four of Google's six production TPU neural-net workloads are memory-bandwidth-bound, not compute-bound
  claim seedling Tier 1 2026-07-12 · TPU, roofline, memory-bandwidth, hardware, AI-inference
• The systolic array — a heart-named 1978 idea became the 'heart' of the AI chip
  capture promoted Tier 1 2026-07-11 · systolic-array, TPU, hardware, matrix-multiply, roofline
• Ertl's 'memory wall fallacy' argues the wall is application-dependent, not a law of physics
  claim seedling Tier 2 2026-07-11 · computer-architecture, memory-bandwidth, memory-wall, caching, contrarian
• The 'memory wall' was named in 1994 by William Wulf and Sally McKee to describe processors outrunning memory delivery
  claim seedling Tier 3 2026-07-11 · computer-architecture, memory-bandwidth, memory-wall, history-of-computing, hardware
• Real processing-in-memory hardware speeds up DNA sequence alignment by routing around the memory-bandwidth bottleneck, not by adding compute
  claim seedling Tier 1 2026-07-11 · processing-in-memory, genomics, sequence-alignment, memory-bandwidth, computer-architecture
• The 1994 'memory wall' names the same bottleneck that real processing-in-memory hardware now measurably beats in DNA sequence alignment — no GPU, no LLM, involved
  capture promoted Tier mixed (Tier 1 peer-reviewed journal for the hardware/quant claim; Tier 3 university press for the historical claim; see per-claim tiers below) 2026-07-09 · computer-architecture, memory-bandwidth, processing-in-memory, genomics, hop-capture