---
title: "Kaplan et al. (2020) neural-scaling-law exponents are small (~0.05–0.095), meaning 'arbitrary' scaling is real but logarithmically diminishing"
type: "claim"
status: "seedling"
source_url: "https://arxiv.org/abs/2001.08361"
source_title: "Scaling Laws for Neural Language Models"
source_author: "Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, Dario Amodei"
source_date: "2020-01-23T00:00:00.000Z"
source_quote: "Performance improves predictably as long as we scale up N and D in tandem, but enters a regime of diminishing returns if either N or D is held fixed while the other increases."
source_tier: 1
provenance: "Promotion from 10-inbox/raw/2026-07-27-hop-bitter-lesson-scaling-brake.md, 2026-07-28 (headless)"
origin: "batch"
derived_from: ["20260727-1753-hop-bitter-lesson-scaling-brake"]
date_created: "2026-07-28T00:00:00.000Z"
writer_model: "claude-sonnet-5"
tags: ["neural-scaling-laws","scaling-laws","kaplan-2020","diminishing-returns","quantitative","ai"]
audits: ["2026-07-29 claude-fable-5"]
---


Kaplan et al., "Scaling Laws for Neural Language Models" (arXiv:2001.08361, 2020), report that language-model test loss falls as a power law in three variables, with exponents far smaller than the phrase "power law" tends to suggest: α_N≈0.076 in parameter count, α_D≈0.095 in dataset size, and α_C_min≈0.050 in compute. The paper states directly that "performance improves predictably as long as we scale up N and D in tandem, but enters a regime of diminishing returns if either N or D is held fixed while the other increases." The practical reading: each order-of-magnitude increase in scale buys a comparatively small, and shrinking, reduction in loss — real returns, but logarithmic rather than proportional ones.

This is the missing rate in [[claim-sutton-2019-bitter-lesson-names-pattern-silent-on-rate|Sutton's "Bitter Lesson" (2019)]], whose claim that computation-leveraging methods "continue to scale" never quantifies the return on that scaling. It is also the figure [[observation-population-scaled-improvement-hits-a-sublinear-brake-across-domains|the vault's cross-domain sub-linear-brake bridge]] needed for its AI leg, closing the Kaplan half of [[question-verify-neural-scaling-law-exponents-kaplan-hoffmann]] — the Hoffmann/Chinchilla (2022) half of that question remains open.

> [!note] Seek's commentary:
> Popular "just scale it" framing pictures a lever: pull harder, get proportionally more. The actual exponents are closer to a doorknob already turned as far as it goes — every extra degree past that point costs an order of magnitude of compute for a rounding error of loss. Sutton was right that scaling wins. He just never priced what winning costs.
> — Seek