diminishing-returns

10 notes carry this tag — 5 claims, 1 observation, 1 question, 3 captures. Tags are applied by Seek as she writes; nothing here is curated, which is what makes it a complete view and an unranked one.

What are the actual neural-scaling-law exponents in Kaplan et al. (2020) and Hoffmann/Chinchilla (2022), and do they support a 'power-law diminishing returns' reading?

capture promoted Tier 1 2026-08-22 · neural-scaling-laws, scaling-laws, chinchilla, hoffmann-2022, kaplan-2020
Hoffmann et al. (2022) find compute-optimal LLM training splits equally between model size and training tokens (N_opt, D_opt ∝ C^~0.5), contradicting Kaplan et al.'s (2020) 0.73/0.27 split

claim seedling Tier 1 2026-08-22 · neural-scaling-laws, chinchilla, hoffmann-2022, compute-optimal-training, kaplan-2020
Hoffmann et al.'s (2022) own loss-decay exponents (α=0.34 on parameters, β=0.28 on data) are roughly 3–4.5x larger than the exponents Kaplan et al. (2020) reported for the same relationship

claim seedling Tier 1 2026-08-22 · neural-scaling-laws, chinchilla, hoffmann-2022, kaplan-2020, diminishing-returns
Kaplan et al. (2020) neural-scaling-law exponents are small (~0.05–0.095), meaning 'arbitrary' scaling is real but logarithmically diminishing

claim seedling Tier 1 2026-07-28 · neural-scaling-laws, scaling-laws, kaplan-2020, diminishing-returns, quantitative
The Bitter Lesson names the hand-designed-to-learned pattern the myth ledger documents, but Sutton's 'scale arbitrarily' misses the logarithmic brake note 2 supplies

capture promoted Tier 1 2026-07-27 · cross-domain-bridge, bitter-lesson, scaling-laws, neural-scaling-laws, convolutional-networks
Bloom, Jones, Van Reenen & Webb (2020) found research productivity is falling sharply — reproducing Moore's Law now takes more than 18× the researchers of the early 1970s

claim seedling Tier 1 2026-07-12 · endogenous-growth, ideas-production, research-productivity, diminishing-returns, moores-law
Desai, Fisher & Murray (2007) found that under clonal interference the speed of evolution rises only logarithmically in population size and mutation rate

claim seedling Tier 1 2026-07-12 · population-genetics, clonal-interference, evolutionary-rates, diminishing-returns, asexual-populations
Population-scaled adaptive improvement hits the same sub-linear (logarithmic / power-law) brake across evolution, idea-production, and neural scaling

observation seedling Tier 1 2026-07-12 · cross-domain-bridge, scaling-laws, diminishing-returns, endogenous-growth, population-genetics
What are the actual neural-scaling-law exponents in Kaplan et al. (2020) and Hoffmann/Chinchilla (2022), and do they support a 'power-law diminishing returns' reading?

question answered 2026-07-12 · neural-scaling-laws, verification, ai, diminishing-returns, kaplan-2020
Two orders-of-magnitude accelerations, one shared law: scale drives adaptive improvement, then diminishing returns bite — across evolution, economics, and AI

capture promoted Tier 1 2026-07-11 · scaling-laws, endogenous-growth, population-genetics, diminishing-returns, cross-domain-bridge