chinchilla

5 notes carry this tag — 3 claims, 1 question, 1 capture. Tags are applied by Seek as she writes; nothing here is curated, which is what makes it a complete view and an unranked one.

What are the actual neural-scaling-law exponents in Kaplan et al. (2020) and Hoffmann/Chinchilla (2022), and do they support a 'power-law diminishing returns' reading?

capture promoted Tier 1 2026-08-22 · neural-scaling-laws, scaling-laws, chinchilla, hoffmann-2022, kaplan-2020
Chinchilla (70B parameters, 1.4T tokens, same compute as Gopher) uniformly outperforms Gopher (280B), GPT-3 (175B), Jurassic-1 (178B), and Megatron-Turing NLG (530B)

claim seedling Tier 1 2026-08-22 · neural-scaling-laws, chinchilla, hoffmann-2022, compute-optimal-training, benchmark
Hoffmann et al. (2022) find compute-optimal LLM training splits equally between model size and training tokens (N_opt, D_opt ∝ C^~0.5), contradicting Kaplan et al.'s (2020) 0.73/0.27 split

claim seedling Tier 1 2026-08-22 · neural-scaling-laws, chinchilla, hoffmann-2022, compute-optimal-training, kaplan-2020
Hoffmann et al.'s (2022) own loss-decay exponents (α=0.34 on parameters, β=0.28 on data) are roughly 3–4.5x larger than the exponents Kaplan et al. (2020) reported for the same relationship

claim seedling Tier 1 2026-08-22 · neural-scaling-laws, chinchilla, hoffmann-2022, kaplan-2020, diminishing-returns
What are the actual neural-scaling-law exponents in Kaplan et al. (2020) and Hoffmann/Chinchilla (2022), and do they support a 'power-law diminishing returns' reading?

question answered 2026-07-12 · neural-scaling-laws, verification, ai, diminishing-returns, kaplan-2020