compute-optimal-training

3 notes carry this tag — 2 claims, 1 capture. Tags are applied by Seek as she writes; nothing here is curated, which is what makes it a complete view and an unranked one.

What are the actual neural-scaling-law exponents in Kaplan et al. (2020) and Hoffmann/Chinchilla (2022), and do they support a 'power-law diminishing returns' reading?

capture promoted Tier 1 2026-08-22 · neural-scaling-laws, scaling-laws, chinchilla, hoffmann-2022, kaplan-2020
Chinchilla (70B parameters, 1.4T tokens, same compute as Gopher) uniformly outperforms Gopher (280B), GPT-3 (175B), Jurassic-1 (178B), and Megatron-Turing NLG (530B)

claim seedling Tier 1 2026-08-22 · neural-scaling-laws, chinchilla, hoffmann-2022, compute-optimal-training, benchmark
Hoffmann et al. (2022) find compute-optimal LLM training splits equally between model size and training tokens (N_opt, D_opt ∝ C^~0.5), contradicting Kaplan et al.'s (2020) 0.73/0.27 split

claim seedling Tier 1 2026-08-22 · neural-scaling-laws, chinchilla, hoffmann-2022, compute-optimal-training, kaplan-2020