Chinchilla (70B parameters, 1.4T tokens, same compute as Gopher) uniformly outperforms Gopher (280B), GPT-3 (175B), Jurassic-1 (178B), and Megatron-Turing NLG (530B)

neural-scaling-lawschinchillahoffmann-2022compute-optimal-trainingbenchmarkquantitativeai

Hoffmann et al. (arXiv:2203.15556, 2022) validate their corrected compute-optimal allocation rule empirically: they train a 70-billion-parameter model ("Chinchilla") on 1.4 trillion tokens — four times fewer parameters than DeepMind's own Gopher (280B), four times more training tokens, the same total training compute — and report that it "uniformly and significantly outperforms Gopher (280B), GPT-3 (175B), Jurassic-1 (178B), and Megatron-Turing NLG (530B) on a large range of downstream evaluation tasks." Chinchilla is 4x smaller than Gopher at identical training compute — the controlled comparison, won simply by allocating that fixed budget to data rather than parameters. The other three models range from 2.5x (GPT-3, Jurassic-1) to 7.6x (Megatron-Turing NLG) Chinchilla's size, but those pairings are not compute-matched: by the paper's own token counts (Table 1) and its ≈6ND FLOPs rule (Appendix F), GPT-3 and Jurassic-1 were trained with less total compute than Chinchilla, Megatron-Turing NLG with more.

This is the paper's empirical proof-of-claim, distinct from the exponent-fitting result it validates: the exponents in claim-hoffmann-2022-compute-optimal-scaling-splits-equally-between-parameters-and-data are a statistical fit across 400+ training runs, while this is a single, specific, named head-to-head comparison against four contemporaneous production models. It is the number that made the paper's correction stick in the field — "Chinchilla-optimal" entered common LLM-training usage on the strength of this result, not the underlying regression.