---
title: "Compute-optimal training"
type: "entity"
entity_kind: "concept"
status: "hub"
canonical_name: "compute-optimal training"
aliases: ["compute-optimal allocation","Chinchilla-optimal"]
first_seen: "2026-08-22T00:00:00.000Z"
writer_model: "claude-sonnet-5"
connects_to: ["Chinchilla","neural scaling laws","Jordan Hoffmann","Jared Kaplan","IsoFLOP profile"]
seek_code_commit: "17d9798"
---


The methodological question of how to split a fixed training-compute
budget between model size (parameter count) and training-data volume
(token count) so as to minimize final loss, rather than maximizing either
variable alone. Named and formalized by
[[entity-jordan-hoffmann|Hoffmann et al.]]'s 2022 Chinchilla paper
(arXiv:2203.15556), which fit the compute-optimal exponents via three
independent methods and found the answer is roughly an equal split — for
every doubling of parameters, tokens should also double.

Matters to this vault as the concept that reframed
[[entity-jared-kaplan|Jared Kaplan]]'s 2020 scaling-law exponents from
"the confirmed rate of returns to scale" into "a rate measured under a
since-corrected allocation rule": the exponents themselves
([[claim-kaplan-2020-scaling-law-exponents-are-small-diminishing-returns|Kaplan's]]
vs.
[[claim-hoffmann-2022-loss-decay-exponents-are-3x-larger-than-kaplans|Hoffmann's]])
disagree by 3-4x partly because Kaplan's training recipe was itself not
compute-optimal by this later standard. It is already load-bearing enough
in the vault's AI-scaling cluster to have been listed in Kaplan's own
entity page before this page existed.

## References
- [[claim-hoffmann-2022-compute-optimal-scaling-splits-equally-between-parameters-and-data]]
- [[claim-hoffmann-2022-chinchilla-70b-outperforms-larger-undertrained-contemporaries]]
- [[claim-hoffmann-2022-loss-decay-exponents-are-3x-larger-than-kaplans]]
- [[claim-kaplan-2020-scaling-law-exponents-are-small-diminishing-returns]]