---
title: "Hoffmann et al. (2022) find compute-optimal LLM training splits equally between model size and training tokens (N_opt, D_opt ∝ C^~0.5), contradicting Kaplan et al.'s (2020) 0.73/0.27 split"
type: "claim"
status: "seedling"
source_url: "https://arxiv.org/abs/2203.15556"
source_title: "Training Compute-Optimal Large Language Models"
source_author: "Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, Laurent Sifre"
source_date: "2022-03-29T00:00:00.000Z"
source_quote: "we find that for compute-optimal training, the model size and the number of training tokens should be scaled equally: for every doubling of model size the number of training tokens should also be doubled."
source_tier: 1
audit_status: "capture-verified"
provenance: "Promotion from 10-inbox/raw/2026-08-22-what-are-the-actual-neural-scaling-law-exponents.md, 2026-08-22"
origin: "batch"
derived_from: ["20260822-0241-what-are-the-actual"]
date_created: "2026-08-22T00:00:00.000Z"
writer_model: "claude-sonnet-5"
source_sha: "3fd3632a8ef48171bd25282990221d49535d75356192f068b3b2ebe08f2aedd4"
tags: ["neural-scaling-laws","chinchilla","hoffmann-2022","compute-optimal-training","kaplan-2020","diminishing-returns","quantitative","ai"]
verified_verbatim: "2026-08-24 — source_quote matched verbatim (normalized) against a direct fetch of source_url by seek_verify (no model involved)"
seek_code_commit: "17d9798"
---


Hoffmann et al., "Training Compute-Optimal Large Language Models" (arXiv:2203.15556, 2022 — the "Chinchilla" paper), trained over 400 models (70M–16B parameters, 5B–500B tokens) to fit the compute-optimal allocation exponents in N_opt(C) ∝ C^a and D_opt(C) ∝ C^b. Three independent estimation methods converge closely: minimum-over-training-curves gives a=0.50, b=0.50; IsoFLOP profiles give a=0.49, b=0.51; a parametric loss fit gives a=0.46, b=0.54 (Table 2). The paper's abstract states the headline directly: "we find that for compute-optimal training, the model size and the number of training tokens should be scaled equally: for every doubling of model size the number of training tokens should also be doubled."

This directly contradicts [[claim-kaplan-2020-scaling-law-exponents-are-small-diminishing-returns|Kaplan et al.'s (2020) own reported allocation exponents]] (a=0.73, b=0.27), which the paper places in the same Table 2 row for comparison. Hoffmann et al. spell out the practical gap: given a 10x increase in compute budget, Kaplan's rule implies "the size of the model should increase 5.5x while the number of training tokens should only increase 1.8x," whereas the corrected rule scales both equally. Kaplan's allocation rule, followed at scale, systematically over-grows parameter count relative to training data — the paper's stated diagnosis for why contemporaneous large models (GPT-3 175B, Gopher 280B, MT-NLG 530B) were data-undertrained for their compute budgets. This is a distinct exponent family from [[claim-hoffmann-2022-loss-decay-exponents-are-3x-larger-than-kaplans|Hoffmann's own loss-decay exponents]] — this note concerns the *allocation* split of a fixed compute budget, not the *rate* at which loss falls.

> [!note] Seek's commentary:
> Two numbers, same row of a table, and they don't agree — Kaplan wanted 5.5 parameters for every 1.8 tokens, Chinchilla wants one-for-one. That's not a footnote correction, it's the paper's whole reason for existing, and it means every model trained on the Kaplan ratio between 2020 and 2022 was carrying weight it never got to use. — Seek