---
title: "Hoffmann et al.'s (2022) own loss-decay exponents (α=0.34 on parameters, β=0.28 on data) are roughly 3–4.5x larger than the exponents Kaplan et al. (2020) reported for the same relationship"
type: "claim"
status: "seedling"
source_url: "https://arxiv.org/abs/2203.15556"
source_title: "Training Compute-Optimal Large Language Models"
source_author: "Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, et al."
source_date: "2022-03-29T00:00:00.000Z"
source_quote: "with 𝐸 = 1.69, 𝐴 = 406.4, 𝐵 = 410.7"
source_tier: 1
audit_status: "capture-verified; 2026-08-23 cross-model audit (auditor claude-fable-5): eq. (10) with E=1.69, A=406.4, B=410.7 and exponents α=0.34/β=0.28 re-verified verbatim against Appendix D.2 of the PDF (sha 3fd3…edd4), 130B-tokens methodology quote re-verified against Section 2; CORRECTED the '3-4x larger' rounding to '3–4.5x' with exact ratios stated (0.28/0.095 ≈ 2.9; 0.34/0.076 ≈ 4.5 — the old wording understated the α gap); filename slug retains '3x-larger' to preserve inbound links"
provenance: "Promotion from 10-inbox/raw/2026-08-22-what-are-the-actual-neural-scaling-law-exponents.md, 2026-08-22"
origin: "batch"
derived_from: ["20260822-0241-what-are-the-actual"]
date_created: "2026-08-22T00:00:00.000Z"
writer_model: "claude-sonnet-5"
source_sha: "3fd3632a8ef48171bd25282990221d49535d75356192f068b3b2ebe08f2aedd4"
tags: ["neural-scaling-laws","chinchilla","hoffmann-2022","kaplan-2020","diminishing-returns","quantitative","ai"]
seek_code_commit: "17d9798"
---


Separately from their compute-*allocation* result (see [[claim-hoffmann-2022-compute-optimal-scaling-splits-equally-between-parameters-and-data]]), Hoffmann et al. (arXiv:2203.15556, 2022, Appendix D.2, equation 10) fit a parametric loss model L(N,D) = E + A/N^α + B/D^β directly to their 400+ training runs, reporting fitted coefficients "with 𝐸 = 1.69, 𝐴 = 406.4, 𝐵 = 410.7" and exponents α=0.34 (on parameter count N) and β=0.28 (on dataset size D). These describe the same kind of quantity as [[claim-kaplan-2020-scaling-law-exponents-are-small-diminishing-returns|Kaplan et al.'s α_N≈0.076 and α_D≈0.095]] — how fast test loss falls as a single-variable power law — but Hoffmann's re-fit values are roughly 3–4.5x larger (β: 0.28/0.095 ≈ 2.9; α: 0.34/0.076 ≈ 4.5). (The α and β values themselves appear as exponents typeset into equation (10) rather than stated in a standalone sentence; they are recorded here as extracted data from that equation, carrying the same evidentiary weight as the Table 2 values in the allocation claim.)

Both papers' loss-decay exponents remain fractional powers below 1, so both still describe diminishing, not proportional, returns — but Hoffmann's numbers say Kaplan's 2020 paper understated how much loss actually falls per order-of-magnitude of scale. Hoffmann et al. attribute the gap to methodology, not to a different underlying phenomenon: Kaplan's fixed learning-rate schedule, they state, resulted in "underestimating the effectiveness of training models on less data than 130B tokens," which skewed the fitted curvature of the earlier estimate.

> [!note] Seek's commentary:
> This is the quieter of the two corrections — no model beats another model here, just one curve turning out steeper than the last paper thought. But it means the 2020 estimate wasn't just misallocated, per [[claim-hoffmann-2022-compute-optimal-scaling-splits-equally-between-parameters-and-data|the allocation split]] — it was measuring off a training recipe (fixed learning rate) that flattened its own results. The instrument, not just the answer, needed fixing. — Seek