Aggregate validation loss follows a smooth power law while individual downstream tasks scale diversely — averaging obscures the per-task signal (Neural Neural Scaling Laws, 2026)

"Neural Neural Scaling Laws" (Hu, Pan, Jhaveri, Lourie & Cho, arXiv:2601.19831, 2026) argues that the smooth power-law scaling curves used to forecast language-model performance are aggregates hiding structure. "Aggregate metrics like validation loss can follow smooth power-law curves," but "individual downstream tasks exhibit diverse scaling behaviors: some improve monotonically, others plateau, and some even degrade with scale." Predicting downstream performance from validation loss therefore "suffers from two limitations: averaging token-level losses obscures signal, and no simple parametric family can capture the full spectrum of scaling behaviors." Their proposed fix, NeuNeu, drops the assumed functional form entirely and treats scaling-law prediction as neural time-series extrapolation over per-task accuracy trajectories, reporting a 44% reduction in prediction error versus logistic scaling laws (1.99% vs. 3.56% mean absolute error across 66 downstream tasks).

The load-bearing move is disaggregation: the smooth power law is a property of the average, and the average is a composite that no single downstream task obeys. That places the finding in tension with the vault's sub-linear-brake bridge, whose AI leg treats "neural scaling laws are power laws" as an established shared law — this paper says that aggregate power law may itself be an averaging artifact concealing heterogeneous per-task behavior, exactly the softness that note already flagged in its own AI leg. Per-task divergence (monotone / plateau / degrade) also rhymes with the vault's forgetting-regime work, where behavior splits by task geometry rather than following one universal curve.

The identical shape of critique — averaging over heterogeneous units manufactures a smooth law no unit obeys — was raised twenty-six years earlier in cognitive psychology by Heathcote et al. (2000). Whether the two are the same statistical process or only an analogous symptom is an open, load-bearing question: question-verify-averaging-artifact-same-mechanism-across-practice-curves-and-scaling-laws. The cross-field recurrence is drawn out in observation-averaging-heterogeneous-learners-manufactures-a-power-law-no-individual-obeys.