Lourie, Hu & Cho (2025) build an explicit six-category taxonomy of downstream scaling behaviors and find clean linear scaling in only 39% of tasks tested

downstream-scalingtask-taxonomyneural-scaling-lawsscaling-lawsaveraging-artifact

Lourie, Hu & Cho, "Scaling Laws Are Unreliable for Downstream Tasks: A Reality Check" (arXiv:2507.00885; Findings of EMNLP 2025), re-examined the 46 downstream tasks studied by Gadre et al. (2025): "we re-examine scaling behavior on the 46 tasks tested by Gadre et al. (2025), classifying them into six categories visualized in Figure 2. We find that linear scaling actually occurs in a minority of cases in their setting: 39% of the time." Figure 2 is captioned "A taxonomy of different scaling behaviors," distinguishing predictable (linear) scaling from inverse, nonmonotonic, trendless, noisy, and breakthrough (sigmoid-jump) categories. Of the 46 tasks, only 18 (39%) showed the smooth, predictable improvement downstream scaling laws assume.

Two of this paper's three authors, Michael Y. Hu and Kyunghyun Cho, are also co-authors of Hu et al.'s 2026 "Neural Neural Scaling Laws", which cites this paper directly for the claim that "no single parametric family can capture the full spectrum of scaling behaviors [Lourie et al., 2025]" — treating this taxonomy as settled background rather than extending it. Taken with McKenzie et al. (2023)'s causal categories for inverse scaling, published research had already characterized which task types diverge and often why before Hu et al. 2026 named the phenomenon in a single background sentence.

The taxonomy is also setup-sensitive rather than purely task-intrinsic: the paper's own §5 reports that CommonsenseQA shows nonmonotonic scaling under one experimental setup and a clean linear fit under another — the same task, two categories, depending on measurement choice.