Wei, Kim, Tay & Le (2023) find most inverse-scaling tasks reverse into U-shaped scaling at larger model sizes — the divergence label is scale-dependent, not fixed to the task

inverse-scalingu-shaped-scalingtask-taxonomyneural-scaling-lawsinverse-scaling-prizedistractor-task

Wei, Kim, Tay & Le, "Inverse scaling can become U-shaped" (arXiv:2211.02011; EMNLP 2023), re-ran McKenzie et al.'s 11 inverse-scaling tasks (the Inverse Scaling Prize set) on PaLM models up to 540B parameters — roughly five times the training compute of the models evaluated in the original prize. "With this increased range of model sizes and training compute, only four out of the eleven tasks remain inverse scaling. Six out of the eleven tasks exhibit 'U-shaped scaling,'" meaning performance first drops with scale and then recovers at the largest sizes tested, rather than continuing to degrade. The authors propose a mechanism for the reversal, "which we attribute to the presence of distractor tasks that only sufficiently large models can avoid" — mid-sized models get good enough at a tempting-but-wrong sub-task to actively hurt performance, while only the largest models learn to bypass the distractor and perform the intended task.

This result complicates any static task-taxonomy of scale-driven divergence: a task's category (plateau, degrade, U-shaped, monotonic improvement) is not fixed to the task alone but depends on where in the scale range it is measured. A taxonomy of which task types diverge under scale therefore needs a scale-position axis, not only a task-identity axis — a caveat Hu et al. (2026) does not address when it cites "inverse scaling" as a fixed per-task behavior.