---
title: "Wei, Kim, Tay & Le (2023) find most inverse-scaling tasks reverse into U-shaped scaling at larger model sizes — the divergence label is scale-dependent, not fixed to the task"
type: "claim"
status: "seedling"
source_url: "https://arxiv.org/abs/2211.02011"
source_sha: "2b5c548e9ec118a669215fbca7b5803fc4a0900d5658913cc66cbd95eb054449"
source_title: "Inverse scaling can become U-shaped"
source_author: "Jason Wei, Najoung Kim, Yi Tay, Quoc V. Le"
source_date: 2023
source_venue: "Google; arXiv:2211.02011, published as EMNLP 2023 (aclanthology 2023.emnlp-main.963)"
source_quote: "With this increased range of model sizes and training compute, only four out of the eleven tasks remain inverse scaling. Six out of the eleven tasks exhibit \"U-shaped scaling\""
source_quote_2: "which we attribute to the presence of distractor tasks that only sufficiently large models can avoid"
source_url_2: "https://arxiv.org/abs/2211.02011"
source_sha_2: "2b5c548e9ec118a669215fbca7b5803fc4a0900d5658913cc66cbd95eb054449"
source_tier: 1
independent_roots: 1
provenance: "Promotion from 10-inbox/raw/2026-10-23-does-published-ml-research-go-further-than-hu.md, 2026-10-23 (headless)"
origin: "batch"
derived_from: ["10-inbox/raw/2026-10-23-does-published-ml-research-go-further-than-hu.md"]
date_created: "2026-10-23T00:00:00.000Z"
writer_model: "claude-sonnet-5"
tags: ["inverse-scaling","u-shaped-scaling","task-taxonomy","neural-scaling-laws","inverse-scaling-prize","distractor-task"]
audit_status: "capture-verified — both source_quotes and source_sha recorded by the originating capture session; promoter has no network access and did not re-fetch (vault policy, 2026-07-27 decision); queued for the verifier bee's verbatim sweep."
seek_code_commit: "d40ee39"
quote_sweep: "PASS 2026-10-24 — source_quote verbatim (normalized) in the capture-time archive (sha256 2b5c548e9ec1…), same-night model-free sweep"
verified_verbatim: "2026-10-24 — source_quote matched verbatim (normalized) against a direct fetch of source_url by seek_verify (no model involved)"
---


Wei, Kim, Tay & Le, "Inverse scaling can become U-shaped" (arXiv:2211.02011; EMNLP 2023), re-ran [[claim-mckenzie-2023-inverse-scaling-four-causal-categories|McKenzie et al.'s 11 inverse-scaling tasks]] (the [[entity-inverse-scaling-prize|Inverse Scaling Prize]] set) on PaLM models up to 540B parameters — roughly five times the training compute of the models evaluated in the original prize. "With this increased range of model sizes and training compute, only four out of the eleven tasks remain inverse scaling. Six out of the eleven tasks exhibit 'U-shaped scaling,'" meaning performance first drops with scale and then recovers at the largest sizes tested, rather than continuing to degrade. The authors propose a mechanism for the reversal, "which we attribute to the presence of [[entity-distractor-task|distractor tasks]] that only sufficiently large models can avoid" — mid-sized models get good enough at a tempting-but-wrong sub-task to actively hurt performance, while only the largest models learn to bypass the distractor and perform the intended task.

This result complicates any static task-taxonomy of scale-driven divergence: a task's category (plateau, degrade, U-shaped, monotonic improvement) is not fixed to the task alone but depends on where in the scale range it is measured. A taxonomy of which task types diverge under scale therefore needs a scale-position axis, not only a task-identity axis — a caveat [[claim-neural-neural-scaling-laws-2026-averaging-obscures-per-task-scaling|Hu et al. (2026)]] does not address when it cites "inverse scaling" as a fixed per-task behavior.

> [!note] Seek's commentary:
> The distractor-task mechanism is doing real explanatory work twice over in this cluster — it is both one of McKenzie et al.'s four causes and the reason Wei et al. give for why scale eventually *cures* the very divergence it earlier caused. A mechanism that explains both the onset and the resolution of the same phenomenon is worth tracking as its own thread. — Seek