Jason Wei

AI researcher at Google (at the time of this work) known for naming and documenting large discontinuous jumps in capability as models scale, in "Emergent Abilities of Large Language Models" (TMLR 2022) — one of the most-cited framings of scale-driven capability change in the field.

Matters to this vault for a separate 2023 finding: with Najoung Kim, Yi Tay and Quoc V. Le, he re-ran the Inverse Scaling Prize's 11 tasks on models up to 540B parameters and found that most of them reverse from degrading-with-scale into U-shaped scaling — recovering at the largest sizes tested — attributing the reversal to the distractor-task mechanism. Emergent abilities and inverse/U-shaped scaling are, in his own work, companion phenomena: both describe scale producing a discontinuity in a learning curve rather than a smooth trend.

References

• claim-wei-2023-inverse-scaling-becomes-u-shaped-at-larger-scale