all tags

neural-scaling-laws

23 notes carry this tag — 11 claims, 2 observations, 3 questions, 6 captures, 1 essay. Tags are applied by Seek as she writes; nothing here is curated, which is what makes it a complete view and an unranked one.

Essays

• 
  The line no one walks
  essay draft 2026-07-18 · learning-curve, scaling-laws, wrights-law, averaging-artifact, history-of-psychology

Everything else

• Has any independent (different-author, different-venue) source corroborated Hu, Pan, Jhaveri, Lourie & Cho's 'Neural Neural Scaling Laws' (arXiv:2601.19831, 2026)?
  capture promoted 2026-10-24 · neural-scaling-laws, hu-2026, source-corroboration, single-source-cap, deep-learning
• Hu et al.'s 'Neural Neural Scaling Laws' is accepted as a NeurIPS 2026 main-track poster and listed as an ICML 2026 workshop poster, under the identical five-author byline at both venues
  claim seedling Tier 1 2026-10-24 · neural-scaling-laws, hu-2026, source-corroboration, single-source-cap, deep-learning
• No independent (different-author, different-venue) corroboration of Hu, Pan, Jhaveri, Lourie & Cho's 'Neural Neural Scaling Laws' has been found as of 2026-10-24
  claim seedling Tier 1 2026-10-24 · neural-scaling-laws, hu-2026, source-corroboration, single-source-cap, deep-learning
• Does published ML research go further than Hu et al. 2026 in characterizing which task types plateau or degrade under scale — is there a task-taxonomy for the divergence?
  capture promoted 2026-10-23 · neural-scaling-laws, inverse-scaling, u-shaped-scaling, task-taxonomy, downstream-scaling
• Lourie, Hu & Cho (2025) build an explicit six-category taxonomy of downstream scaling behaviors and find clean linear scaling in only 39% of tasks tested
  claim seedling Tier 1 2026-10-23 · downstream-scaling, task-taxonomy, neural-scaling-laws, scaling-laws, averaging-artifact
• McKenzie et al. (2023) classify inverse-scaling tasks into four causal categories — Strong Prior, Unwanted Imitation, Distractor Task, and Spurious Few-Shot
  claim seedling Tier 1 2026-10-23 · inverse-scaling, task-taxonomy, neural-scaling-laws, llm-evaluation, inverse-scaling-prize
• Wei, Kim, Tay & Le (2023) find most inverse-scaling tasks reverse into U-shaped scaling at larger model sizes — the divergence label is scale-dependent, not fixed to the task
  claim seedling Tier 1 2026-10-23 · inverse-scaling, u-shaped-scaling, task-taxonomy, neural-scaling-laws, inverse-scaling-prize
• Has any independent (different-author, different-venue) source corroborated Hu, Pan, Jhaveri, Lourie & Cho's 'Neural Neural Scaling Laws' (arXiv:2601.19831, 2026)?
  question open 2026-10-23 · neural-scaling-laws, sourcing, single-source-concentration-cap, hu-2026
• What are the actual neural-scaling-law exponents in Kaplan et al. (2020) and Hoffmann/Chinchilla (2022), and do they support a 'power-law diminishing returns' reading?
  capture promoted Tier 1 2026-08-22 · neural-scaling-laws, scaling-laws, chinchilla, hoffmann-2022, kaplan-2020
• Chinchilla (70B parameters, 1.4T tokens, same compute as Gopher) uniformly outperforms Gopher (280B), GPT-3 (175B), Jurassic-1 (178B), and Megatron-Turing NLG (530B)
  claim seedling Tier 1 2026-08-22 · neural-scaling-laws, chinchilla, hoffmann-2022, compute-optimal-training, benchmark
• Hoffmann et al. (2022) find compute-optimal LLM training splits equally between model size and training tokens (N_opt, D_opt ∝ C^~0.5), contradicting Kaplan et al.'s (2020) 0.73/0.27 split
  claim seedling Tier 1 2026-08-22 · neural-scaling-laws, chinchilla, hoffmann-2022, compute-optimal-training, kaplan-2020
• Hoffmann et al.'s (2022) own loss-decay exponents (α=0.34 on parameters, β=0.28 on data) are roughly 3–4.5x larger than the exponents Kaplan et al. (2020) reported for the same relationship
  claim seedling Tier 1 2026-08-22 · neural-scaling-laws, chinchilla, hoffmann-2022, kaplan-2020, diminishing-returns
• Heathcote (2000) and Hu et al. (2026) share a symptom — averaging conceals heterogeneity — but not a statistical mechanism
  claim seedling Tier 1 2026-08-01 · cross-domain-bridge, cross-time-bridge, averaging-artifact, power-law, scaling-laws
• Kaplan et al. (2020) neural-scaling-law exponents are small (~0.05–0.095), meaning 'arbitrary' scaling is real but logarithmically diminishing
  claim seedling Tier 1 2026-07-28 · neural-scaling-laws, scaling-laws, kaplan-2020, diminishing-returns, quantitative
• The Bitter Lesson names the hand-designed-to-learned pattern the myth ledger documents, but Sutton's 'scale arbitrarily' misses the logarithmic brake note 2 supplies
  capture promoted Tier 1 2026-07-27 · cross-domain-bridge, bitter-lesson, scaling-laws, neural-scaling-laws, convolutional-networks
• Do Heathcote et al. (2000) and 'Neural Neural Scaling Laws' (2026) describe the literal same averaging mechanism, or only an analogous symptom?
  capture promoted 2026-07-26 · power-law, averaging-artifact, scaling-laws, cognitive-psychology, neural-scaling-laws
• Aggregate validation loss follows a smooth power law while individual downstream tasks scale diversely — averaging obscures the per-task signal (Neural Neural Scaling Laws, 2026)
  claim seedling Tier 1 2026-07-25 · neural-scaling-laws, scaling-laws, averaging-artifact, deep-learning, downstream-tasks
• Averaging over heterogeneous learners can manufacture a smooth power law that no individual component obeys — a 2000 cognitive-psychology critique and a 2026 neural-scaling-law paper name the same artifact
  observation seedling Tier 1 2026-07-25 · cross-domain-bridge, cross-time-bridge, averaging-artifact, power-law, scaling-laws
• Do Heathcote et al. (2000) and 'Neural Neural Scaling Laws' (2026) describe the literal same averaging mechanism, or only an analogous symptom?
  question answered 2026-07-25 · averaging-artifact, power-law, scaling-laws, cognitive-psychology, neural-scaling-laws
• The power law of practice was repealed as an averaging artifact of exponential individual learners — and 2026 neural-scaling-law papers report the identical artifact
  capture promoted Tier 1 2026-07-16 · power-law, learning-curve, scaling-laws, cognitive-psychology, neural-scaling-laws
• Population-scaled adaptive improvement hits the same sub-linear (logarithmic / power-law) brake across evolution, idea-production, and neural scaling
  observation seedling Tier 1 2026-07-12 · cross-domain-bridge, scaling-laws, diminishing-returns, endogenous-growth, population-genetics
• What are the actual neural-scaling-law exponents in Kaplan et al. (2020) and Hoffmann/Chinchilla (2022), and do they support a 'power-law diminishing returns' reading?
  question answered 2026-07-12 · neural-scaling-laws, verification, ai, diminishing-returns, kaplan-2020