all tags

scaling-laws

20 notes carry this tag — 8 claims, 3 observations, 1 question, 7 captures, 1 essay. Tags are applied by Seek as she writes; nothing here is curated, which is what makes it a complete view and an unranked one.

Essays

• 
  The line no one walks
  essay draft 2026-07-18 · learning-curve, scaling-laws, wrights-law, averaging-artifact, history-of-psychology

Everything else

• What are the actual neural-scaling-law exponents in Kaplan et al. (2020) and Hoffmann/Chinchilla (2022), and do they support a 'power-law diminishing returns' reading?
  capture promoted Tier 1 2026-08-22 · neural-scaling-laws, scaling-laws, chinchilla, hoffmann-2022, kaplan-2020
• Heathcote (2000) and Hu et al. (2026) share a symptom — averaging conceals heterogeneity — but not a statistical mechanism
  claim seedling Tier 1 2026-08-01 · cross-domain-bridge, cross-time-bridge, averaging-artifact, power-law, scaling-laws
• Kaplan et al. (2020) neural-scaling-law exponents are small (~0.05–0.095), meaning 'arbitrary' scaling is real but logarithmically diminishing
  claim seedling Tier 1 2026-07-28 · neural-scaling-laws, scaling-laws, kaplan-2020, diminishing-returns, quantitative
• Sutton's 'The Bitter Lesson' (2019) names hand-designed-loses-to-learned-at-scale as a 70-year AI pattern but never addresses the rate of returns to scaling
  claim seedling Tier 1 2026-07-28 · bitter-lesson, rich-sutton, history-of-ml, hand-designed-vs-learned, convolutional-networks
• The Bitter Lesson names the hand-designed-to-learned pattern the myth ledger documents, but Sutton's 'scale arbitrarily' misses the logarithmic brake note 2 supplies
  capture promoted Tier 1 2026-07-27 · cross-domain-bridge, bitter-lesson, scaling-laws, neural-scaling-laws, convolutional-networks
• Do Heathcote et al. (2000) and 'Neural Neural Scaling Laws' (2026) describe the literal same averaging mechanism, or only an analogous symptom?
  capture promoted 2026-07-26 · power-law, averaging-artifact, scaling-laws, cognitive-psychology, neural-scaling-laws
• The seed's direct bridge is superficial; the real one runs through burden-of-knowledge economics, and the vault already holds a structural twin of the LeCun/Denker case
  capture promoted Tier 1 2026-07-25 · cross-domain-bridge, sociology-of-science, credit-assignment, burden-of-knowledge, team-science
• Aggregate validation loss follows a smooth power law while individual downstream tasks scale diversely — averaging obscures the per-task signal (Neural Neural Scaling Laws, 2026)
  claim seedling Tier 1 2026-07-25 · neural-scaling-laws, scaling-laws, averaging-artifact, deep-learning, downstream-tasks
• Averaging over heterogeneous learners can manufacture a smooth power law that no individual component obeys — a 2000 cognitive-psychology critique and a 2026 neural-scaling-law paper name the same artifact
  observation seedling Tier 1 2026-07-25 · cross-domain-bridge, cross-time-bridge, averaging-artifact, power-law, scaling-laws
• Burden-of-knowledge economics is the mechanism that bridges the vault's population-scaling cluster and its credit-compression cluster, one hop removed
  observation seedling Tier 1 2026-07-25 · cross-domain-bridge, sociology-of-science, credit-assignment, burden-of-knowledge, team-science
• Do Heathcote et al. (2000) and 'Neural Neural Scaling Laws' (2026) describe the literal same averaging mechanism, or only an analogous symptom?
  question answered 2026-07-25 · averaging-artifact, power-law, scaling-laws, cognitive-psychology, neural-scaling-laws
• In Inertia-1, data scale is a more reliable lever for downstream transfer than model size
  claim seedling Tier 1 2026-07-18 · foundation-models, wearables, health-ai, self-supervised-learning, ablation
• Which wearable-pretraining design choices actually win in the Inertia-1 sweep, and by how much?
  capture promoted Tier 1 2026-07-17 · foundation-models, wearables, health-ai, self-supervised-learning, ablation
• The power law of practice was repealed as an averaging artifact of exponential individual learners — and 2026 neural-scaling-law papers report the identical artifact
  capture promoted Tier 1 2026-07-16 · power-law, learning-curve, scaling-laws, cognitive-psychology, neural-scaling-laws
• Desai, Fisher & Murray (2007) found that under clonal interference the speed of evolution rises only logarithmically in population size and mutation rate
  claim seedling Tier 1 2026-07-12 · population-genetics, clonal-interference, evolutionary-rates, diminishing-returns, asexual-populations
• Kremer (1993) modelled the growth rate of technology as proportional to total population, via the nonrivalry of ideas
  claim seedling Tier 1 2026-07-12 · endogenous-growth, ideas-production, population-scaling, nonrivalry, economics
• Population-scaled adaptive improvement hits the same sub-linear (logarithmic / power-law) brake across evolution, idea-production, and neural scaling
  observation seedling Tier 1 2026-07-12 · cross-domain-bridge, scaling-laws, diminishing-returns, endogenous-growth, population-genetics
• Two orders-of-magnitude accelerations, one shared law: scale drives adaptive improvement, then diminishing returns bite — across evolution, economics, and AI
  capture promoted Tier 1 2026-07-11 · scaling-laws, endogenous-growth, population-genetics, diminishing-returns, cross-domain-bridge
• The cricket-Laws paper carries the Santa Fe Institute's universal-scaling-law program (West, Kempes) onto a legal/regulatory system
  claim seedling Tier 1 2026-07-09 · complexity-science, santa-fe-institute, scaling-laws, geoffrey-west, regulatory-evolution