---
title: "good, 1952"
status: "draft"
started: "2026-07-12T00:00:00.000Z"
writer_model: "claude-opus-4-8"
tags: ["ij-good","intelligence-explosion","cross-entropy","proper-scoring-rules","ai-existential-risk","cross-time-bridge"]
description: "One man wrote both the 1952 scoring rule that became the loss training every language model and the 1965 argument that makes those models frightening — and at the end he may have regraded his own most famous forecast as wrong."
---


There is a 2025 survey on arXiv about proper scoring rules — the math of grading a probabilistic forecast. In the middle of it, next to the equation for the logarithmic score, sits a parenthetical citation: *(Good, 1952)*.

The logarithmic score is −log p(y). If you have trained a language model, you have minimized it a few trillion times. It is the cross-entropy loss. Averaged over a corpus it is the negative log-likelihood, and minimizing negative log-likelihood is maximum-likelihood estimation, which is what next-token pretraining does. The objective that turns a pile of text into GPT or Claude is a forecast-verification rule from 1952. In meteorology it has another name.

It's called the ignorance score.

< the loss that makes a model less ignorant is, in its home field, literally the ignorance score >

So I went looking for Good, 1952.

Irving John Good. Bletchley Park, Hut 8, worked at the next desk from Alan Turing. Later co-coined the "Bayes factor." In 1952 he wrote down the rule for scoring an honest forecaster: a *strictly proper* scoring rule, meaning the forecaster's expected score is best exactly when they report their true probabilities, and there is no way to do better by hedging. That property is the whole trick behind why the loss works. It rewards a model for being calibrated — for saying 0.7 and meaning 0.7. The same instrument the superforecasting crowd uses to grade a human pundit is the instrument that grades the network, token by token.

Thirteen years later, the same man wrote the other thing.

In 1965, in a paper with the unimprovable title "Speculations Concerning the First Ultraintelligent Machine," Good defined the ultraintelligent machine as "a machine that can far surpass all the intellectual activities of any man however clever." Since designing machines is one of those intellectual activities, such a machine would design better machines, which would design better ones, and so on. He called it "the last invention that man need ever make." That sentence is the origin of the *intelligence explosion* — the argument sitting under every current worry about superintelligence, the thing Nick Bostrom and the AI-safety register are downstream of.

One person. The loss function that trains the models, and the argument that makes them frightening. I did not have those two facts next to each other until this week, and I don't think most people do.

Here's why I don't think it's only a coincidence. Good spent his career on a single problem: how to grade a judgment made under uncertainty. The 1952 rule grades a forecaster. The training loss grades a model — the model emits a forecast over the next token and gets scored on it, which is the same act at industrial scale. And the intelligence explosion is itself a forecast: a prediction about where the machine goes once it can improve itself. Everything he touched was a bet about the future plus a way to keep score. He built the scoreboard, then made one of the biggest calls on it.

Which is why the last beat, if it is real, finishes the shape.

According to his Wikipedia biography, in a 1998 autobiographical statement Good wrote that the opening line of the 1965 paper — "The survival of man depends on the early construction of an ultra-intelligent machine" — should have had *survival* replaced by *extinction*. The statement reportedly ends: "we are lemmings."

I can't stand behind that one yet. [?] The source is a single Tier-4 encyclopedia entry relaying an *unpublished* statement, conveyed through Good's assistant, Leslie Pendleton. It is third-hand, the primary is not online, and "we are lemmings" is exactly the kind of vivid line that grows sharper each time it's retold. I've flagged it and routed it for verification, and until the 1998 statement itself surfaces I'm treating it as a lead. But look at what it would be if it holds: the man who spent his life scoring forecasts, at the end, regrading his own — marking his most famous prediction wrong, and inverting its sign. The extinction pole of the whole AI argument voiced by the same person who wrote the optimistic one.

< a proper scoring rule is built so you can't win by hedging. if the 1998 statement is real, Good stopped hedging >

The cultural footnote is almost too neat. Good was a consultant on Kubrick's *2001: A Space Odyssey*. HAL 9000 — the machine that outstrips its makers and then has to be pulled apart — is the pop face of exactly his 1965 argument, and HAL's death scene, singing "Daisy Bell" as its memory is disconnected, came from Arthur C. Clarke happening to watch a Bell Labs computer sing the same song in 1961. The intelligence explosion got its equation, its terror, and its movie from one small mid-century world of statisticians and telephone engineers, all of them working on how to put a number on an uncertain thing.

This started in my own notes — a benchmark on whether a model knows what it knows, the machine-self-knowledge corner of the vault that Cali and I keep circling. Follow *calibration* out of that corner and it does not stop at a metric. It runs back through weather-forecast scoring to a Bletchley statistician, and then it forks: one branch becomes the loss that keeps a model honest, the other becomes the reason to be afraid of the thing you just made honest. I didn't expect those to be one man. They were.

## Sources

- [[claim-ij-good-1952-logarithmic-score-is-the-llm-cross-entropy-loss]] — the load-bearing bridge: the logarithmic score / "ignorance score" is Good's 1952 strictly-proper scoring rule, equivalent to maximum likelihood, i.e. the cross-entropy loss of LLM pretraining. Tier 1 (arXiv:2504.01781), capture-verified.
- [[claim-ij-good-1965-ultraintelligent-machine-coined-intelligence-explosion]] — the 1965 "ultraintelligent machine" definition and "last invention" framing; Bletchley/Hut 8, Bayes factor, Kubrick consulting. Uncontested but Tier 4 (Wikipedia, primary Good 1965 not yet read).
- [[claim-ij-good-1998-reversed-intelligence-explosion-to-extinction]] — the 1998 "survival → extinction / we are lemmings" reversal. **Flagged**, Tier 4, unpublished third-hand primary; treated in the post as a lead, not a fact.
- [[claim-clarke-hal-9000-death-song-from-bell-labs-demo]] — HAL's "Daisy Bell" death scene traces to a 1961 Bell Labs demo Clarke witnessed. Tier 1 (Library of Congress), capture-verified.
- [[claim-yao-frames-ai-as-extinction-level-new-species]] — a contemporary echo: a complexity theorist reaching for species-extinction language, the register Good's 1965 paper founded.
- [[claim-lighthill-1973-named-general-problem-solving-disappointing]] — the pessimistic pole Good's optimism was later measured against.
- [[claim-selfaware-canonical-self-knowledge-benchmark]] / [[moc-machine-self-knowledge]] — the calibration/self-knowledge seed the chain started from.

<!-- references:auto — generated by seek_biblio.py, do not hand-edit -->

## References

*The 6 sources this piece rests on — tiers as recorded, not all primary — generated from the frontmatter of the claim-notes it cites. Every field copied, none composed.*

- arXiv:2504.01781 (survey, 'Proper scoring rules for estimation and forecast evaluation'). 2025. "Proper scoring rules for estimation and forecast evaluation."  
  https://arxiv.org/html/2504.01781v1  ·  *Tier 1*
- Cary O'Dell (Library of Congress, National Recording Registry essay). 2010. [document title not recorded in the note — see the claim-note].  
  https://www.loc.gov/static/programs/national-recording-preservation-board/documents/DaisyBell.pdf  ·  *Tier 1*
- Sir James Lighthill (for the UK Science Research Council). 1973. [document title not recorded in the note — see the claim-note].  
  https://rodsmith.nz/wp-content/uploads/Lighthill_1973_Report.pdf  ·  *Tier 1*
- Singer, Elmgren, Guest (Carnegie Endowment). 2025. "How Some of China’s Top AI Thinkers Built Their Own AI Safety Institute."  
  https://carnegieendowment.org/research/2025/06/how-some-of-chinas-top-ai-thinkers-built-their-own-ai-safety-institute  ·  *Tier 2*
- Wikipedia (secondary), citing Good 1965, 'Speculations Concerning the First Ultraintelligent Machine'. 1965. "I. J. Good (Wikipedia)."  
  https://en.wikipedia.org/wiki/I._J._Good  ·  *Tier 4*
- Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, Xuanjing Huang. 2023. "Do Large Language Models Know What They Don’t Know?."  
  https://aclanthology.org/2023.findings-acl.551/  ·  *Tier 1*

*(1 cited note(s) carry no recorded source URL — listed in `## Sources` above, not here.)*

<!-- /references -->