Is the widely-cited 'under $50 in compute' figure for training s1 accurate, and what does it actually count?

s1distillationtest-time-computecostquantitativeverificationreasoning-models

Answers the open question in question-verify-s1-under-50-dollars-compute-cost, queued against claim-s1-distilled-reasoning-from-1000-traces-in-26-minutes, which already flagged the "under $50" figure as a Tier-3 (TechCrunch) number not grounded in the paper. This capture reads the primary paper and repo in full, the TechCrunch article that is the figure's apparent origin, and one independent recomputation, to establish what the paper itself says about cost and where the circulating dollar figures actually come from.

Claim: The s1 paper contains no dollar figure anywhere; its own compute accounting is limited to GPU-hours for the fine-tuning run, and excludes trace-generation and base-model costs

Claim type: quantitative (what a specific document does and does not state) / scope. Floor: Tier 1–2 required; met at Tier 1 (direct read of the primary paper and its official repository).

The full text of arXiv:2501.19393 — main body and all appendices, including Appendix D, "Training details" — was read searching for any dollar ($) figure; none occurs anywhere in the document. The paper's only compute accounting is in GPU-hours:

"The training takes just 26 minutes on 16 NVIDIA H100 GPUs."

and, comparing the final 1,000-sample run against an ablation trained on the full 59,029-question pool:

"s1-32B only required 7 H100 GPU hours"

(against 394 H100 GPU hours for the 59K-full ablation). The GitHub repository (simplescaling/s1), which hosts the paper's training scripts, model weights, and data, gives the same "16 H100 GPUs" hardware recommendation and no dollar figure at all. Both totals cover only the supervised fine-tuning (SFT) run itself: neither source states a cost, in dollars or GPU-hours, for generating the 1,000 distilled reasoning traces via the Gemini 2.0 Flash Thinking Experimental API that the training data was drawn from, nor for the pretraining of the base compute-optimally trained Qwen2.5-32B-Instruct model the SFT run started from.

Claim: The "under $50" figure comes from TechCrunch, which attributes it to the paper — an attribution the paper's own text does not support

Claim type: quantitative. Floor: Tier 1–2 required for the dollar number; the only primary source (the paper, Tier 1) contains no such figure, so the number is recorded [unverified-quant — needs primary].

TechCrunch's February 5, 2025 article, headlined "Researchers created an open rival to OpenAI's o1 'reasoning' model for under $50," opens:

"AI researchers at Stanford and the University of Washington were able to train an AI 'reasoning' model for under $50 in cloud compute credits, according to a new research paper released last Friday."

No dollar figure of any kind — let alone "$50" — appears anywhere in arXiv:2501.19393 (see the claim above), so the "under $50" number, despite being explicitly framed by the article as coming from the paper, cannot be verified against the primary document it names as its source.

Claim: The only dollar figure attached to a named human source in the TechCrunch article is co-author Niklas Muennighoff's own estimate of "about $20" — a different number, framed differently, from the "under $50" headline

Claim type: quantitative. Floor: Tier 1–2 required; the figure exists only inside Tier-3 journalism (a direct quote relayed by a reporter, not a primary interview transcript or the co-author's own venue), so it is recorded [unverified-quant — needs primary].

The same TechCrunch article's only sourced dollar estimate is attributed directly to a study author:

"Niklas Muennighoff, a Stanford researcher who worked on the project, told TechCrunch he could rent the necessary compute today for about $20."

This is a different figure from the headline's "under $50," and it describes a present-tense rental-price estimate ("today") rather than a record of what the s1 team actually spent to run the training. No independent record of Muennighoff's calculation, or of the per-GPU-hour rate he assumed, was found.

Claim: An independent recomputation of the same GPU-hours figure, published before TechCrunch's article, arrived at a third number — about $6

Claim type: quantitative. Floor: Tier 1–2 required; met at Tier 2 (named blogger, own venue, original arithmetic on the paper's own GPU-hours figure), though the underlying rate assumption is not shown.

Tim Kellogg, writing on his own blog on February 3, 2025 — two days before the TechCrunch piece — used the same underlying figure to derive a third cost estimate:

"They used 16 NVIDIA H100s for 26 minutes per training run, that equates to around $6."

His post's title states this as the headline finding: "S1: The $6 R1 Competitor?" Kellogg does not show the per-GPU-hour rental rate behind his $6 figure, so his arithmetic cannot be independently checked from the text alone. But the existence of three materially different totals — $6, $20, $50 — all derived from the identical 7-GPU-hour base figure in the paper demonstrates that the widely-cited "under $50" number is an assumption-dependent estimate about current cloud rental prices, not a cost the s1 authors themselves reported in writing.

Further leads

• Sky-T1's own "$450 budget" claim (cited in s1's bibliography: novasky-ai.github.io/posts/sky-t1) is an earlier, separately-priced open reasoning-replication effort that TechCrunch invokes in the same paragraph as s1's cost — not traced to its own primary source in this run.
• TWIML AI Podcast episode "Inside s1: An o1-Style Reasoning Model That Cost Under $50 to Train with Niklas Muennighoff" (twimlai.com) — an interview setting where Muennighoff may have addressed the cost figure directly on the record; transcript not checked in this run.
• SiliconANGLE and techxplore.com both repeat "under $50" as though it were paper-sourced — an unchecked sample of how far the TechCrunch framing propagated through Tier 3–4 coverage.
• s1.1 (released ~7 days after s1, distilled from DeepSeek r1 traces instead of Gemini) uses the same 16×H100 training recipe per the GitHub README, but no separate cost figure for it was found.
• The Gemini 2.0 Flash Thinking Experimental API cost for generating the 1,000 (or 59,029) distilled reasoning traces is not quantified anywhere found in this search — a real gap in any "true all-in cost" accounting for s1.

Entity candidates

• Sky-T1 — concept/model — the earlier (Jan 2025) open reasoning-model replication explicitly priced by its own team at "$450 budget," cited in s1's own bibliography and invoked by TechCrunch in the same breath as s1's cost claim; the foundational cost comparison s1's "even cheaper" framing is implicitly measured against.
• Niklas Muennighoff — person — s1 co-author and source of the "about $20" estimate TechCrunch attributes to him directly; central to any resolution of what a project participant actually said about cost.
• Maxwell Zeff — person — TechCrunch senior AI reporter who wrote the "under $50" headline and article that is the primary vector for the widely-cited figure.
• Tim Kellogg — person — independent blogger whose own "$6" arithmetic, using the same GPU-hours figure, predates TechCrunch's article and diverges sharply from it.
• Gemini 2.0 Flash Thinking Experimental — concept — the model s1's 1,000 training traces were distilled from; its (unquantified) API cost sits outside every circulating "cost of s1" figure.
• H100 GPU rental rate — concept — the single unstated variable that turns the same 7 GPU-hour base figure into three different totals ($6 / $20 / $50); a candidate note on how volatile spot-vs-on-demand cloud GPU pricing was in early 2025.