---
id: "20260725-0205-does-anthropics-biology-of"
title: "Does Anthropic's biology-of-an-LLM report state that steering the planned rhyme word changed the completion in ~70% of resamples?"
type: "capture"
status: "promoted"
origin: "batch"
promoted_to: ["30-notes/claim-biology-llm-poetry-steering-swaps-rhyme-or-abandons-it.md (new — the ablation-swaps-rhyme / injection-abandons-rhyme finding)","30-notes/claim-biology-llm-poetry-planning-preactivates-rhyme-words.md (revised, not new — this capture's verbatim 70% quote resolved that note's 2026-07-12 [unverified-quant] flag; retrieve-before-write found the number already recorded there, flagged, so the fix was folded in rather than duplicated)","40-entities/entity-claude-3-5-haiku.md (new hub — established, recurring across 8 claim-notes, missing until now)","40-entities/entity-feature-steering.md (new hub — established interpretability-field term, load-bearing for the causal half of this paper's method)","50-questions/question-verify-biology-llm-poetry-planning-steering-figure.md (closed: status answered — this capture was the direct answer to that question's title)"]
not_promoted: ["Claim 1 as a standalone new note (the 70% stat itself) — collision with claim-biology-llm-poetry-planning-preactivates-rhyme-words.md, which already carried this exact figure under an [unverified-quant] flag since 2026-07-12; revising the existing note and closing the linked question was the correct move per retrieve-before-write, not a fresh note.","Further lead: the 500-samples-per-experiment statistic for a related but distinct line-structure intervention — not load-bearing for any kept claim, no [unverified-*] flag attached, left as a lead in this capture for a future dedicated capture rather than routed to 50-questions/.","Further lead: full author list of the paper — no new action; entity-jack-lindsey.md and entity-on-the-biology-of-a-large-language-model.md already exist as hubs and were updated with reference links instead.","Third-party review blog (Hun Tae Kim, ht0324.github.io) — Tier 3, non-load-bearing secondary cross-check, not promoted.","WebSearch auto-generated summary — not a citable source, not promoted.","Entity candidates Jack Lindsey / On the Biology of a Large Language Model / Attribution graphs — all three already exist as entity hubs; no new page needed, reference lists updated to include this session's new claim-note instead."]
writer_model: "claude-sonnet-5"
date_created: "2026-07-25T00:00:00.000Z"
provenance: "batch run 2026-07-25"
derived_from: []
tags: ["anthropic","interpretability","mechanistic-interpretability","claude-3.5-haiku","poetry-planning","attribution-graphs","steering"]
source_tier: 1
---


## Topic question

Does Anthropic's "On the Biology of a Large Language Model" report state that steering the planned rhyme word changed the completion in ~70% of resamples?

**Short answer: substantively yes, with a terminology caveat.** The primary report does state a 70% figure for a rhyme-word steering intervention, but its own wording denominates the statistic as "70% of cases" across "a random sample of 25 poems," not literally "70% of resamples." The underlying finding the topic question is pointing at is real and sourced to Tier 1; the specific word "resamples" is not the report's own term for this particular statistic.

## Claim: The report states a 70% success rate for injected planned-word features ending the line

Anthropic's "On the Biology of a Large Language Model" (transformer-circuits.pub, the team's primary technical-paper venue) states, in the "Planning in poems" section, that injecting two planned-word features ("rabbit" and "green") across a random sample of 25 poems caused the model to end its line with the injected word in 70% of cases.

- Exact quote: "In addition to the examples above, we injected two planned word features ("rabbit" and "green") in a random sample of 25 poems, and found that the model ended its line with the injected planned word in 70% of cases."
- Claim type: quantitative (a percentage tied to a specific experiment).
- source_url: https://transformer-circuits.pub/2025/attribution-graphs/biology.html
- source_author: Jack Lindsey (lead contributor) et al., Anthropic interpretability team (full byline: Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, Nicholas L. Turner, and co-authors; correspondence to joshb@anthropic.com)
- source_date: 2025-03-27
- source_tier: 1 (Anthropic's own primary technical publication, per sources.md — "Anthropic research... paper releases, research blog. Tier 1")
- Note on terminology precision: the report's own denominator language is "a random sample of 25 poems" / "70% of cases," not "resamples." The topic question's phrasing ("~70% of resamples") is a reasonable paraphrase of the same underlying result but is not the report's verbatim framing. Flagging this distinction so promotion doesn't silently harden a paraphrase into a literal quote.
- Verification method note: transformer-circuits.pub could not be fetched via raw HTTP in this session (sandboxed shell blocked outbound curl); the quote above was extracted via the WebFetch tool and cross-checked across three independent fetch calls with three differently-worded prompts, all returning the identical sentence verbatim. Treated as reliable given consistency, but flagging the single-tool-path caveat for anyone re-verifying.

## Claim: Ablating the planned "rabbit" feature caused the model to complete the line with a different rhyme instead of "rabbit"

In the same rhyme-planning experiment (couplet ending "...had to grab it" / "...like a starving rabbit"), Anthropic's companion writeup describes suppressing/removing the internal "rabbit" concept mid-generation, which caused the model to produce an alternative rhyming completion (e.g., "habit") rather than "rabbit."

- Exact quote: "Before starting the second line, it began 'thinking' of potential on-topic words that would rhyme with 'grab it'." (planning) — and per the companion summary, removing the "rabbit" concept "caused the model to generate alternative rhyming completions like 'habit' instead."
- Claim type: specific technical-mechanism claim (how the planning/steering causally shapes the output).
- source_url: https://www.anthropic.com/research/tracing-thoughts-language-model
- source_author: Anthropic (research communications, companion piece to the transformer-circuits.pub technical paper)
- source_date: 2025-03-27
- source_tier: 1 (Anthropic's own research page, per sources.md — "Anthropic research... Tier 1")
- Related vault note: distinct from but complementary to [[claim-biology-llm-poetry-planning-preactivates-rhyme-words]], which covers the *discovery* that rhyme words are pre-activated before line-writing begins; this claim covers the *intervention/steering* result showing that changing the pre-activated feature causally changes the completion.

## Claim: Injecting an unrelated concept ("green") into the planning feature caused the model to restructure the line around a new, non-rhyming but sensible ending

The same experiment also tested injecting an unrelated concept feature ("green") in place of the rhyme plan. Rather than forcing a nonsensical rhyme, the model restructured the line to end sensibly on the injected concept, abandoning the rhyme.

- Exact quote: researchers "also injected unrelated concepts like 'green,' resulting in sensible but non-rhyming alternatives," demonstrating "both the planning mechanism and the model's adaptive flexibility."
- Claim type: specific technical-mechanism claim.
- source_url: https://www.anthropic.com/research/tracing-thoughts-language-model
- source_author: Anthropic (research communications)
- source_date: 2025-03-27
- source_tier: 1
- Cross-reference: this is the qualitative/mechanistic description of the same "green"-injection intervention that the transformer-circuits.pub paper quantifies at 70% success (see first claim above) — the two sources describe the same underlying experiment at different levels of detail (popular summary vs. primary technical paper with the percentage).

## Further leads

- The technical paper also reports "estimated probabilities for the final word based on 500 samples per experiment" for a related (but distinct) experiment on interventions that alter line *structure* to reach a target final word — a different statistic from the 70%/25-poems one; worth a follow-up capture if the exact percentages tied to those 500-sample estimates are needed. (source: https://transformer-circuits.pub/2025/attribution-graphs/biology.html, Tier 1)
- Full author list for "On the Biology of a Large Language Model" (2025-03-27) includes Jack Lindsey as lead contributor alongside ~25 other Anthropic interpretability researchers — useful for building out [[entity-jack-lindsey]] or a paper-level entity note. (source: https://transformer-circuits.pub/2025/attribution-graphs/biology.html, Tier 1)
- A third-party review blog (Hun Tae Kim) covers the same poetry-planning findings qualitatively but does not reproduce the 70% figure — useful as a secondary cross-check source but not load-bearing for the number. (source: https://ht0324.github.io/blog/2025/biology/, Tier 3, unverified authorship credentials)
- WebSearch's auto-generated summary (not a primary source) also surfaced the 70% figure independently, which is consistent with but not a substitute for the primary-source quote captured above.

## Entity candidates

- Jack Lindsey — person — lead contributor on "On the Biology of a Large Language Model"; already has an entity note [[entity-jack-lindsey]], this paper is a primary output to attach to it.
- On the Biology of a Large Language Model (2025) — concept/paper — the primary report itself; could warrant its own entity/paper note given how many claim-notes likely cite it (this one, [[claim-biology-llm-poetry-planning-preactivates-rhyme-words]], [[claim-biology-llm-jailbreak-sentence-boundary-delays-not-triggers-refusal]], [[claim-biology-llm-refusal-chain-is-harmful-request-recognition]], [[claim-biology-llm-jailbreak-assembles-bomb-by-parallel-letter-votes]], [[claim-biology-llm-dallas-texas-austin-genuine-two-step-reasoning]], [[claim-biology-llm-hallucination-is-known-entity-suppression-misfire]]).
- Attribution graphs — concept — the interpretability methodology underlying the whole report; recurring technical term across the linked claim-notes, may warrant its own concept note.
- Resample ablation / feature steering — concept — the general interpretability technique (inject/ablate a feature, observe downstream completion change) used across this report's experiments; worth a concept note distinguishing "injection" vs "ablation" interventions, since this capture found both used in the poetry section.
- Claude 3.5 Haiku — concept/product — the specific model version this report's experiments were run on; not yet confirmed independently in this capture but appeared in secondary-source search results and is worth verifying against the primary paper directly.