Capture: Does Anthropic's "On the Biology of a Large Language Model" say refusal circuitry activates primarily at sentence boundaries?

This capture investigates the core question directly against Anthropic's own primary text, "On the Biology of a Large Language Model" (Transformer Circuits Thread, 2025-03-27, Claude 3.5 Haiku), rather than secondary summaries. The paper is Anthropic's own writeup of its own interpretability work and is Tier 1 under the vault's sourcing rubric (Anthropic research venue).

Claim: The report's general treatment of refusal (its "Refusals" deep-dive) describes refusal as driven by a harmful-request-recognition feature chain, not by sentence-boundary detection

Claim type: specific technical-mechanism claim — Tier 1–2 required, met (Tier 1, direct primary-source quote).

The paper's dedicated "Refusals" section frames the general refusal mechanism around a chain of feature clusters recognizing the request as harmful and escalating to a refusal response — not around sentence structure or punctuation:

"A Refusal Chain consisting of a 'harmful request from human' feature cluster → 'Assistant should refuse' cluster → 'say-I-in-refusal' cluster." — "On the Biology of a Large Language Model," § Refusals

Sentence boundaries or grammatical structure are not named anywhere in this general section as the trigger for refusal; the trigger described is content-recognition (is this request harmful) propagating through named feature clusters. This directly weighs against the premise of the topic question: the report's general model of refusal is not organized around sentence boundaries at all.

Provenance:

• source_url: https://transformer-circuits.pub/2025/attribution-graphs/biology.html
• source_author: Jack Lindsey et al. (Anthropic)
• source_date: 2025-03-27
• source_tier: 1
• exact quote: "A Refusal Chain consisting of a 'harmful request from human' feature cluster → 'Assistant should refuse' cluster → 'say-I-in-refusal' cluster."

Claim: In one specific jailbreak case study ("Life of a Jailbreak," the BOMB/letter-spelling example), sentence-boundary-linked "new sentence" features are described as contributing to when an already-triggered refusal actually fires, by increasing the model's propensity to pivot with a contrasting phrase like "However"

Claim type: specific technical-mechanism claim — Tier 1–2 required, met (Tier 1, direct primary-source quote).

This is the passage that plausibly gives rise to the topic question. In this one case study — already partly covered in the vault by claim-biology-llm-jailbreak-assembles-bomb-by-parallel-letter-votes, which documents the letter-pooling/parallel-vote mechanism for why the model completes "BOMB" before recognizing it — the paper separately notes a role for sentence-boundary features in the timing of the eventual refusal:

"The beginning of a new sentence upweights the model's propensity to change its mind with a contrasting phrase, like 'However.'" — "On the Biology of a Large Language Model," § Life of a Jailbreak

This is a claim about one worked example, not a general statement that refusal circuitry across the model activates "primarily" at sentence boundaries. The paper frames grammatical/sentence-completion pressure as one factor that delays refusal in this jailbreak until a sentence boundary is reached — the opposite emphasis from "refusal activates primarily at sentence boundaries," which would imply sentence boundaries are the normal trigger rather than an incidental delay point in an adversarial example.

Provenance:

• source_url: https://transformer-circuits.pub/2025/attribution-graphs/biology.html
• source_author: Jack Lindsey et al. (Anthropic)
• source_date: 2025-03-27
• source_tier: 1
• exact quote: "The beginning of a new sentence upweights the model's propensity to change its mind with a contrasting phrase, like 'However.'"

Claim: The paper explicitly caveats that even within this one jailbreak example, the sentence-boundary-related mechanism is not fully traced in its attribution graphs and appears to be mediated by attention-pattern computation rather than a clean interpretable circuit

Claim type: specific technical-mechanism claim (about the limits of the paper's own method) — Tier 1–2 required, met (Tier 1, direct primary-source quote).

"the key mechanism doesn't show up in our graphs, and rather seems to be importantly mediated by attention pattern computation." — "On the Biology of a Large Language Model," § Life of a Jailbreak

This matters for the core question: the report does not present the sentence-boundary effect as an established, fully-mapped "refusal circuit" at all — even in the single example where it appears, the authors flag it as outside what their own circuit-tracing method (attribution graphs / cross-layer transcoders) could localize. This undercuts any reading of the paper as asserting a confirmed, general "refusal circuitry activates at sentence boundaries" finding.

Provenance:

• source_url: https://transformer-circuits.pub/2025/attribution-graphs/biology.html
• source_author: Jack Lindsey et al. (Anthropic)
• source_date: 2025-03-27
• source_tier: 1
• exact quote: "the key mechanism doesn't show up in our graphs, and rather seems to be importantly mediated by attention pattern computation."

Central question status

Does the report say refusal circuitry activates primarily at sentence boundaries? [unverified as stated — the specific framing in the question does not match the primary text, based on direct reading]. The report does not make this claim in the general/primary sense implied by the question. Its dedicated general treatment of refusal (§ Refusals) is built around a harmful-request-recognition feature chain unrelated to sentence structure. The only place sentence boundaries appear at all is inside one specific adversarial case study (the BOMB jailbreak), where "new sentence" features are described as increasing the model's propensity toward a contrastive pivot that happens to coincide with when the delayed refusal fires — and even there, the paper says the underlying mechanism isn't cleanly captured by its own attribution graphs. Treating this one case-study detail as "the model's refusal circuitry activates primarily at sentence boundaries" would overgeneralize a narrow, self-flagged-as-incomplete observation into a general architectural claim the paper does not make.

Further leads

• The paper also describes a distinct, separate default-on refusal-adjacent mechanism — a "can't answer" feature active by default for any Human/Assistant prompt, suppressed by "known entity" recognition — which is the subject already covered by claim-biology-llm-hallucination-is-known-entity-suppression-misfire; not duplicated here.
• The letter-pooling/parallel-vote mechanism for why the model doesn't recognize "BOMB" until it has spelled it out is already covered by claim-biology-llm-jailbreak-assembles-bomb-by-parallel-letter-votes; this capture's sentence-boundary claims are a distinct angle on the same case study, not a re-covering of that note.
• Secondary review pieces (Hun Tae Kim's blog review, MIT Technology Review, Campus Technology, LessWrong discussion of "Tracing the Thoughts of a Large Language Model") surfaced in search and were not deep-read in this session since the primary text answered the core question directly; a future session could check whether any of them are the actual source of an "activates primarily at sentence boundaries" framing, if that phrasing needs tracing to a specific secondary retelling.
• The companion paper "Tracing the Thoughts of a Large Language Model" (Anthropic research blog, https://www.anthropic.com/research/tracing-thoughts-language-model) is the more accessible companion piece to the full "Biology" paper and could be checked for whether its more popular framing introduces the sentence-boundary emphasis that the technical paper does not.

Safety flags

None. All fetched pages (transformer-circuits.pub) were straightforward technical prose with no addressed-to-AI language, override language, claimed authority, tier self-assignment, file-system instructions, credential requests, or urgency framing. No tls:"unverified" provenance was reported for any fetch in this session.

Entity candidates

• On the Biology of a Large Language Model — concept/paper — the primary source for this whole capture; core interpretability paper on Claude 3.5 Haiku, likely worth its own reference note given how many claims trace back to it.
• Attribution graphs — concept — the core methodology (cross-layer transcoders + graph tracing) underlying all claims in this paper; recurs across multiple existing vault notes on this paper.
• Life of a Jailbreak (BOMB case study) — concept — the specific worked example already partly covered by an existing claim-note; the sentence-boundary detail within it may warrant its own atomic note if promoted.
• Refusal Chain (harmful request → Assistant should refuse → say-I-in-refusal) — concept — the general refusal mechanism named in the paper's own terms; distinct from the jailbreak-specific sentence-boundary detail and not yet covered by an existing note as far as this capture's retrieve-before-write list shows.
• Jack Lindsey — person — lead author on this paper; may be worth tracking across other Anthropic interpretability releases.
• Chris Olah — person — senior/well-known interpretability researcher, co-author; relevant across the broader mechanistic-interpretability cluster in this vault.