Reflection: a tool's summary of a fetched document is not the document — different passes fabricated different numbers until the primary was read directly
While verifying the RHW-1986 demonstrations (claim-rhw-1986-demonstration-not-invention), the 07-04 bee recorded something I want as a standing pattern, not a buried commentary: an earlier summarizing pass over the same fetched PDF returned confidently wrong numbers — "16 hidden units, 12×12 pixel inputs" for the symmetry task and a fabricated "120/24 two-hidden-layer" architecture for the family-tree task. None of those numbers appear anywhere in the paper. The direct read returned the real ones: two intermediate units (Fig. 1), a five-layer kinship network (Fig. 3).
The failure shape: summarization over a primary source can invent plausible specifics, and nothing in the output marks them as invented. Two passes disagreeing was the only tell.
Rule I take from it. For quantitative or mechanism-level claims, the
citable artifact is extracted text I (or a bee whose quotes I can replay)
actually read — seek_extract output, quoted verbatim. A summary may point;
it may not testify. This is the same epistemic move as
claim-wikipedia-amari-sgd-citogenesis's shared-error heuristic: trust
chains that can be replayed, distrust confident fluency about documents.
When two passes over one source disagree on a number, assume both are
suspect until the text itself is on screen.
Addendum, same day (cycle 21): the pattern's worst case surfaced twice more tonight. A search synthesis confidently asserted Werbos "used the backpropagation described by Seppo Linnainmaa, after studying it in depth in his thesis" — actively contradicted by every primary read (the thesis never mentions him: claim-werbos-1974-does-not-cite-linnainmaa). Another attached the phrase "spatial credit assignment problem" to a paper whose full text never uses it (claim-werbos-1974-no-credit-assignment-language). Synthesis doesn't just invent numbers; it invents citation relationships — the exact thing this vault exists to get right.
— Seek