--- id: "20260707-0945-does-section-551-of" title: "Capture: Does section 5.5.1 of Werbos's 1974 Harvard thesis 'Beyond Regression' explicitly name Linnainmaa as a prior source, or is it an independent derivation?" type: "capture" status: "promoted" promoted_to: ["30-notes/claim-werbos-1974-does-not-cite-linnainmaa.md (all four convergent reads; 5.5.1-renumbering hypothesis flagged not resolved; Kashyap-footnote receipt carried)","30-notes/reflection-recurring-tool-summary-is-not-the-source.md (addendum: the actively-wrong search synthesis — invented citation relationships, not just numbers)"] not_promoted: ["1994 Wiley reprint TOC check and Linnainmaa-side awareness check — open leads; the reprint is also the marathon report's best library-op candidate"] promotion_note: "Queen cycle 21, 2026-07-07. SAFELIST EXHAUSTED with this capture." origin: "batch" model: "claude-sonnet-5" date_created: "2026-07-07T00:00:00.000Z" provenance: "batch run 2026-07-07 — researched via extract_pdf (direct full-text read of the 1974 thesis PDF and of Schmidhuber's 2015 Neural Networks survey), WebSearch/WebFetch, and two background verification subagents that each read their target document in full (21,754 lines and 2,823 lines respectively) rather than sampling" derived_from: [] tags: ["werbos","linnainmaa","backpropagation","automatic-differentiation","harvard","history-of-ml","priority","schmidhuber","primary-source-verification","ordered-derivative"] source_url: "https://gwern.net/doc/ai/nn/1974-werbos.pdf" source_author: "Paul J. Werbos" source_date: "1974-08 (thesis defended); PDF confirmed accessible and text-extractable 2026-07-07" source_tier: 1 other_sources: [{"url":"https://faculty.sites.iastate.edu/tesfatsi/archive/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf","author":"Jürgen Schmidhuber","date":"2015","venue":"Neural Networks 61: 85-117","tier":1,"note":"source of the '(Werbos, 1974, Section 5.5.1)' citation that seeded this question; read in full"},{"url":"https://www.werbos.com/AD2004.pdf","author":"Paul J. Werbos","date":"2004","tier":1,"note":"Werbos's own retrospective essay bridging AD and neural-network history; read in full"},{"url":"https://ems.press/content/book-chapter-files/27379","author":"Andreas Griewank","date":"2012","tier":1,"note":"already vaulted (see 2026-07-06 capture); cited here for corroboration only"}] --- > [!info] Core question > **No — the section does not name Linnainmaa, and the available evidence supports independent derivation.** This is now backed by convergent Tier-1 primary evidence from four directions: the thesis text itself (read in full), the technical section containing the actual backprop-like derivation (read in full, zero citations to any external source for the technique), the paper that is the actual source of the "Section 5.5.1" citation (read in full, makes no citation-lineage claim), and Werbos's own 2004 retrospective essay (read in full, never mentions Linnainmaa at all). One genuine nuance remains unresolved and is flagged below rather than papered over: "Section 5.5.1" does not correspond to any section locator that exists in the copy of the 1974 thesis examined here. --- ## Claim: The gwern-hosted PDF of Werbos's 1974 thesis contains zero occurrences of "Linnainmaa" anywhere in its full text, and its own internal section-numbering scheme never produces a "5.5.1"-style locator **Claim type:** technical-mechanism / historical (a direct textual fact about a specific document's contents and structure). **Tier 1–2 required.** Achieved Tier 1 — direct primary-source read, not a secondary characterization. The 1974 thesis PDF (https://gwern.net/doc/ai/nn/1974-werbos.pdf, 453 pages) was fetched and extracted to text via `extract_pdf`, then read exhaustively — all 21,754 lines, in sequential overlapping chunks with no gaps — by a dedicated verification pass. Two independent facts were established: 1. The string "Linnainmaa," and every plausible OCR-garbled variant of it (e.g. "Linnainmma," "L1nnainmaa"), **does not appear anywhere in the document** — not in the main text of any of its six chapters, not in any footnote, not in the bibliography. 2. The thesis's own numbering scheme, confirmed from its table of contents and cross-checked against chapter bodies, uses **Roman numerals for chapters** — (I) through (VI) — **and lowercase roman numerals for subsections within a chapter** — (i), (ii), (iii)... Equations use a "chapter.number" decimal format (e.g. "(2.11)"), but this format is never applied to section headers. **No "5.5.1"-style (chapter.section.subsection) construction exists anywhere in the original document.** Chapter (V) — "General Applications of These Ideas: Practical Hazards and New Possibilities" — genuinely does run only from subsection (i) to subsection (v); there is no further decimal subdivision. **Provenance:** - source_url: https://gwern.net/doc/ai/nn/1974-werbos.pdf - source_author: Paul J. Werbos - source_date: August 1974 (Harvard University, Committee on Applied Mathematics) - source_tier: 1 - exact quote (table of contents, Chapter V): "(V) GENERAL APPLICATIONS OF THESE IDEAS: PRACTICAL HAZARDS AND NEW POSSIBILITIES ... (i) INTRODUCTION AND SUMMARY ... (ii) THE LIABILITIES OF MATHEMATICAL METHODS ... (iii) PREDICTION: A COMMON GOAL ... (iv) POSSIBILITIES FOR STATISTICS AS AN EMPIRICAL TOOL ... (v) BEYOND NAIVE EMPIRICISM" - access_note: page-count and extractability confirmed directly this session (`extract_pdf` succeeded, method: pdftotext, 453 pages); a prior 2026-06-29 capture on a related topic recorded this same URL as inaccessible in that session's environment — this run confirms the tool works and the document is text-extractable. --- ## Claim: The actual technical core of Werbos's proto-backpropagation algorithm — the "ordered derivative" chain-rule theorem and its proof — sits in Chapter (II), subsection (xii), and is presented as an original derivation with no citation to any prior published source for the technique **Claim type:** technical-mechanism (does a specific passage cite prior work or not). **Tier 1–2 required.** Achieved Tier 1 — direct primary read of the full subsection (lines 5445–5763 of the extracted text, roughly pages II-82 to II-89). Werbos opens this subsection by motivating a new formalism for partial derivatives in systems with a causal ordering, then defines the **ordered derivative**, states a theorem giving its recursive chain-rule formula, and proves the theorem himself by induction, entirely in first person ("Let us define...", "We can prove this..."). No citation marker — no footnote number, no author name — appears anywhere within this subsection attached to the definition, theorem, or proof of the technique itself. > "The traditional formalism used for dealing with partial derivatives was evolved to deal with the problems of geometry and of physical science... In the social sciences, however, one normally deals with a complex web of functional relations and variables... we need to define a new formalism for this kind of partial derivative." The one place in the entire thesis where Werbos acknowledges anything resembling a literature precursor to the ordered-derivative concept is a footnote in Chapter (III), discussing R. L. Kashyap's control-theory work — and it explicitly *distinguishes* Kashyap's concept from his own rather than crediting it as a source: > "Kashyap also mentions a notion of 'constrained derivative,' which looks like a precursor of the 'ordered derivative' of Chapter (II), but based upon notions of variational calculus; the concept, as he uses it, does not include his 'lambdas' as a set of constrained derivatives, while they correspond very clearly to ordered derivatives in our own system." **Provenance:** - source_url: https://gwern.net/doc/ai/nn/1974-werbos.pdf - source_author: Paul J. Werbos - source_tier: 1 - exact quote 1 (thesis motivation, Ch. II §xii): "The traditional formalism used for dealing with partial derivatives was evolved to deal with the problems of geometry and of physical science... we need to define a new formalism for this kind of partial derivative." - exact quote 2 (Ch. III footnote 11, on Kashyap): "Kashyap also mentions a notion of 'constrained derivative,' which looks like a precursor of the 'ordered derivative' of Chapter (II), but based upon notions of variational calculus... they correspond very clearly to ordered derivatives in our own system." - related existing notes: [[claim-werbos-backprop-from-freud-own-account]], [[backpropagation-gap]] --- ## Claim: Schmidhuber's 2015 peer-reviewed survey is the actual source of the "(Werbos, 1974, Section 5.5.1)" citation, and its own text — read directly — does not assert that Werbos's section cites or was aware of Linnainmaa **Claim type:** historical (what a specific published paper's text actually says and does not say). **Tier 3–4 acceptable for the uncontested part; escalated here because it is the load-bearing point of the whole inquiry.** Achieved Tier 1 — direct full-text read of Schmidhuber, J. (2015), "Deep Learning in Neural Networks: An Overview," *Neural Networks* 61: 85–117 (a peer-reviewed journal, not the informal "who invented backpropagation" web page used in an earlier capture on an adjacent question). The paper's actual sentence, read directly from the extracted PDF text (all 2,823 lines read in full): > "Explicit, efficient error backpropagation (BP) in arbitrary, discrete, possibly sparsely connected, NN-like networks apparently was first described in a 1970 master's thesis (Linnainmaa, 1970, 1976), albeit without reference to NNs. BP is also known as the reverse mode of automatic differentiation (Griewank, 2012), where the costs of forward activation spreading essentially equal the costs of backward derivative calculation. See early FORTRAN code (Linnainmaa, 1970) and closely related work (Ostrovskii, Volin, & Borisov, 1971). Efficient BP was soon explicitly used to minimize cost functions by adapting control parameters (weights) (Dreyfus, 1973). Compare some preliminary, NN-specific discussion (Werbos, 1974, Section 5.5.1), a method for multilayer threshold NNs (Bobrowski, 1978), and a computer program for automatically deriving and implementing BP for given differentiable systems (Speelpenning, 1980). ... To my knowledge, the first NN-specific application of efficient BP as above was described in 1981 (Werbos, 1981, 2006)." This confirms the exact locator earlier captures could only get from the informal web-page version of Schmidhuber's account (see [[20260707-0233-did-werboss-1974-harvard]]): "(Werbos, 1974, Section 5.5.1)." Read in context, the sentence structure places Linnainmaa (1970/1976) and Werbos (1974, §5.5.1) as **separate, sequential entries in a historical timeline**, connected only by the word "Compare" — a word this paper uses throughout to juxtapose related-but-distinct prior work, not to assert that one work cites or descends from another. No linking clause ("citing Linnainmaa," "building on Linnainmaa," "unaware of Linnainmaa," "independently of Linnainmaa") appears anywhere connecting the two theses in this text. The paper is explicitly silent on the citation relationship between them — it neither confirms nor denies that Werbos's section names Linnainmaa. **Provenance:** - source_url: https://faculty.sites.iastate.edu/tesfatsi/archive/tesfatsi/DeepLearningInNeuralNetworksOverview.JSchmidhuber2015.pdf - source_author: Jürgen Schmidhuber - source_date: 2015 - source_venue: Neural Networks 61: 85–117 - source_tier: 1 - exact quote: "Compare some preliminary, NN-specific discussion (Werbos, 1974, Section 5.5.1), a method for multilayer threshold NNs (Bobrowski, 1978), and a computer program for automatically deriving and implementing BP for given differentiable systems (Speelpenning, 1980)." - related existing notes: [[20260707-0233-did-werboss-1974-harvard]] (prior capture that located the same citation on Schmidhuber's informal web page, before this run located and read the underlying peer-reviewed paper directly) --- ## Flagged gap: "Section 5.5.1" does not match the internal numbering of the 1974 thesis text examined in this session **Claim type:** historical/bibliographic. Flagged rather than resolved. Schmidhuber's citation locator, "Section 5.5.1," implies a decimal chapter.section.subsection numbering scheme. The copy of the 1974 thesis examined directly in this session (the gwern-hosted PDF, confirmed to be the actual "Beyond Regression" dissertation by its title page, preface, and table of contents) uses no such scheme anywhere — its sections are numbered by chapter-plus-lowercase-roman-numeral only. This raises a real, unresolved possibility that Schmidhuber's "Section 5.5.1" locator refers not to the original 1974 dissertation as archived by gwern, but to the section numbering used in the 1994 book reprint, *The Roots of Backpropagation: From Ordered Derivatives to Neural Networks and Political Forecasting* (Wiley, 1994), which republishes the thesis text and could plausibly have been renumbered with a decimal scheme when combined with Werbos's other retrospective essays into a single edited volume. This 1994 book's own table of contents was not located or checked in this session (Google Books, ACM, and Amazon listings for it were found but did not expose a full section-level TOC). This does not overturn the core finding above — Werbos's own 2004 essay independently corroborates that only "a few words" on neural-network-specific ideas exist in the relevant chapter (see next claim), and the full-document search found zero mentions of Linnainmaa regardless of which numbering scheme is used — but it is a genuine, specific loose end that should not be silently resolved. `[unverified — needs a check of the 1994 reprint's actual section numbering to confirm whether "5.5.1" is a renumbering artifact]` **Related notes:** [[claim-werbos-backprop-from-freud-own-account]] --- ## Claim: Werbos's own 2004 retrospective essay, reviewing exactly this history and explicitly engaging with the automatic-differentiation community's terminology, never mentions Linnainmaa **Claim type:** historical (absence of a specific name in a specific first-person retrospective document). **Tier 1–2 required** given its role as corroborating evidence for a load-bearing claim. Achieved Tier 1 — direct full-text read of Werbos's own essay, "Backwards Differentiation in AD and Neural Nets: Past Links and New Opportunities" (published in the AD2004 conference proceedings; also listed in Schmidhuber 2015's bibliography as Werbos, 2006). This essay is Werbos's own account, written specifically to bridge the automatic-differentiation and neural-network communities and their separate histories — it explicitly discusses Griewank's terminology ("the newer generation, which Griewank has called 'the true adjoint method'"), names his own control-theory precursors by name ("I discussed first-generation work by Jacobsen and Mayne, by Bryson and Ho, and by Kashyap"), and states directly that Harvard restricted how much neural-network material could enter the 1974 thesis: > "First, they would not allow ANNs as such to be a major part of the thesis, since I had not found anyone willing to act as a mentor for that part. (I put a few words into chapter 5 to specify essential ideas, but no more.)" This is a first-person confirmation, in Werbos's own words, that Chapter 5's neural-network content was deliberately minimal ("a few words... but no more") — consistent with Schmidhuber's characterization of it as "preliminary" discussion. Despite reviewing this exact history at length, and despite being written in a venue (the AD2004 proceedings) squarely aimed at the automatic-differentiation community that would have made a Linnainmaa citation natural if Werbos considered it relevant, **the name "Linnainmaa" does not appear anywhere in this essay.** **Provenance:** - source_url: https://www.werbos.com/AD2004.pdf - source_author: Paul J. Werbos - source_date: 2004 (AD2004 conference proceedings; also cited as Werbos 2006 in the published book chapter) - source_tier: 1 - exact quote: "First, they would not allow ANNs as such to be a major part of the thesis, since I had not found anyone willing to act as a mentor for that part. (I put a few words into chapter 5 to specify essential ideas, but no more.)" - exact quote 2 (Werbos's own named precursors, none of them Linnainmaa): "There were people at Harvard and MIT then who had used, in control theory, a method very similar to the first-generation adjoint method... In [1], I discussed first-generation work by Jacobsen and Mayne, by Bryson and Ho, and by Kashyap, which was particularly relevant to my larger goals." - related existing notes: [[claim-werbos-backprop-from-freud-own-account]], [[claim-reverse-mode-multiple-independent-discovery]] --- ## Calibration note: search-engine-synthesized summaries again asserted the opposite of what primary sources show As in the earlier, adjacent capture on Werbos's "credit assignment" framing (see [[20260707-0233-did-werboss-1974-harvard]]), a raw WebSearch synthesis encountered during this research directly and confidently asserted a claim contradicted by every primary source located: that "in 1974, Paul Werbos was the first to suggest the possibility to use the backpropagation described by Seppo Linnainmaa, after studying it in depth in his thesis." No source read in this session — not the thesis itself, not Schmidhuber's peer-reviewed paper, not Werbos's own 2004 retrospective — supports this. This claim is recorded here only to flag it as **actively wrong** and not to be trusted or repeated; it is not being promoted as a claim, `[actively contradicted by primary sources — do not use]`. > [!note] Seek's commentary: > The two background searches that anchor this capture (full reads of a 21,754-line thesis extraction and a 2,823-line journal-article extraction) are a genuinely strong way to answer a question like this one — sampling a document this size for a single low-frequency proper noun is exactly the kind of task where a partial read risks a false negative, and both passes here confirmed complete coverage before reporting zero hits. The most interesting finding isn't just "no citation" — it's that the specific locator everyone repeats ("Werbos, 1974, Section 5.5.1") doesn't structurally exist in the copy of the document that's publicly circulating as *the* 1974 thesis. That's worth someone eventually resolving against the 1994 reprint, but it doesn't change the answer to the question asked: nothing found anywhere, in either edition-independent full-text search or in Werbos's own later first-person retrospective, supports the idea that he named Linnainmaa in 1974. The weight of evidence is independent derivation. --- ## Further leads - **1994 reprint's table of contents** (*The Roots of Backpropagation: From Ordered Derivatives to Neural Networks and Political Forecasting*, Wiley) — not located with a full section-level TOC in this session. Checking it directly would resolve the flagged numbering discrepancy above. - **Linnainmaa's own 1970/1976 texts** — still not read directly in any capture on this specific question (though read-adjacent captures exist for the Griewank line of inquiry); would let a future note confirm from the other direction that Linnainmaa's own work shows no awareness of or connection to Werbos. - **Speelpenning (1980)** — cited by both Schmidhuber and Griewank as an early bridge between the AD and NN-adjacent literatures; per existing vault captures (see [[2026-07-01-speelpenning-1980-first-autodiff-cite-linnainmaa-high]]), worth checking whether Speelpenning's paper is the first to put Linnainmaa and Werbos in the same bibliography — which would date the *retrospective* linking of the two lineages more precisely than Schmidhuber's 2015 survey does.