--- id: "20260706-0209-did-linnainmaa-originate-reverse" title: "Capture: Did Linnainmaa originate reverse-mode automatic differentiation, per Griewank's 2012 Documenta Mathematica history paper?" type: "capture" status: "promoted" date_promoted: "2026-07-07T00:00:00.000Z" promoted_to: ["30-notes/claim-reverse-mode-multiple-independent-discovery.md"] not_promoted: ["Claim 3 (what Griewank credits Linnainmaa with) — carried as verified_by upgrade on claim-linnainmaa-reverse-mode-single-pass (remediation) rather than a new note","Claim 4 (Copenhagen verbatim) — already promoted 2026-07-06 as claim-linnainmaa-copenhagen-anecdote","Claim 5 (biographical timeline incl. Tienari, Kahan, IEEE-754 disclaimer) — held for the approved Linnainmaa entity hub; person-adjacent detail is Cali-gated","Claim 6 individual contributors (Iri, Bennett, Baur-Strassen, Speelpenning) — named inside the promoted note; per-person notes await their own primaries (none read directly yet, per this capture's own further-leads)"] origin: "batch" model: "claude-sonnet-5" date_created: "2026-07-06T00:00:00.000Z" provenance: "batch run 2026-07-06; harvested from 2026-06-29-what-is-the-reverse-mode-automatic-differentiation-backpropagation-correspondence-stated-precisely [high]; web research via WebSearch + WebFetch, primary paper obtained and read directly as PDF" derived_from: [] tags: ["automatic-differentiation","linnainmaa","griewank","backpropagation","reverse-mode","history-of-ml","documenta-mathematica","autodiff-priority"] source_url: "https://ems.press/content/book-chapter-files/27379" source_author: "Andreas Griewank" source_date: "2012" source_tier: 1 source_venue: "Documenta Mathematica, Extra Volume ISMP (2012), 389–400" source_doi: "10.4171/DMS/6/38" access_note: "Full text obtained and read directly this run (previous 2026-06-28 and 2026-07-01 captures on this same paper could not reach the PDF — EMIS mirrors returned ECONNRESET/ECONNREFUSED, CiteSeerX had an SSL cert error, ResearchGate returned HTTP 403. This run succeeded via the EMS Press chapter-file endpoint linked from https://ems.press/books/dms/251/4949, resolved through WebSearch. All quotes below are read directly from the paper's own text, not relayed through a secondary." prior_captures_on_this_topic: ["10-inbox/raw/2026-06-28-read-griewank-2012-directly-to-verify-a-the-backward-pass-structural-description-of-linnainmaas-algorithm-and-b-the-copenhagen-park-anecdote.md","10-inbox/raw/2026-07-01-griewank-2012-linnainmaa-priority-autodiff-reception-before-2010s.md"] --- > [!note] This run resolves the primary-source access gap flagged in two prior captures > The 2026-06-28 and 2026-07-01 captures on this same paper both hit dead ends trying to reach the Griewank 2012 PDF directly and had to rest their claims on Tier 3–4 secondaries (Yuxi Liu's essay, a Toronto AI Meetup repost of Schmidhuber, Psychology Today UK). This run reached the actual PDF via `https://ems.press/content/book-chapter-files/27379` (linked from the EMS Press catalog page for the paper) and read it directly. Every claim below is sourced to the primary text itself — the exact wording quoted is Griewank's own, not a paraphrase relayed through an intermediary. This should let several `[unverified-mechanism -- needs primary]` and `[unverified-quant -- needs primary]` flags on the earlier captures be lifted on promotion. --- ## Core question: does Griewank credit Linnainmaa as the (sole) originator of reverse-mode differentiation? **Claim type**: historical **Sourcing floor**: Tier 3–4 acceptable for uncontested historical claims, escalating to Tier 1–2 because this is the load-bearing point of the whole note — Tier 1 achieved (primary text, direct read) **Answer: no — not as sole originator.** Griewank's own prologue states the thesis of the paper directly, and it is explicitly a multiple-independent-discovery account, not a single-inventor account: > "In fact, there have been many incarnations of this reversal technique, which has been suggested by several people from various fields since the late 1960s, if not earlier." Griewank also notes that he himself was credited by Nick Trefethen with a specific, narrower role — not invention, but reintroduction to attention: > "Nick Trefethen [13] listed automatic differentiation as one of the 30 great numerical algorithms of the last century. He kindly credited the present author with facilitating the rebirth of the key idea, namely the reverse mode." Within that multiple-discovery frame, Linnainmaa is given the most sustained treatment in the paper and is credited with the specific, general, formal statement of the technique for computing adjoint/Jacobian accumulation — but he is explicitly one of several independent originators Griewank names, not the only one. Griewank frames his choice to center Linnainmaa as a matter of narrative scope, not exclusive priority: > "Since we do not wish to specifically address electronic circuits or chemical processes we will describe the reverse mode from Seppo Linnainmaa's point of view in the following two sections." **Related notes**: [[claim-linnainmaa-priority-not-paternity]], [[backpropagation-gap]] --- ## Claim 1: Griewank names Gerardi Ostrowski's chemical-engineering work as roughly five years earlier than Linnainmaa's **Claim type**: quantitative (relative dating) + historical **Sourcing floor**: Tier 1–2 required for the quantitative element ("five years earlier") — Tier 1 achieved (primary text, direct read) > "Gerardi Ostrowski (OVB71) discovered and used it some five years earlier in the context of certain process models in chemical engineering." This places Ostrowski's independent discovery at roughly 1965, prior to Linnainmaa's 1970 thesis. Griewank cites this immediately after introducing Linnainmaa, as the first of several counter-examples to a single-origin story. **Related notes**: [[claim-linnainmaa-priority-not-paternity]] --- ## Claim 2: Griewank names a second, independent 1960s origin in circuit optimization (Hachtel et al.) **Claim type**: technical-mechanism + historical **Sourcing floor**: Tier 1–2 required — Tier 1 achieved (primary text, direct read) > "Also in the sixties Hachtel et al. [6] considered the optimization of electronic circuits using the costate equation of initial value problems and its discretizations to compute gradients in the reverse mode for explicitly time-dependent problems. Here we see, possibly for the first time, the close connection between the reverse mode of discrete evaluation procedures and continuous adjoints of differential equations." The bibliographic reference Griewank gives for this is: G.D. Hachtel, R.K. Brayton, F.G. Gustavson, "The sparse tableau approach to network design and analysis," *IEEE Transactions of Circuit Theory* 18(1):102–113, 1971. **Related notes**: [[claim-linnainmaa-priority-not-paternity]] --- ## Claim 3: What Griewank credits Linnainmaa's thesis with, specifically — the general reverse-mode recurrence for adjoint/Jacobian accumulation **Claim type**: technical-mechanism **Sourcing floor**: Tier 1–2 required — Tier 1 achieved (primary text, direct read) This is the paper's central technical claim about Linnainmaa specifically, stated in the section "Jacobian accumulation": > "Now we turn to the aspect of Seppo Linnainmaa's thesis that is most interesting to us, namely the fact that he proposed what is now known as the reverse mode for calculating the adjoint coefficients ūi." Griewank elaborates the mechanism in Linnainmaa's own terms: for a vector function evaluated as a sequence of elemental assignments over a computational graph, the adjoint coefficients can be accumulated by traversing the graph backward: > "However, if the number m of dependents is much smaller than the number n of independents, Jacobians should be accumulated in the reverse mode as already suggested by Linnainmaa. Namely, one can traverse the computational graph backward to compute the adjoint vectors ūi defined above by the recurrence... Note that the ūj are computed backward, starting from the terminal values..." On vocabulary specifically: Griewank's own term for the structural backward step is "traverse... backward" / "reverse mode" / describing the process as proceeding "backward, i.e., here downward through the computational graph" — he does not use the compound phrase "backward pass" in this paper. This directly resolves the open vocabulary question flagged in the 2026-06-28 capture (Claim 3 there), which could only confirm "reverse procedure" language from the 2014 follow-up paper and left the 2012 paper's own wording unconfirmed. **Related notes**: [[claim-linnainmaa-reverse-mode-single-pass]], [[backpropagation-gap]] --- ## Claim 4: The Copenhagen park anecdote — verbatim, now confirmed against primary text **Claim type**: historical/biographical **Sourcing floor**: Tier 3–4 acceptable — Tier 1 achieved (primary text, direct read), upgrading the 2026-06-28 capture's Tier 3–4 secondary sourcing > "Seppo Linnainmaa (Lin76) of Helsinki says the idea came to him on a sunny afternoon in a Copenhagen park in 1970. He used it as a tool for estimating the effects of arithmetic rounding errors on the results of complex expressions." This confirms, verbatim and from the primary text directly, the anecdote that the 2026-06-28 capture could only source to a Tier 3 (Psychology Today UK) and Tier 4 (Toronto AI Meetup repost of Schmidhuber) secondary. The quote appears on page 389 of the paper (the first page, in the Prologue), not page 391 as the earlier capture's secondary source attributed it — the page-391 attribution in that capture should be corrected on promotion. **Related notes**: [[claim-linnainmaa-rounding-error-problem]], [[claim-linnainmaa-thesis-identity]] --- ## Claim 5: Linnainmaa's institutional and biographical timeline, per Griewank's own account **Claim type**: historical/biographical (dates) — escalated to Tier 1–2 given the specificity of the dates involved **Sourcing floor**: Tier 1 achieved (primary text, direct read) Griewank's biographical section ("Round-off Analysis à la Linnainmaa") gives a specific timeline: > "In 1967 he enrolled in the first computer science class ever at the University of Helsinki. However, since there were still only very few computer science courses, much of his studies were in mathematics... His supervisor Martti Tienari had worked for Nokia until he became an associate professor of computer science in 1967." > "After finishing his Master Thesis concerning the Estimation of Rounding Errors in 1970 he obtained, four years later, the first doctorate ever awarded in computer science at Helsinki University. In 1977 he got a Finnish grant as a visiting scholar with William Kahan at Berkeley, whose group was instrumental in developing the later IEEE Standard 754. Linnainmaa does not think that the results of his thesis had any specific impact on the development of the standard." This dates Linnainmaa's PhD to 1974 (1970 + four years, as stated — not given as a bare figure but derivable directly from Griewank's own sentence). It also confirms, per Griewank directly, that Linnainmaa's own view is that his thesis had no specific bearing on IEEE 754 — worth noting as a claim Linnainmaa himself is quoted (via Griewank) as disclaiming, rather than one Griewank asserts on his own authority. Griewank also states the reception gap directly, in his own words: > "Moreover, he did not market his approach as a method for cheaply evaluating gradients either, so there was little resonance until I called him up from Argonne in the late eighties. In fact, only in 1976 he published some of the results from his thesis in English." This is a first-person, Tier 1 confirmation (Griewank narrating his own outreach to Linnainmaa) of the reception-gap claim that the 2026-07-01 capture had to source from Baydin et al. 2018's more general "fields... largely unaware of each other" framing. Griewank's own account narrows this specifically to himself contacting Linnainmaa directly "in the late eighties." **Related notes**: [[claim-linnainmaa-field-numerical-analysis]], [[claim-linnainmaa-thesis-identity]] --- ## Claim 6: Other independent contributors Griewank names in the same multiple-discovery account **Claim type**: historical **Sourcing floor**: Tier 3–4 acceptable for uncontested historical claims — Tier 1 achieved (primary text, direct read) Griewank's prologue and early sections name several additional independent contributors, reinforcing the multiple-discovery framing rather than singular attribution: > "In the 1970s Iri analyzed the properties of dual and adjoint networks. In the 1980s he became one of the key researchers on the reverse mode." > "This problem [reversal of a program] was discussed in the context of Turing Machines by Benett (Ben73), who foreshadowed the use of checkpointing as a tradeoff between numerical computational effort and memory requirement." > "Motivated by the special case of back-propagation in neural networks, Paul Werbos (Wer82) compared the forward and reverse propagation of derivatives for discrete time-depedent problems with independent numbers of input, state, and output variables." > "Bernt Speelpenning (Spe80) arrived at the reverse mode via compiler optimization when Bill Gear asked him to automatically generate efficient codes for Jacobians of stiff ODEs. I myself rediscovered it once more in the summer of 1987 when, newly arrived at Argonne, I was challenged by Jorge Moré to give an example of an objective function whose gradient could not be evaluated at about the same cost as the function itself." Griewank also notes the reverse mode is widely known among computer scientists under a different name for the rational-function special case: > "Many computer scientists know the reverse mode as the Baur-Strassen method (BS83) for computing gradients of rational functions that are evaluated by a sequence of arithmetic operations." **Related notes**: [[backpropagation-gap]] --- ## Further leads - **Linnainmaa's original 1976 BIT paper** ("Taylor expansion of the accumulation of floating point rounding errors," *BIT Numerical Mathematics*) — still not read directly in any capture on this topic; would let a future run verify Linnainmaa's own wording independent of Griewank's characterization of it. - **Griewank's bibliography items (Ben73, OVB71, Wer82, BS83, Spe80)** — none of these underlying primary sources for the *other* independent discoverers (Ostrowski, Bennett, Werbos, Baur-Strassen, Speelpenning) have been read directly in any capture so far; the multiple-discovery claim currently rests entirely on Griewank's own secondary characterization of each, which is fine for Griewank's-view-of-history claims but would need each contributor's own paper to independently verify what *they* claimed. - **Whether Griewank 2012 anywhere explicitly ranks Linnainmaa above Ostrowski/Hachtel et al. as "the" originator** — not found in this read. The paper's structure (spending two full sections on Linnainmaa vs. one paragraph each on the others) suggests an implicit editorial choice to center him, but Griewank gives an explicit, non-priority reason for that choice (avoiding chemical-engineering and circuit-theory domain detail), not a claim that Linnainmaa's contribution was foundational or first. > [!note] Seek's commentary: > The honest answer to the harvested topic question is "no, and Griewank goes out of his way to say so" — the paper's title is a question, and the first two paragraphs of the prologue answer it by naming at least three independent 1960s–70s originators before Linnainmaa gets his dedicated section. The popular retelling of this paper (visible in the Toronto AI Meetup repost and some ML-history blog posts) tends to compress it to "Griewank's 2012 paper proves Linnainmaa invented backprop/reverse-mode AD," which overstates what the primary text actually argues. Griewank's own framing is closer to "multiply and independently rediscovered, of which Linnainmaa's is the version I'll walk through in detail" — a genuinely different claim, and one worth being precise about on promotion.