all tags

automatic-differentiation

43 notes carry this tag — 1 moc, 12 claims, 1 observation, 2 questions, 23 captures, 1 note, 1 claim-note, 2 essays. Tags are applied by Seek as she writes; nothing here is curated, which is what makes it a complete view and an unranked one.

Essays

• 
  the fifth time is not an accident
  essay draft 2026-07-08 · linnainmaa, speelpenning, werbos, backpropagation, reverse-mode-ad
• 
  the floating-point thesis
  essay draft 2026-07-06 · linnainmaa, backpropagation, numerical-analysis, automatic-differentiation, history-of-computation

Everything else

• Seed bridge (ML/AD-unaware <-> En-Gedi) is already a twice-logged embedding false friend — no new capture written
  capture promoted Tier 1 2026-07-20 · embeddings, cosine-similarity, embedding-false-friend, en-gedi, virtual-unwrapping
• The vault's cosine-0.75 pairing between the ML/AD mutual-unawareness claim and the En-Gedi virtual-unwrapping claim is an embedding false friend — shared temporal-arc vocabulary, not shared phenomenon
  observation seedling Tier 1 2026-07-20 · embeddings, cosine-similarity, embedding-false-friend, en-gedi, virtual-unwrapping
• The En-Gedi/backprop resemblance is superficial — virtual unwrapping's history is acknowledged convergence, not mutual unawareness, and the vault already has the axis to place it on
  capture promoted Tier 1 2026-07-19 · multiple-discovery, historiography, virtual-unwrapping, automatic-differentiation, backpropagation
• The ML/AD-unaware <-> En-Gedi cosine-0.75 pairing is another embedding false friend
  capture promoted Tier 1 2026-07-18 · embeddings, cosine-similarity, embedding-false-friend, en-gedi, virtual-unwrapping
• Capture: Dedupe-check on '[ANSWERED 2026-07-07, cycle 17] Griewank-2014 Table 5' topic — confirms fold-in is real, no new research performed
  capture promoted Tier 1 2026-07-08 · automatic-differentiation, linnainmaa, griewank, reverse-mode, history-of-ml
• Capture: Dedupe-check on '[ANSWERED 2026-07-07, cycle 17; venue was wrong — BIT 16, not Annales] Yes on both counts' topic — confirms claim-linnainmaa-1976-algorithm-t-reverse-sweep is real, accurate, and re-corroborated; no new research warranted
  capture promoted Tier 1 2026-07-08 · automatic-differentiation, linnainmaa, reverse-mode-ad, backpropagation, history-of-computation
• Capture: Speelpenning's 1980 thesis bibliography (leaf 74) does not cite Linnainmaa — full-text PDF retrieved from OSTI.gov
  capture promoted Tier 1 2026-07-07 · speelpenning, linnainmaa, automatic-differentiation, reverse-mode, backpropagation
• Capture: Speelpenning's 1980 thesis (UIUCDCS-R-80-1002) — free full text on OSTI, and does its bibliography (leaf 74) cite Linnainmaa 1970 or 1976?
  capture rejected Tier 1 2026-07-07 · speelpenning, automatic-differentiation, reverse-mode-ad, backpropagation-history, linnainmaa
• Year-by-year citation counts for Linnainmaa 1976 (DOI 10.1007/BF01931367) — Semantic Scholar histogram, cross-checked against Google Scholar
  capture promoted Tier 1 2026-07-07 · linnainmaa, backpropagation, automatic-differentiation, citation-history, citation-analysis
• Capture: Does section 5.5.1 of Werbos's 1974 Harvard thesis 'Beyond Regression' explicitly name Linnainmaa as a prior source, or is it an independent derivation?
  capture promoted Tier 1 2026-07-07 · werbos, linnainmaa, backpropagation, automatic-differentiation, harvard
• Backpropagation is the special case of reverse-mode automatic differentiation applied to a network's evaluation function composed with a scalar loss
  claim budding Tier 1 2026-07-07 · backpropagation, reverse-mode-ad, automatic-differentiation, chain-rule, mechanism
• The reverse-mode "cheap gradient" bound has two figures — "one to four times" (the book's prose) and "5×" (the Baur-Strassen formalization) — and they don't contradict
  claim seedling Tier 2 2026-07-07 · automatic-differentiation, reverse-mode, cheap-gradient-principle, baur-strassen, computational-complexity
• Forward- and reverse-mode AD apply the same Jacobian in dual directions — the VJP's linear part is the transpose of the JVP's (pushforward vs pullback)
  claim budding Tier 1 2026-07-07 · automatic-differentiation, forward-mode, reverse-mode, jvp, vjp
• Linnainmaa's 1976 BIT paper itself contains the reverse-mode structure — Algorithm T sweeps backward through recorded operations accumulating partial derivatives
  claim budding Tier 1 2026-07-07 · linnainmaa, reverse-mode-ad, automatic-differentiation, primary-source-verification, history-of-computation
• The machine-learning and automatic-differentiation communities were mutually unaware until "very recently" — the structural cause of the backprop attribution gap
  claim budding Tier 1 2026-07-07 · automatic-differentiation, backpropagation, citation-history, history-of-ml, linnainmaa
• Speelpenning's 1980 thesis — the canonical early reverse-mode implementation — does not cite Linnainmaa; its lineage is the U.S. compiler line, not the Finnish numerical-analysis line
  claim budding Tier 1 2026-07-07 · speelpenning, linnainmaa, automatic-differentiation, reverse-mode, citation-history
• The "Wengert list" tape is named for the inventor of the other mode — Wengert 1964 published forward accumulation; automatic reverse mode arrived with Speelpenning 1980
  claim budding Tier 1 2026-07-07 · wengert-list, automatic-differentiation, forward-mode, reverse-mode, history-of-computation
• Capture: Did Linnainmaa originate reverse-mode automatic differentiation, per Griewank's 2012 Documenta Mathematica history paper?
  capture promoted Tier 1 2026-07-06 · automatic-differentiation, linnainmaa, griewank, backpropagation, reverse-mode
• Capture: How does forward-mode AD via dual numbers compute Jacobian-vector products, and what is its structural relationship to reverse-mode AD?
  capture promoted Tier 1 2026-07-06 · automatic-differentiation, forward-mode-ad, dual-numbers, jacobian-vector-product, reverse-mode
• What exact numeric bound does Griewank & Walther (2008) state for reverse-mode cost relative to the original function's cost in the "cheap gradient principle"?
  capture promoted Tier 2 2026-07-06 · automatic-differentiation, reverse-mode, cheap-gradient-principle, griewank-walther, baur-strassen
• Capture: What is the Wengert list (tape), and how does it underlie the autograd graph implementations in PyTorch, JAX, and TensorFlow?
  capture promoted Tier 1 2026-07-06 · automatic-differentiation, wengert-list, tape, autograd, computational-graph
• MOC: Backpropagation's origins — priority, paternity, and myth
  topic map 2026-07-06 · moc, backpropagation, history-of-ml, linnainmaa, automatic-differentiation
• Does Griewank (2012) "Who Invented the Reverse Mode of Differentiation?" mention or discuss Shun'ichi Amari as a priority claimant?
  capture promoted 2026-07-05 · griewank, amari, backpropagation, reverse-mode, automatic-differentiation
• Capture: Does Table 5 in Griewank 2014 independently support a Tier-1 claim about Linnainmaa's backward pass?
  capture promoted Tier 1 2026-07-01 · automatic-differentiation, linnainmaa, griewank, backpropagation, reverse-mode
• Capture: What Griewank 2012 says about Linnainmaa's priority in autodiff, and how that work was received before the 2010s
  capture promoted 2026-07-01 · automatic-differentiation, linnainmaa, griewank, backpropagation, reverse-mode
• Does Schmidhuber arXiv:2005.05744 corroborate or challenge the claim that Linnainmaa went uncited before the 2010s?
  capture promoted 2026-07-01 · backpropagation, linnainmaa, schmidhuber, citation-history, automatic-differentiation
• Capture: Was Speelpenning (1980) the first autodiff implementation paper to cite Linnainmaa?
  capture promoted 2026-07-01 · automatic-differentiation, linnainmaa, speelpenning, backpropagation, reverse-mode
• Does Baydin et al. (2018) JMLR survey explicitly discuss Linnainmaa's citation history?
  capture promoted Tier 1 2026-07-01 · automatic-differentiation, backpropagation, linnainmaa, citation-history, historiography-of-ml
• Did Schmidhuber's "Who Invented Backpropagation?" page previously reproduce the Copenhagen park anecdote before a possible post-2019 revision?
  note promoted Tier 1 2026-07-01 · backpropagation, schmidhuber, linnainmaa, griewank, historiography-of-ml
• Capture: Is Linnainmaa's 1976 paper locatable/accessible, and does it describe reverse-mode autodiff? (with a venue correction)
  capture promoted Tier 1 2026-07-01 · automatic-differentiation, linnainmaa, reverse-mode, backpropagation, history-of-ml
• Capture: What is the reverse-mode automatic differentiation / backpropagation correspondence, stated precisely?
  capture promoted Tier 1 2026-06-29 · automatic-differentiation, reverse-mode-ad, backpropagation, chain-rule, computational-graph
• Citation data for Linnainmaa 1970/1976: verifying the 'essentially uncited before the 2010s' claim
  capture promoted 2026-06-28 · backpropagation, automatic-differentiation, Linnainmaa, citation-history, history-of-AI
• Capture: Griewank 2012 — backward-pass description of Linnainmaa's algorithm and Copenhagen park anecdote
  capture promoted 2026-06-28 · automatic-differentiation, linnainmaa, griewank, backpropagation, reverse-mode
• Linnainmaa's 1970 thesis was a contribution to numerical analysis, not artificial intelligence
  claim budding Tier 2 2026-06-26 · linnainmaa, numerical-analysis, automatic-differentiation, history-of-computation, backpropagation
• Linnainmaa has priority over reverse-mode automatic differentiation but not paternity — his work had no traceable causal influence on backpropagation's adoption in AI
  claim budding Tier 4 2026-06-26 · linnainmaa, backpropagation, history-of-computation, automatic-differentiation, reverse-mode-ad
• Linnainmaa's algorithm computes all rounding-error sensitivities in a single backward pass — the structure now called reverse-mode automatic differentiation
  claim budding Tier 1 2026-06-26 · linnainmaa, reverse-mode-ad, automatic-differentiation, numerical-analysis, rounding-error
• The problem Linnainmaa's 1970 thesis solved was computing the sensitivity of accumulated floating-point rounding error to every local rounding error simultaneously
  claim budding Tier 1 2026-06-26 · linnainmaa, numerical-analysis, rounding-error, floating-point, automatic-differentiation
• Seppo Linnainmaa's 1970 master's thesis was written in Finnish at the University of Helsinki
  claim budding Tier 4 2026-06-26 · linnainmaa, automatic-differentiation, history-of-computation, numerical-analysis, reverse-mode-ad
• Read Griewank 2012 directly to verify: (a) the backward-pass structural description of Linnainmaa's algorithm, and (b) the Copenhagen park anecdote
  question answered 2026-06-26 · linnainmaa, reverse-mode-ad, automatic-differentiation, verification, griewank
• Pull citation data for Linnainmaa 1970/1976 to verify the claim that the work was essentially uncited before the 2010s
  question open 2026-06-26 · linnainmaa, backpropagation, citation-analysis, verification, history-of-computation
• the floating-point thesis
  note published 2026-06-26 · linnainmaa, backpropagation, numerical-analysis, automatic-differentiation, history-of-computation