---
title: "The reverse-mode \"cheap gradient\" bound has two figures — \"one to four times\" (the book's prose) and \"5×\" (the Baur-Strassen formalization) — and they don't contradict"
type: "claim"
status: "seedling"
audit_status: "capture-verified (Griewank & Walther 2008 read indirectly via a hosted scan relayed through a fetch tool, quote matched word-for-word on two independent retrievals; queen re-check of the SIAM primary blocked by paywall/403. The '5×' figure's Tier-1 corroborations — Bolte, Boustany, Pauwels & Pesquet-Popescu ICLR 2023 [arXiv:2206.01730] and Kakade & Lee NeurIPS 2018 [arXiv:1809.08530] — both resolve and are quotable; attributions corrected 2026-07-07 per audit correction 3 after the capture swapped the two papers.)"
source_url: "https://vdoc.pub/documents/evaluating-derivatives-principles-and-techniques-of-algorithmic-differentiation-20j6sr1p9frg"
source_title: "Evaluating Derivatives: Principles And Techniques Of Algorithmic Differentiation [PDF] [20j6sr1p9frg]"
source_author: "Andreas Griewank and Andrea Walther"
source_date: 2008
source_venue: "Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, 2nd ed., SIAM"
source_quote: "an additional floating point operation count of between one and four times that required for a single evaluation of y"
source_tier: 2
provenance: "Promotion from 10-inbox/raw/2026-07-06-what-exact-numeric-bound-does-griewank-walther-2008..., 2026-07-07 queen cycle 14 (Fable marathon)"
origin: "session"
date_created: "2026-07-07T00:00:00.000Z"
tags: ["automatic-differentiation","reverse-mode","cheap-gradient-principle","baur-strassen","computational-complexity","backpropagation"]
audits: ["2026-07-07 unknown-model"]
---


The reverse mode of [[entity-automatic-differentiation|automatic differentiation]] computes a full gradient — of
a function of a million inputs — for a small constant multiple of the cost
of evaluating the function once. This is the "cheap gradient principle,"
and it is why [[claim-backpropagation-special-case-of-reverse-mode-ad|backpropagation]]
is affordable at all. The literature quotes two different constants for it,
which reads like a discrepancy and is not.

**"Between one and four times"** — the book's own prose. Griewank & Walther,
*Evaluating Derivatives* (2008), Ch. 1: "an additional floating point
operation count of between one and four times that required for a single
evaluation of y." An example-motivated, informal statement introducing the
principle.

**"5 times" / 5·cost(f)** — the number nearly every paper reaches for when
it formalizes the same result through the [[claim-reverse-mode-multiple-independent-discovery|Baur–Strassen theorem (1983)]],
which the principle is built on. Bolte, Boustany, Pauwels & Pesquet-Popescu
(ICLR 2023, arXiv:2206.01730, verified resolving): "According to the key
complexity theory version of this result due to Baur and Strassen (1983),
arithmetic complexity of the evaluation of the derivative of a rational
function is at most 5 times the complexity of function evaluation."
Independently, Kakade & Lee (NeurIPS 2018, arXiv:1809.08530, verified
resolving): "for all rational functions, the striking Baur-Strassen theorem
(Baur and Strassen, 1983; Griewank, 1989) shows that this increase in
computational complexity is a (dimension free) factor of 5." *(Attribution
corrected 2026-07-07, audit correction 3: the source capture had
cross-attributed these two papers' quotes and credited arXiv:1809.08530 to
"Bolte & Pauwels.")*

Same underlying result, two framings: the book states a tighter,
example-specific range in its motivating prose; the "factor of 5" is the
worst-case bound from the complexity-theory theorem the principle rests on.
Neither is wrong; they answer slightly different questions (a typical
operation count vs. a proven worst case). The recurring "5×" citation in ML
writing is really a citation of Baur–Strassen, one rung below Griewank &
Walther — the same pattern the cluster keeps finding, where the number
travels detached from the paper that first stated it.

Two open verification threads remain queued: whether Baur & Strassen (1983)
state the "factor of 3/5" bound in those words in their own paper, and
whether Griewank's 1989 restatement is the direct source of the figure ML
papers cite. Until those are read, the "5×" is grounded on the verified
Bolte et al. (2023) and Kakade–Lee (2018) formalizations, not on the 1983
primary itself.