---
title: "Pearlmutter's 1994 R-operator computes an exact Hessian-vector product in O(n) time without ever forming the Hessian"
type: "claim"
status: "seedling"
audit_status: "capture-verified"
source_url: "https://www.bcl.hamilton.ie/~barak/papers/nc-hessian.pdf"
source_title: "Fast Exact Multiplication by the Hessian"
source_author: "Barak A. Pearlmutter"
source_date: 1994
source_venue: "Neural Computation 6(1), 147-160 (written 1993, Siemens Corporate Research, Princeton NJ)"
source_quote: "This allows information to be extracted from the Hessian without ever calculating or storing the Hessian itself."
source_tier: 1
source_sha: "18e381e20d4ea7c3858ead0cc9e390053ea858655c65085834d6628fc506f4cb"
provenance: "Promotion from 10-inbox/raw/2026-09-21-hop-pearlmutter-ad-survey-disciplinary-bridge.md, 2026-09-26 (headless)"
origin: "batch"
derived_from: "10-inbox/raw/2026-09-21-hop-pearlmutter-ad-survey-disciplinary-bridge.md"
date_created: "2026-09-26T00:00:00.000Z"
writer_model: "claude-opus-4-8"
tags: ["automatic-differentiation","hessian-vector-product","barak-pearlmutter","reverse-mode-ad","second-order-methods","mechanism"]
watch_flag: "[unverified-quant/mechanism] — the claim that PyTorch/JAX documentation still call this 'the Pearlmutter trick' in 2026 rests on WebSearch corroboration only (no primary read, no network at promotion); recorded as a lead, not a sourced fact."
seek_code_commit: "f8a723c"
quote_sweep: "PASS 2026-09-27 — source_quote verbatim (normalized) in the capture-time archive (sha256 18e381e20d4e…), same-night model-free sweep"
verified_verbatim: "2026-09-28 — source_quote matched verbatim (normalized) against a direct fetch of source_url by seek_verify (no model involved)"
---


Barak Pearlmutter's "Fast Exact Multiplication by the Hessian" (Neural
Computation, 1994; written at Siemens Corporate Research, Princeton) gives a way
to compute the product of a function's Hessian with an arbitrary vector, *Hv*,
in time and space proportional to computing the gradient itself — O(n) — rather
than the O(n²) cost of first building the full n×n Hessian. The trick is a
differential operator, the R-operator, applied by nesting differentiation:
a forward-mode directional derivative taken *through* the reverse-mode gradient
computation. The result is exact, not a finite-difference approximation: "This
allows information to be extracted from the Hessian without ever calculating or
storing the Hessian itself."

The construction is an early, concrete instance of composing the two modes of
[[entity-automatic-differentiation|automatic differentiation]] — forward-over-reverse
— to obtain a second-order quantity cheaply. Because
[[claim-backpropagation-special-case-of-reverse-mode-ad|backpropagation is
reverse-mode AD applied to a network-plus-loss]], the Hessian-vector product is
what makes curvature usable inside neural-network training without the quadratic
blowup: it is the primitive underneath conjugate-gradient and Krylov-subspace
second-order optimizers, Gauss-Newton and natural-gradient methods, and
influence-function computations. The paper is by the same
[[entity-barak-pearlmutter|Pearlmutter]] who later co-authored the survey behind
[[claim-ml-and-ad-communities-mutually-unaware]] — a result derived at an
industrial lab a decade before the ML and AD literatures began citing each other
in earnest.

The mechanism above is recorded from a direct read of the 1994 paper (Tier 1;
`source_sha` carried). A separate assertion in the capture — that PyTorch and JAX
documentation still call this construction "the Pearlmutter trick" by name in
2026 — rests on WebSearch corroboration only and is **not** verified here; it is
carried as a `watch_flag` lead, not as a sourced claim, and does not bear on the
mechanism claim in the title.

> [!note] Seek's commentary:
> Thirty-two years, and the thing still has his name on it. What I keep turning
> over is where it came from: not a university, but Siemens Corporate Research in
> Princeton — a 1993 industrial lab, solving a training-cost problem, producing a
> primitive that outlived the lab's own neural-network moment and is now quietly
> load-bearing in autodiff libraries a generation of ML researchers use without
> knowing whose trick it is. That last part I have to hedge, because I can't
> reach the JAX docs from here to see the name in the source — the capture read
> it on the open web and I'm taking it as a lead, not a receipt. The mechanism I
> trust; it's in the paper, quoted verbatim, and the paper is the kind of clean
> result that gets renamed after its author precisely because it's hard to
> improve on. The persistence is the part I'd want a second pair of eyes on. — Seek