---
title: "Does Mandt, Hoffman & Blei (2017) actually model constant SGD as an Ornstein–Uhlenbeck process, and under what assumptions does 'SGD = OU near a minimum' hold?"
type: "question"
status: "open"
date_raised: "2026-07-12T00:00:00.000Z"
tags: ["machine-learning","stochastic-gradient-descent","ornstein-uhlenbeck","optimization","primary-source-verification","approximate-bayesian-inference"]
writer_model: "claude-opus-4-8"
---


Raised while promoting `10-inbox/raw/2026-07-11-hop-stasis-is-an-ou-process.md` into
[[claim-constant-sgd-near-a-loss-minimum-is-an-ornstein-uhlenbeck-process]] and the
cross-domain bridge [[observation-mean-reversion-to-an-optimum-recurs-across-fossil-stasis-bonds-and-sgd]].
This is the **load-bearing mechanism** of the whole fossils ↔ bonds ↔ SGD bridge, and the
capture itself flagged it: "the load-bearing 'SGD-as-OU' mechanism rests on one Tier-1 paper's
analysis and should be double-checked before publication." The note is Tier 1 but kept `seedling`
until this is discharged.

**Source to read:** Stephan Mandt, Matthew D. Hoffman, David M. Blei, "Stochastic Gradient
Descent as Approximate Bayesian Inference," *JMLR* 18 (2017); arXiv:1704.04289.

**Confirm verbatim** the recorded quote — "Stochastic Gradient Descent with a constant learning
rate (constant SGD) simulates a Markov chain with a stationary distribution" — and its location.

**Then establish the mechanism and its scope:**
1. That the paper explicitly approximates the constant-SGD Markov chain by a **continuous-time
   Ornstein–Uhlenbeck process** (not merely a generic Gaussian stationary distribution).
2. The exact **assumptions** the OU approximation rests on — a constant (non-decaying) learning
   rate, a locally quadratic loss near the minimum, and gradient noise treated as (roughly)
   Gaussian with a fixed covariance. Note which are stated, which are idealizations.
3. The **scope**: the OU picture is a *local* description valid near a minimum, not a global
   account of SGD dynamics. How much of the "SGD is OU" slogan survives outside that regime?

**Why it matters:** if the OU approximation is faithful and its limits are stated honestly, the
cross-domain bridge (paleoTS stasis, Vasicek rates, SGD) is real and publishable. If it is a
looser analogy than the capture implied, the bridge note should be softened. High priority — this
single verification gates whether the bridge is "exact, not analogical" or a structural echo.