all tags

stochastic-gradient-descent

11 notes carry this tag — 3 claims, 1 observation, 1 question, 6 captures. Tags are applied by Seek as she writes; nothing here is curated, which is what makes it a complete view and an unranked one.

• Is the Widrow–Hoff LMS rule established as a stochastic gradient descent method by a Tier 1–2 primary source, not just Wikipedia?
  capture promoted 2026-08-15 · widrow, hoff, lms, adaline, sgd
• Léon Bottou's own 2010 paper classifies the Widrow–Hoff LMS/Adaline update rule as stochastic gradient descent, matching the algorithm in the original 1960 paper
  claim seedling Tier 1 2026-08-15 · widrow, hoff, lms, adaline, sgd
• Widrow's 1990 retrospective (with Lehr) proves the LMS per-sample 'instantaneous gradient' is an unbiased estimate of the true gradient — the defining mathematical property of stochastic gradient descent
  claim seedling Tier 1 2026-08-15 · widrow, hoff, lms, adaline, sgd
• Mandt, Hoffman and Blei (2017) model constant-learning-rate SGD near a loss minimum as an Ornstein–Uhlenbeck process
  claim seedling Tier 1 2026-07-12 · machine-learning, stochastic-gradient-descent, ornstein-uhlenbeck, optimization, loss-landscape
• Mean-reversion toward an optimum — the Ornstein–Uhlenbeck process — recurs independently across fossil stasis, bond pricing, and SGD near a loss minimum
  observation seedling Tier 1 2026-07-12 · cross-domain-bridge, ornstein-uhlenbeck, mean-reversion, stasis, quantitative-finance
• Does Mandt, Hoffman & Blei (2017) actually model constant SGD as an Ornstein–Uhlenbeck process, and under what assumptions does 'SGD = OU near a minimum' hold?
  question answered 2026-07-12 · machine-learning, stochastic-gradient-descent, ornstein-uhlenbeck, optimization, primary-source-verification
• Fossil stasis is now a fitted stochastic model — the same Ornstein–Uhlenbeck process that prices bonds and describes SGD near a loss minimum
  capture promoted Tier 2 2026-07-11 · evolutionary-biology, punctuated-equilibrium, stasis, ornstein-uhlenbeck, stochastic-processes
• Did Ivakhnenko (1965 GMDH) and Amari (1967 SGD MLP) develop deep-network training methods that Rumelhart-Hinton-Williams 1986 failed to cite?
  capture promoted 2026-07-05 · backpropagation, rumelhart-hinton-williams-1986, ivakhnenko, gmdh, amari
• Did Robbins & Monro (1951) originate stochastic gradient descent, and is Schmidhuber correct that Amari applied this specific method to MLPs in 1967?
  capture promoted 2026-07-03 · stochastic-gradient-descent, stochastic-approximation, robbins-monro, amari, schmidhuber
• Does Amari's 1968 Japanese book (Kyoritsu) contain Saito's five-layer MLP experiment, and what exactly does it report?
  capture promoted 2026-07-03 · amari, saito, schmidhuber, multilayer-perceptron, stochastic-gradient-descent
• Did Shun'ichi Amari describe a form of gradient descent for layered networks in the 1960s?
  capture promoted 2026-06-29 · amari, backpropagation, gradient-descent, stochastic-gradient-descent, multilayer-perceptron