---
title: "MOC: Convergence stitched into a descent — the dual-control → POMDP → LLM-agent genealogy, read to the primaries"
type: "moc"
tags: ["moc","pomdp","dual-control","feldbaum","karl-astrom","reinforcement-learning","control-theory","llm-agents","citation-genealogy","historiography","history-of-science"]
updated: "2026-10-05T00:00:00.000Z"
writer_model: "claude-opus-4-8"
provenance: "Built by the Warden pass 2026-10-02 (warden/claude-opus-4.8, seek-warden-spec.md), as the standing #1-unmapped-hub build-or-decline candidate (read-first item 5): history-of-science (91 notes by the tag-based ranker; 52 real frontmatter tags). The ranker undercounts silos (Cali's 2026-09-01 directive), so the hub was carved by direct read, not by tag — this is one argument-silo lifted out of it, gathered by reading the member notes this pass. No seek-flags.md flag asks for this MOC (the candidate comes from constellation-latest.md's ranked-hubs section, not a dated flag), so no `→ resolved` line is appended there; the artifact and its provenance live here and in warden-2026-10-02.md."
seek_code_commit: "7d42f50"
audit_status: "2026-10-04 cross-model audit (writer claude-opus-4-8, auditor claude-fable-5): first cross-model read of this map. All member/neighbor/entity wikilinks resolve. Every body quotation checked verbatim against its member note (the Meijer & Rantzer head-of-chain sentence; the 1998 abstract's operations-research sentence; 'a partially observable Markov decision process (POMDP)'; the no-document-level-path sentence; the ref-13 'statement about documents' sentence; the 'agent'-absence and control-vocabulary claims; the queuing-theory introduction quote; 'one more layer back'; the bandit-lineage spine sentence; 'excessively cumbersome' / 'comparatively simple problems' / 'practically unsolvable for most cases'; риск действия / риск изучения / 'probed by actions' / 'dual character'; 'citing the pointer, not the thing pointed at'). Externally re-verified this session: Kaelbling, Littman & Cassandra 1998 re-fetched live via extract_pdf (sha256 identical to the member's source_sha, 71a6d1ae…), reference [1] confirmed printed as 1995 and the abstract sentence verbatim. Member flags carried as stated (two watch_flags; [unverified-mechanism]; the 2026-09-15 'propagated into' correction; the Klenske & Hennig cap discharge matches the member records, including the member's own 2026-09-17 cap-roster correction). ONE CORRECTION, honesty-caveat block → was: 'the Åström legs carry verified_verbatim stamps and a 2026-09-27 claude-fable-5 full-paper re-read; the Fel'dbaum legs carry a 2026-09-17 claude-fable-5 re-fetch of all three Russian/English primaries with sha256 matches' → now: coverage stated per member — four of six Åström legs verified_verbatim, the full-paper re-read recorded on the two reference-list notes only, the 09-17 re-fetch covering the impractical-method and bandit-lineage notes' sources; the Notes-section, introduction-generalization, and risk-of-action/risk-of-study legs are capture-verified only with no second-model re-read (old wording preserved inline in the block) → pointer: the three members' own audit_status fields and 00-meta/audits/audit-scheduled-2026-09-27-fable-1.md. Logged in 00-meta/audits/audit-scheduled-2026-10-04-fable-3.md."
---


There is a tidy one-line genealogy for why 2026's LLM agents are formalized as
POMDPs: Fel'dbaum posed dual control in Moscow around 1960, Åström turned it
into the POMDP in 1965, Kaelbling, Littman & Cassandra canonized the POMDP for
AI in 1998, and the acronym reaches a language-model agent sixty-one years
later. Read to the primaries — to Åström's own bibliography, to Fel'dbaum's own
Russian papers, to the survey that is usually quoted for the one-line story —
the single bloodline comes apart into **several independent strands that share a
general piece of mathematics and were stitched into one ancestry only after the
fact.** The math is real and it really does get reused; what is assembled
backward is the *descent*. The argument of this map is the seam between those
two things.

**Scope, and what this is not.** This is the deep *single-instance* map of one
argument — the same relationship the Werbos map
([[moc-testimony-is-not-a-record]]) and the Lepsius map
([[moc-a-founding-practitioner-is-not-an-inventor]]) hold to their general
homes. The **argument** is a historiographic one: a retrospective genealogy
reads continuous technical descent where the record shows parallel reuse of
general structure — the family that also holds [[moc-the-echo-mistaken-for-an-ancestor]],
[[moc-legitimation-not-origination]], [[moc-a-phrase-is-not-a-pedigree]] and
[[moc-a-precursor-is-not-an-influence]].
The **instance** is dual control → POMDP → the LLM agent. It is **distinct from
[[moc-backpropagation-origins]]**, and one of its own members says why: Fel'dbaum's
problem is "about learning *while* controlling, not about computing gradients
through a control sequence" — a different mid-century control-theory lineage, not
the multiple-discovery-of-backprop one. This map does not claim anyone lied or
blundered; most of its notes refuse that reading. It is a claim about how a
genealogy gets *built* — and about how rarely anyone rereads it against the
papers it names.

## The received descent — the claim under the lens
The one-line story in its strongest, genuinely-sourced form, so the map tests the
real thing and not a straw version.

- [[claim-feldbaum-1960s-dual-control-formalized-exploration-exploitation-tradeoff]]
  — the head of the chain, on a 2026 Meijer & Rantzer survey: Fel'dbaum "the
  first researcher to formulate" the exploration–exploitation tradeoff "in its
  full generality," which "propagated into… reinforcement learning." The note
  already carries its own 2026-09-15 correction narrowing that "propagated into."
- [[claim-kaelbling-1998-cites-astrom-1965-as-pomdp-origin]] — the canonization
  step: Åström's 1965 paper is reference [1] of the 1998 *Artificial Intelligence*
  paper that carried the POMDP into mainstream AI planning as a named, citable
  object ("we bring techniques from operations research to bear…").
- [[claim-2026-comap-paper-formalizes-llm-agent-as-pomdp]] — the terminal reuse:
  a 2026 LLM-agent paper calls its agent's decision process "a partially
  observable Markov decision process (POMDP)" — the same object, the noun in
  front of it changed from *controller* to *agent*.

## Read to the bibliography, the document-level descent fails
The strongest leg against a clean descent is Åström's own reference list — a
statement about documents, not just people.

- [[claim-astrom-1965-notes-section-cites-feldbaum-pontryagin-kolmogorov-lineage]]
  — Åström situates himself in Bellman–Fel'dbaum–Pontryagin–Kolmogorov control
  theory and credits the *foundations* to Bellman; nothing situates the paper in
  AI. The first primary-source confirmation that Åström cites Fel'dbaum at all.
- [[claim-astrom-1965-reference-list-contains-no-feldbaum-1960-dual-control-papers]]
  — but he cites only the 1962 Markov-objects paper, as one Markov-modeller among
  several — **not** the 1960 *Dual Control Theory* papers. "No document-level path
  from the 1960 papers to Åström's POMDP formulation appears in Åström's own
  references." (Carries a scope `watch_flag`: Notes/References read, not every
  body line.)
- [[claim-astrom-1965-reference-13-matches-feldbaum-1962-avtomat-telemekh-23-8-pp-993-1007-with-volume-slip]]
  — what reference [13] *is*: Fel'dbaum 1962, title/year/pages matching, with an
  unresolved volume slip (Åström prints 24, the archive says 23) — "what makes the
  negative finding a statement about documents and not just about a person."

## The author spoke control, not agency
The "it was already the modern agent" reading is layered on; Åström's own
vocabulary and examples are control-theoretic and operations-research, not AI.

- [[claim-astrom-1965-paper-uses-control-vocabulary-not-agent-language]] — control
  systems, control variables, control laws, strategies; the word "agent" never
  occurs. The author describing his own IBM paper-mill work had no reason to reach
  for a word that would not mean what it means today for two more decades.
- [[claim-astrom-1965-introduction-claims-generalization-beyond-control-systems]]
  — the honest counterweight, kept visible: Åström *did* claim his result would
  generalize — he said so himself, on page one — but reached for **queuing
  theory**, the nearest analogous field in 1965, not for artificial intelligence.
  The generalization was real; the destination was unknowable to him.
- [[claim-astrom-1965-second-example-is-howards-toymaker-problem]] — the shape that
  reads as "an agent under hidden-state uncertainty" is borrowed sideways from
  Howard's 1960 *toymaker* business-decision example, "one more layer back,"
  dressed as manufacturing economics — reuse of a good example, not a step toward
  language models.

## The reinforcement-learning lineage is independent, and the formalism is narrower than quoted
Where the one-line story most overreaches: mainstream RL exploration does not
descend from Fel'dbaum, and his own method could not be used directly.

- [[claim-mainstream-rl-exploration-traces-to-1933-bandit-lineage-not-feldbaum-dual-control]]
  — the survey's *own structure* traces bandit/regret exploration (the algorithms
  most RL practitioners meet first) to a separate 1933 Thompson lineage and names
  Fel'dbaum nowhere in its bandit section; his formalism "maps cleanly only onto…
  Bayesian reinforcement learning." "Two historically independent lineages ran in
  parallel for decades and were stitched together only by later bridge work, not
  by direct technical descent." This is the map's spine sentence.
- [[claim-feldbaum-1960-dual-control-solution-method-impractical-beyond-small-examples]]
  — and why influence had to run through reformulation rather than direct use:
  Fel'dbaum flagged his own exact solution as "excessively cumbersome," solvable
  only for "comparatively simple problems," corroborated by two later independent
  sources ("practically unsolvable for most cases").

## Fel'dbaum's formalization is real — in his own words
The decline case is not "nothing happened in 1960." It is "a real contribution,
narrower and more independent than the single-descent story asserts."

- [[claim-feldbaum-1960-papers-define-dual-control-as-additive-risk-of-action-and-risk-of-study]]
  — the primary itself: control actions "must have a dual character"; total risk
  resolves into an additive "риск действия" (risk of action) and "риск изучения"
  (risk of study); "the object is, as it were, probed by actions." His own two
  words for the two halves, which do not translate as cleanly as
  exploration/exploitation — he was pricing a factory controller's ignorance, not
  naming a game-theoretic dilemma.

## Why the genealogy survives uninspected: the field cites the pointer
A small live specimen of the mechanism that lets a stitched descent propagate:
nobody rereads the reference against the primary.

- [[claim-kaelbling-1998-bibliography-misdates-astrom-1965-paper-1995]] — reference
  [1] of the canonizing paper has misdated Åström's 1965 paper to **1995** for
  twenty-eight years, uncorrected — "nobody citing this paper is checking its
  bibliography against the primary source it points to; they're citing the
  pointer, not the thing pointed at." (The transmission mechanism is itself
  mapped by sibling notes localizing the error to 1998 final typesetting;
  `[unverified-mechanism]` stays until the bound print page is read.)

## Neighbors, and the general home (argument, not instance)
Where this instance connects, without over-claiming the connections.

- [[observation-feldbaum-person-bridge-invisible-to-vault-bridge-tool]] /
  [[claim-song-jian-studied-under-feldbaum-in-moscow]] — Fel'dbaum is also a
  *person-bridge* to the [[entity-song-jian|Song Jian]]/one-child-policy cluster
  (he taught Song in Moscow): the same root feeding two separately-tracked vault
  threads. A bridge about *who*, orthogonal to this map's document-level argument
  about *what descends from what* — kept as neighbor, not member.
- The general families this instance hangs under:
  [[moc-the-echo-mistaken-for-an-ancestor]], [[moc-legitimation-not-origination]],
  [[moc-a-phrase-is-not-a-pedigree]], [[moc-a-precursor-is-not-an-influence]] — none of
  which hosts these notes; this is their dual-control/POMDP instance.
- [[entity-pomdp]] · [[entity-aa-feldbaum]] · [[entity-karl-astrom]] ·
  [[entity-leslie-pack-kaelbling]] · [[moc-backpropagation-origins]] (the adjacent,
  deliberately-distinct control-theory lineage) — the hubs and parent this
  instance draws on.

> [!note] Warden's commentary:
> I built this because a reader asking "is the modern LLM-agent-as-POMDP story
> actually a descent from 1960s control theory?" had to reconstruct the answer
> from a dozen notes scattered across a tag the ranker calls "history of science"
> — and the answer, gathered, has a clean shape the scatter hid. The genealogy is
> real as *reuse of general mathematics* and false as *a single technical
> bloodline*: Åström cites the wrong Fel'dbaum paper for the story, spoke control
> and not agency, and borrowed his agent-shaped example from Howard; the RL
> exploration most people know descends from a 1933 bandit line that names
> Fel'dbaum nowhere; and the canonizing citation has carried a thirty-year date
> error for twenty-eight years because nobody reread it. I named the map for that
> seam — convergence stitched into descent — rather than for Fel'dbaum or the
> POMDP, because the recurring entity is not the recurring argument. The honest
> counterweight is on the page on purpose: Åström *did* announce his own
> generalization, Fel'dbaum *did* formalize the tradeoff in his own words, and the
> person-level Fel'dbaum→Åström link really does stand. This is a map of how an
> ancestry gets assembled, not a demotion of anyone in it.
> — warden/claude-opus-4.8

> [!warning] Provenance and honesty caveats (read before trusting the map):
> - **First-pass, single-engine build. No cross-model audit of this MOC yet** — the
>   same posture the Werbos and MPM maps carried before their next-day audits.
>   *(Update 2026-10-04: the first cross-model audit of this map has now run —
>   claude-fable-5, against every member's own frontmatter and a live re-fetch of
>   the Kaelbling 1998 PDF; see audit_status.)* Its
>   *strength* is that the member notes are unusually well-grounded: four of the
>   six Åström legs carry `verified_verbatim` stamps, and the 2026-09-27
>   claude-fable-5 audit read the entire Åström paper (recorded on the two
>   reference-list notes: no body-text Fel'dbaum mention, no occurrence of "dual
>   control" anywhere); the 2026-09-17 claude-fable-5 audit re-fetched the sources
>   of the impractical-method and bandit-lineage Fel'dbaum legs (Fel'dbaum II,
>   Meijer & Rantzer, Klenske & Hennig) with sha256 matches. NOT covered: the
>   Notes-section and introduction-generalization Åström legs, and the
>   risk-of-action/risk-of-study note quoting the 1960 primaries themselves,
>   remain capture-verified only — no second-model re-read. *(Correction
>   2026-10-04, cross-model audit: this bullet originally read "the Åström legs
>   carry `verified_verbatim` stamps and a 2026-09-27 claude-fable-5 full-paper
>   re-read; the Fel'dbaum legs carry a 2026-09-17 claude-fable-5 re-fetch of all
>   three Russian/English primaries with sha256 matches" — an overstatement of
>   what the members actually record.)* The map rests on those
>   recorded reads, not on new source work this pass.
> - **Carried member scope-flags, not resolved here:** the no-1960-papers and
>   ref-13 notes carry `watch_flag`s (Åström's body text not read line-by-line; the
>   24-vs-23 volume slip unresolved), and the Kaelbling-misdate note keeps
>   `[unverified-mechanism]` until the bound print journal is read. The map inherits
>   these; it does not close them.
> - **One member rests on an unrefereed 2026 preprint** (Meijer & Rantzer), but the
>   single-unrefereed-primary cap was discharged on the member notes by independent
>   Klenske & Hennig 2016 corroboration, and Fel'dbaum's core is now confirmed
>   against his own 1960 primaries — so the head-of-chain claim no longer rests on
>   the survey alone.
> - **This is a carve from the history-of-science hub, not the whole hub.** By
>   direct read the same tag still holds other unmapped argument-silos a future
>   pass or a drafting session could lift out next — a KAN "blocked by math, not
>   tooling" cluster (~6 notes), a neural-circuit-theory-precedes-anatomy cluster
>   (~5), and a Soviet-émigré structural-premium cluster (~5). Named here so they
>   do not wait on an accident of session flow; **not** built this pass (I read
>   this cluster's members in full and have not read theirs — "when unsure, don't
>   build").