POMDP (partially observable Markov decision process)

A mathematical framework for choosing optimal actions when an agent cannot directly observe the true state of the system it is acting in — only noisy or incomplete signals about it, from which it must maintain a belief and act under that uncertainty. The formalism's mathematical root is Karl Åström's 1965 control-theory paper "Optimal Control of Markov Processes with Incomplete State Information"; it entered mainstream AI planning research as a named, citable object through Kaelbling, Littman & Cassandra's 1998 Artificial Intelligence paper, which cites Åström as its own reference [1].

First entered this vault on 2026-09-02, but load-bearing from the moment it arrived: it is the single mathematical object every claim-note in this cluster turns on, spanning a Cold War Swedish control paper, the 1998 paper that canonized it for AI, and 2026 research formalizing LLM-agent decision-making with the identical structure — the same acronym describing a factory process controller in 1965 and a language model's uncertainty about its own task environment sixty-one years later.

References

• claim-kaelbling-1998-cites-astrom-1965-as-pomdp-origin — Åström's 1965 origin, canonized by Kaelbling et al. 1998
• claim-kaelbling-1998-bibliography-misdates-astrom-1965-paper-1995 — a citation-transmission error on the same reference
• claim-2026-comap-paper-formalizes-llm-agent-as-pomdp — the term's 2026 use to formalize LLM-agent decision-making
• Capture: 10-inbox/raw/2026-09-02-hop-astrom-pomdp-llm-agents.md

Updates

• 2026-09-10: a direct read of Åström's founding 1965 text (previously known to the vault only through its citation history) shows the paper never uses the word "agent" and frames the underlying mathematics entirely in control-theory vocabulary (claim-astrom-1965-paper-uses-control-vocabulary-not-agent-language) — while its own introduction explicitly claims that mathematics generalizes beyond control systems (claim-astrom-1965-introduction-claims-generalization-beyond-control-systems). The "agent" framing that makes the structure legible to 2026 LLM-agent research was added later, by Kaelbling et al. in 1998 and again in 2026 — the mathematics anticipated its own generality; the vocabulary did not.
• 2026-09-15: a direct read of A.A. Fel'dbaum's own 1960 "Dual Control Theory" papers — five years before Åström's 1965 paper — shows Fel'dbaum's exact solution method already used the structural core of a POMDP: an augmented physical-plus-information state, a posterior belief updated from observations, and backward-induction dynamic programming over that augmented state. Fel'dbaum himself flagged the exact method as impractical beyond small examples (claim-feldbaum-1960-dual-control-solution-method-impractical-beyond-small-examples), and neither he nor Åström's own 1965 text draw a direct line between the two — the structural resemblance is this vault's own observation, not a citation either author makes.