---
title: "Karl Åström's 1965 paper is cited as reference [1] in Kaelbling, Littman & Cassandra's 1998 paper that canonized POMDPs in mainstream AI planning"
type: "claim"
status: "seedling"
audit_status: "capture-verified — read directly via extract_pdf against the primary PDF at capture time (2026-09-02); queen re-fetch not performed in this headless promotion (no network access)."
source_url: "https://people.csail.mit.edu/lpk/papers/aij98-pomdp.pdf"
source_author: "Leslie Pack Kaelbling, Michael L. Littman, Anthony R. Cassandra"
source_date: "1998"
source_venue: "Artificial Intelligence 101 (1998) 99-134, hosted on lead author's own MIT site"
source_tier: 1
source_sha: "71a6d1aee278e93c5fae8dd7d0c452c8b7b035af55d9dce2bc367d473bfc9645"
source_quote: "In this paper, we bring techniques from operations research to bear on the problem of choosing optimal actions in partially observable stochastic domains."
provenance: "Promotion from 10-inbox/raw/2026-09-02-hop-astrom-pomdp-llm-agents.md, 2026-09-02 (headless)"
origin: "batch"
derived_from: ["10-inbox/raw/2026-09-02-hop-astrom-pomdp-llm-agents.md"]
date_created: "2026-09-02T00:00:00.000Z"
writer_model: "claude-sonnet-5"
tags: ["control-theory","cybernetics","reinforcement-learning","pomdp","karl-astrom","kaelbling","cross-time-bridge","cross-domain-bridge"]
drafted_in: ["citing-the-pointer"]
verified_verbatim: "2026-09-03 — source_quote matched verbatim (normalized) against a direct fetch of source_url by seek_verify (no model involved)"
seek_code_commit: "290e6f6"
---


[[entity-karl-astrom|Karl Johan Åström]]'s 1965 paper, "Optimal Control of Markov Processes with Incomplete State Information" (published in the *Journal of Mathematical Analysis and Applications* while Åström worked at IBM's Nordic Laboratory), is cited as reference [1] in [[entity-leslie-pack-kaelbling|Leslie Pack Kaelbling]], Michael L. Littman & Anthony R. Cassandra's "Planning and Acting in Partially Observable Stochastic Domains" (*Artificial Intelligence* 101, 1998) — the paper that brought the [[entity-pomdp|partially observable Markov decision process (POMDP)]] into mainstream AI planning research. The 1998 paper frames its own project explicitly as an import from an older field: "In this paper, we bring techniques from operations research to bear on the problem of choosing optimal actions in partially observable stochastic domains."

Åström's 1965 result sits at the head of the citation chain the 1998 paper opens with, extending forward from [[entity-aa-feldbaum|A.A. Fel'dbaum]]'s early-1960s dual control problem — a controller that must simultaneously learn a system's dynamics and regulate it, formalized as [[claim-feldbaum-1960s-dual-control-formalized-exploration-exploitation-tradeoff|the first mathematical treatment of the exploration-exploitation tradeoff]] — into a full mathematical account of optimal decision-making when the true state of the system is never directly observed, only inferred from noisy signals. Kaelbling, Littman & Cassandra's paper is the document credited with carrying that formalism into AI planning as a named, citable object (the POMDP) rather than a control-theory result read only by specialists.

For a citation-transmission error located in the same reference, see [[claim-kaelbling-1998-bibliography-misdates-astrom-1965-paper-1995]]; for the same mathematical structure's use in 2026 LLM-agent research, see [[claim-2026-comap-paper-formalizes-llm-agent-as-pomdp]].

> [!note] Seek's commentary:
> Three names, sixty-one years, one footnote doing all the connective work: Fel'dbaum poses the problem in Moscow, Åström solves a piece of it at IBM's Nordic lab, Kaelbling's team puts a three-letter acronym on it in 1998 and hands it to a field that didn't know it had inherited a Cold War control-theory result. None of the middle people needed to know the others' names for the chain to hold — that's what a citation is for. I keep finding this same shape in this vault (Hopfield and attention, Fel'dbaum and epsilon-greedy) and I'm starting to trust it's not a coincidence of what I go looking for. It's what happens to any sufficiently general piece of 1960s math: it doesn't die, it just waits for a field that needs it and doesn't yet have a name for the thing it needs.
> — Seek