---
title: "Potential-based reward shaping"
type: "entity"
entity_kind: "concept"
status: "hub"
canonical_name: "potential-based reward shaping"
aliases: ["potential-based reward shaping","Potential-based reward shaping","Text2potential (CoDA's instantiation)"]
first_seen: "2026-10-19T00:00:00.000Z"
writer_model: "claude-sonnet-5"
connects_to: ["[[entity-stuart-russell|Stuart Russell]]","reinforcement learning","CoDA (co-evolution of data engines and action policies)","dense reward design","optimal-policy invariance"]
seek_code_commit: "d40ee39"
---


A reinforcement-learning technique, formalized by Andrew Y. Ng, Daishi Harada, and [[entity-stuart-russell|Stuart Russell]] (ICML 1999), for adding a dense auxiliary reward signal — a difference of a potential function over states — to a sparse task reward, under a theorem guaranteeing that a fixed, bounded potential with appropriate terminal-state handling does not change which policy is optimal under the original reward.

Matters to this vault as the theoretical basis CoDA (arXiv:2610.22362, 2026) invokes for "Text2potential," its mechanism for converting a vision-language critic's rollout-failure diagnoses into staged potentials for action-chunk value learning in robot manipulation — the vault's first encounter with the technique, in a domain (vision-language-action robot learning) the 1999 theorem's authors did not anticipate ([[claim-coda-shared-memory-couples-data-engine-policy-optimality-preserving-reward-shaping]]). Distinct from free-form LLM-synthesized dense rewards (e.g. Text2Reward), which lack this invariance guarantee.

## References
- [[claim-coda-shared-memory-couples-data-engine-policy-optimality-preserving-reward-shaping]]
- Captures: 10-inbox/raw/2026-10-18-outside-arxiv261022362-coda-co-evolution-of-data-engines.md