CoDA couples a generative-simulation data engine and a robot policy through shared memory, compiling a vision-language critic's failure diagnoses into regeneration programs and optimality-preserving dense rewards

roboticsvision-language-actiongenerative-simulationreinforcement-learningreward-shapingmemory-architecturearxivcs-ro

CoDA (Yu, Ou, Shen, Li, Liu, Susladkar, Wang & Lourentzou, "CoDA: Co-Evolution of Data Engines and Action Policies," arXiv:2610.22362, 2026) couples a generative-simulation data engine with a robot action policy through a shared heterogeneous memory. The data engine is a tool-augmented coding agent that writes executable scene programs against a simulator SDK; an action policy collects rollouts in the resulting environments; and a vision-language critic diagnoses failures by identifying the responsible scene factor, trajectory phase, and supporting rollout evidence. The memory is organized into episodic (individual records), semantic (recurring difficulty modes), and procedural (compiled programs) tiers — a shape the paper takes from the language-agent memory literature rather than inventing, crediting Sumers, Yao, Narasimhan & Griffiths, "Cognitive architectures for language agents" (arXiv:2309.02427, 2023), its reference [30]. The memory turns each validated diagnosis into two paired outputs: a generation program that recreates the diagnosed difficulty for further training, and a training program the authors call "Text2potential," which converts the same diagnosis into a staged potential-based reward shaping function over simulator states — giving a value critic dense supervision over action chunks instead of sparse task reward alone.

The paper states this added signal preserves the policy that is optimal under the original sparse task objective, invoking the classical potential-based-shaping invariance result of Andrew Ng, Daishi Harada, and Stuart Russell (ICML 1999, reference [20]) as its theoretical basis: "for a fixed, bounded potential with appropriate terminal-state handling, potential-based shaping preserves the policies that maximize the original task return." The nearest named precedent for the co-evolution framing itself is POET (Wang, Lehman, Clune & Stanley, 2019), which the paper's related-work section describes in a single clause — "POET [35] coevolves agents and parameterized environments" — as one entry in a list of curriculum-learning methods. The paper claims no priority for POET and does not present its own loop as descending from it; reading POET as the ancestor of CoDA's title is this vault's inference, not the paper's statement.