Mechanisms of Misgeneralization in Physical Sequence Modeling
Mechanisms of Misgeneralization in Physical Sequence Modeling
This research paper investigates 'physical misgeneralization' in generative sequence models used for robotics and mechanical simulations. The authors demonstrate that while individual generated trajectories may appear plausible, the aggregate distribution over physical quantities (e.g., travel distance, mechanical energy) can deviate from intended constraints due to propagated local errors. The study formalizes this problem, introduces a 'data deviation kernel' to estimate these errors and predict distribution shifts, and proposes a mitigation strategy involving coordinate system transformation to reduce quantity drift during generation.
This paper identifies 'physical misgeneralization' in generative sequence models where local errors propagate to distort aggregate distributions over physical quantities like travel distance or energy, proposing a data deviation kernel for prediction and coordinate system transformation as a mitigation.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
and formalize the problem, develop and validate an account of its mechanism, and use that account to compare and design mitigations. Using controlled synthetic tasks, we show that physical misgeneralization arises when local errors typical of the model class propagate through the physical measurement to shift the recovered distribution. We estimate these errors with a data deviation kernel, and we use it to predict
Open source S4
arXiv:2605.20299v2 Announce Type: replace-cross Abstract: Generative sequence models are often trained to plan motion in physical domains, from robotics to mechanical simulations. When constructing a dataset to train such a model, engineers may curate demonstrations to specify how trajectories should be distributed over a physical quantity like travel distance or mechanical energy. For example, a roboticist building
Open source S2
a maze navigation agent might choose demonstrations whose travel distances cover a fixed range uniformly, hoping to constrain the agent's expected power usage. We find that standard deep learning can violate this intent: each generated trajectory can seem plausible on its own, but the aggregate distribution over the physical quantity is wrong. We call this failure physical misgeneralization. In this work, we identify
Open source S3
which physical quantity values gain or lose mass under real diffusion sequence models trained on both toy and applied trajectory generation tasks. Finally, our mechanistic interpretation helps identify which mitigations are structurally promising. We first compare interventions that modify the dataset and generative interface, then show that the mechanistic understanding we develop throughout this paper suggests a no
Open source S5
vel way to construct data representations informed by the kernel. We show that our proposed mitigation, which transforms the coordinate system under which the generator learns and samples, effectively reduces quantity drift.
Open source S6
Source：arXiv Robotics — research abstracts · arxiv.org