Closing the Dynamics Gap through Self-Directed Robotic Interaction
Closing the Dynamics Gap through Self-Directed Robotic Interaction
This research presents CUE, an agentic system designed to bridge the 'dynamics gap' in Real2Sim2Real workflows. By interacting with the environment itself rather than relying on resets, CUE reconstructs scenes in simulation and identifies physical parameters such as friction and restitution that are unobservable from camera data alone. The system autonomously builds perception and targets interactions where its simulation is least accurate. In evaluations on billiards, CUE reduced held-out ball-position error by 72% relative to the strongest baseline across 25 hidden-physics conditions in simulation. The authors further demonstrate deployment of the system on a real pool table.
The paper introduces CUE, an agentic system that closes the 'dynamics gap' in Real2Sim2Real workflows by self-directing interactions to identify physical parameters like friction and restitution, achieving a 72% reduction in ball-position error on billiards simulations across 25 hidden-physics conditions before deployment on a real pool table.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- Environment
- SimulationOpen source S2
Reported numbers
number of hidden-physics conditions
25
Reported trials
View original evidence
agent builds its own perception, identifies physical parameters, and targets interactions where its simulation is least accurate. On billiards, CUE reduces held-out ball-position error by 72% relative to the strongest baseline across 25 hidden-physics conditions in simulation, and we further deploy it on a real pool table. More details can be found on the project website: https://cue-dynamics.github.io/
Open source S3
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
arXiv:2611.02691v1 Announce Type: new Abstract: Agentic Real2Sim2Real systems can reconstruct a scene in simulation from camera observations, but contact-rich dynamics such as friction and restitution cannot be observed without interaction. We call this the dynamics gap and present CUE, an agentic system that closes it by interacting with the environment itself. Given only a high-level instruction and no resets, the
Open source S2
agent builds its own perception, identifies physical parameters, and targets interactions where its simulation is least accurate. On billiards, CUE reduces held-out ball-position error by 72% relative to the strongest baseline across 25 hidden-physics conditions in simulation, and we further deploy it on a real pool table. More details can be found on the project website: https://cue-dynamics.github.io/
Open source S3
Source：arXiv Robotics — research abstracts · arxiv.org