Grounded-Dreamer: Robot Learning from World Model Synthetic Data

This paper presents Grounded-Dream, a framework that synthesizes physically grounded robot manipulation data using video world models and simulator physics, without requiring human teleoperation. Starting from real images, the method automatically reconstructs objects and camera geometry, aligns them in simulation, and generates physically valid trajectories. These are used to train video world models and inverse dynamics models, enabling the generation of realistic robot manipulation videos and actions from real images. Empirical results show that Grounded-Dreamer can synthesize physically grounded data and significantly improve policy learning without human demonstrations.