SimForcing: Distilling Simulation Motion Priors into Real-Domain Robot World Models
SimForcing: Distilling Simulation Motion Priors into Real-Domain Robot World Models
This research introduces SimForcing, a simulation-guided framework that transfers motion knowledge from a simulation teacher to real-domain robot world models using latent-motion distillation and multi-block simulation conditioning. The method balances internalized motion priors with simulation guidance via classifier-free guidance, enabling the generation of realistic videos without external embodied pretraining. Evaluated on Bridge and InternData-A1 datasets, SimForcing achieves superior performance in PSNR, SSIM, LPIPS, and FVD metrics compared to other methods. Additionally, initializing a vision-language-action model with the trained world model improves success rates in the LIBERO benchmark, demonstrating its utility for downstream policy learning tasks.
The paper proposes SimForcing, a framework that distills motion priors from simulation to improve real-domain robot world model generation and downstream policy learning, evaluated on Bridge and InternData-A1 datasets.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
- weights: not_reported
- dataset: not_reported
- code: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
arXiv:2610.06598v2 Announce Type: replace Abstract: Action-conditioned robot world models must respond precisely to robot trajectories while preserving realistic visual dynamics, yet learning both from heterogeneous robot videos remains challenging. Simulation offers structured motion supervision, but appearance differences hinder direct transfer, and inaccurate simulation predictions can misguide real-video generati
Open source S2
on. We present SimForcing, a simulation-guided framework that uses simulation both as a source of transferable motion knowledge and as a controllable reference for prediction. First, we transfer motion knowledge from a simulation teacher through latent-motion distillation, aligning temporal changes in latent space to internalize motion priors while mitigating the influence of appearance differences. Second, we introd
Open source S3
uce multi-block simulation conditioning with condition dropout to exploit predicted simulation trajectories without relying excessively on their accuracy. Our simulation-conditioning classifier-free guidance scheme unifies these two ideas by balancing predictions based on internalized motion knowledge with those additionally guided by simulation latents. The jointly trained student generates both simulation condition
Open source S4
s and real-domain videos, requiring no additional world model at inference. On Bridge, SimForcing achieves the best PSNR, SSIM, LPIPS, and FVD among the compared methods without external embodied pretraining. Evaluation on InternData-A1 further supports its applicability across robot datasets. Moreover, using our trained world model to initialize a vision-language-action model improves LIBERO success, suggesting its
Open source S5
utility for downstream policy learning. https://github.com/Wang-Xiaodong1899/SimForcing
Open source S6
SimForcing: Distilling Simulation Motion Priors into Real-Domain Robot World Models
Open source S1
Source：arXiv Robotics — research abstracts · arxiv.org