WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RL

This research introduces WoVR, a reinforcement learning framework designed to train Vision-Language-Action (VLA) policies using imperfect world models. The authors address the issue of hallucination and error accumulation in simulated rollouts by regulating how RL interacts with these models. Key techniques include a controllable action-conditioned video world model, Keyframe-Initialized Rollouts to reduce effective error depth, and World Model-Policy co-evolution to maintain alignment. Experiments show that WoVR enables stable long-horizon imagined rollouts and achieves superior performance on the LIBERO benchmark while providing consistent gains across multiple robotic platforms, demonstrating that controlled world models can serve as practical simulators for RL.