Future Representation Distillation for Imagination-Free World Action Models
Future Representation Distillation for Imagination-Free World Action Models
This paper introduces DreamFree, a World Action Model (WAM) that repurposes a pretrained video diffusion model as a predictive visual encoder through representation distillation. By predicting the frozen model's representations from current observations alone, DreamFree enables action prediction without generating future videos. The method achieves real-time closed-loop control at 11Hz with a 14B video diffusion backbone and outperforms state-of-the-art VLAs and WAMs across real-world and simulated tasks. It also supports learning new robot tasks from video demonstrations of different embodiments without requiring action annotations for those new tasks.
The authors propose DreamFree, a World Action Model that distills visual representations from a frozen video diffusion model to enable real-time robot control without iterative video generation, achieving higher success rates than state-of-the-art VLAs and WAMs in both simulated and real-world tasks while supporting cross-embodiment learning.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
control frequency
11
View original evidence
relevant information from the visual encoder. Across real-world and simulated tasks, DreamFree achieves higher success rates than state-of-the-art VLAs and WAMs, while performing real-time closed-loop control at 11Hz with a 14B video diffusion backbone. Experiments show that DreamFree can translate stronger video priors and additional video data into better robot control and enable learning new robot tasks from video
Open source S4
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
relevant information from the visual encoder. Across real-world and simulated tasks, DreamFree achieves higher success rates than state-of-the-art VLAs and WAMs, while performing real-time closed-loop control at 11Hz with a 14B video diffusion backbone. Experiments show that DreamFree can translate stronger video priors and additional video data into better robot control and enable learning new robot tasks from video
Open source S4
demonstrations, including videos from a different embodiment, without action annotation for the new tasks.
Open source S5
Future Representation Distillation for Imagination-Free World Action Models
Open source S1
Source：arXiv Robotics — research abstracts · arxiv.org