Seeing and Imagining as One: Unified Video Generation and 4D Perception for Embodied World Modeling
Overview
This research presents GenSight, a unified embodied world model that integrates future video generation with dense perception of geometric, semantic, and motion structures into a single pretrained video diffusion transformer. By using a shared DiT backbone and lightweight task modulation, GenSight predicts all modalities within the same architecture and supports both multi-step future video generation with aligned dense predictions and single-step dense perception from observed videos. Trained on embodied manipulation data spanning multiple robot embodiments and human-hand interactions, GenSight demonstrates strong performance in both tasks, highlighting the effectiveness of unifying seeing and imagining for embodied world modeling.
Generated from attributed reports · Updated 2 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
No current evidence-backed claims. Missing information remains not reported.
Evidence dependency changes (1)
- 2026-11-02T20:03:20.617Z · evidence updated · source revision 2. Evidence extraction was updated; current source attributions are shown above.
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalSeeing and Imagining as One: Unified Video Generation and 4D Perception for Embodied World Modeling
This research presents GenSight, a unified embodied world model that integrates future video generation with dense perception of geometric, semantic, and motion structures into a single pretrained video diffusion transformer. By using a shared DiT backbone and lightweight task modulation, GenSight predicts all modalities within the same architecture and supports both multi-step future video generation with aligned dense predictions and single-step dense perception from observed videos. Trained on embodied manipulation data spanning multiple robot embodiments and human-hand interactions, GenSight demonstrates strong performance in both tasks, highlighting the effectiveness of unifying seeing and imagining for embodied world modeling.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).