Robot-Factored World Models via Robot Rendering
Overview
This paper introduces Robot-Factored World Models (RoFacto), which decouple action realization and robot rendering from scene-response prediction. By rolling command sequences through the robot's own controller and kinematics to generate nominal trajectories, and rendering these trajectories using the robot URDF, RoFacto factors out geometry, kinematics, and appearance into explicit rendered geometry. The model uses paired end-effector and scene depth for geometric cues regarding contact and occlusion, while combining camera-aware static RGB context with rendered robot geometry to form a shared visual interface across viewpoints and embodiments. Experiments demonstrate that this rendered interface outperforms vector-conditioned baselines, supports transfer to unseen robot embodiments, and enables the generation of robot manipulation videos from retargeted human demonstrations.
Generated from attributed reports · Updated 1 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
No current evidence-backed claims. Missing information remains not reported.
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalRobot-Factored World Models via Robot Rendering
This paper introduces Robot-Factored World Models (RoFacto), which decouple action realization and robot rendering from scene-response prediction. By rolling command sequences through the robot's own controller and kinematics to generate nominal trajectories, and rendering these trajectories using the robot URDF, RoFacto factors out geometry, kinematics, and appearance into explicit rendered geometry. The model uses paired end-effector and scene depth for geometric cues regarding contact and occlusion, while combining camera-aware static RGB context with rendered robot geometry to form a shared visual interface across viewpoints and embodiments. Experiments demonstrate that this rendered interface outperforms vector-conditioned baselines, supports transfer to unseen robot embodiments, and enables the generation of robot manipulation videos from retargeted human demonstrations.
Event coverage history
There is not enough continuous observation data to show a trend.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).