Right Ranking, Wrong Scale: Reproducing the Robotic World Model's Dynamics Model and Testing the Uncertainty of Its Follow-up
Right Ranking, Wrong Scale: Reproducing the Robotic World Model's Dynamics Model and Testing the Uncertainty of Its Follow-up
This research abstract describes an independent reimplementation of the dynamics model for the Robotic World Model (RWM) to evaluate the uncertainty estimates used by its follow-up system, RWM-U. The authors find that while ensemble disagreement effectively ranks rollout errors, the reported spread significantly understates actual error, with one-sigma coverage at 4.6% versus the expected 68% for a calibrated Gaussian. They demonstrate that rolling ensemble members on their own predictions improves coverage to 33%, and that horizon-specific rescaling restores appropriate one-sigma coverage under leave-one-episode-out evaluation. The study identifies defects in the released evaluation pipeline, noting limitations to one robot, one gait, and one dataset, and confirms that no policy was trained as part of this work.
The study independently reimplements the dynamics model of the Robotic World Model (RWM) and evaluates the uncertainty calibration of its follow-up, RWM-U, finding that ensemble disagreement ranks errors well but spread substantially understates error without horizon-specific rescaling.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
gait types
1
View original evidence
n sampled predictions drives each member's variance toward zero. A horizon-specific rescaling restores one-sigma coverage under leave-one-episode-out evaluation, but not two-sigma coverage. We also identify two defects in the released evaluation pipeline. The study covers one robot, one gait and one dataset; it trains no policy.
Open source S4datasets
1
View original evidence
n sampled predictions drives each member's variance toward zero. A horizon-specific rescaling restores one-sigma coverage under leave-one-episode-out evaluation, but not two-sigma coverage. We also identify two defects in the released evaluation pipeline. The study covers one robot, one gait and one dataset; it trains no policy.
Open source S4Spearman rho
0.9
View original evidence
odes, but loses on short one-step windows. RWM-U's ensemble disagreement ranks rollout errors better than forecast step (Spearman rho = 0.90 at 100 steps), yet its spread substantially understates error: one-sigma coverage on the released checkpoint is 4.6% at 100 steps, versus 68% for a calibrated Gaussian. Rolling ensemble members on their own predictions raises coverage to 33%. The implemented squared-error loss o
Open source S3training horizon steps
100
View original evidence
odes, but loses on short one-step windows. RWM-U's ensemble disagreement ranks rollout errors better than forecast step (Spearman rho = 0.90 at 100 steps), yet its spread substantially understates error: one-sigma coverage on the released checkpoint is 4.6% at 100 steps, versus 68% for a calibrated Gaussian. Rolling ensemble members on their own predictions raises coverage to 33%. The implemented squared-error loss o
Open source S3
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
n sampled predictions drives each member's variance toward zero. A horizon-specific rescaling restores one-sigma coverage under leave-one-episode-out evaluation, but not two-sigma coverage. We also identify two defects in the released evaluation pipeline. The study covers one robot, one gait and one dataset; it trains no policy.
Open source S4
arXiv:2611.02699v1 Announce Type: new Abstract: We reimplement the dynamics model of the Robotic World Model (RWM) and test the uncertainty estimate that its follow-up, RWM-U, uses to penalize reward in offline policy optimization. Our implementation matches the released model's gradients. On held-out data, training on the model's own rollouts beats one-step training at the preregistered long horizon for all ten epis
Open source S2
odes, but loses on short one-step windows. RWM-U's ensemble disagreement ranks rollout errors better than forecast step (Spearman rho = 0.90 at 100 steps), yet its spread substantially understates error: one-sigma coverage on the released checkpoint is 4.6% at 100 steps, versus 68% for a calibrated Gaussian. Rolling ensemble members on their own predictions raises coverage to 33%. The implemented squared-error loss o
Open source S3
Source：arXiv Robotics — research abstracts · arxiv.org