WorldSketch: Sketching World Models for Zero-Shot Mobile Manipulation
WorldSketch: Sketching World Models for Zero-Shot Mobile Manipulation
This research paper proposes WorldSketch, a new paradigm for agentic robotics where a vision-language model constructs a minimal kinematic world model by actively querying perception tools. The system captures task-relevant objects, interaction geometry, and possible motions, which are then used by a geometric planner to optimize contact and whole-body motion. The world model is self-improving through execution feedback that updates uncertain hypotheses. Demonstrations on RoboCasa, LIBERO-PRO, and R2R-CE navigation show zero-shot success rates of 91%, 97%, and 76% respectively, outperforming baselines while using significantly fewer VLM input tokens.
The paper introduces WorldSketch, a paradigm where a vision-language model constructs a minimal kinematic world model via perception queries rather than directly generating actions, enabling zero-shot performance improvements on RoboCasa (91% success rate), LIBERO-PRO (97%), and R2R-CE navigation (76%) compared to baselines.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
Reported success rate
91%
Reported zero-shot result; the excerpt does not specify an evaluation protocol.
Reported trials
View original evidence
l is self-improving: execution feedback verifies outcomes, updates uncertain hypotheses, and enables the agent to revise the model when its predictions are contradicted. We demonstrate the generality of this paradigm across RoboCasa, LIBERO-PRO, and R2R-CE navigation, achieving zero-shot success rates of 91%, 97%, and 76%, respectively. On RoboCasa, WorldSketch outperforms the next-best zero-shot agent baseline by 58
Open source S4reduction in VLM input tokens for navigation
6
View original evidence
percentage points while using 10 times fewer VLM input tokens; on LIBERO-PRO, it exceeds CaP-X by 82 percentage points. In navigation, it matches the success rate of step-by-step VLM agents while using 6 times fewer input tokens.
Open source S5
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
What remains unknown
Not established in the collected evidence: Environment, Control, Data origin.
Reported performance applies to the described task. It does not establish general autonomy or deployment readiness.
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
arXiv:2611.00513v1 Announce Type: new Abstract: Most agentic robotic systems use vision-language models (VLMs) either to orchestrate predefined skills or to directly generate actions, constraints, and control programs. The former is limited by the available skill library, while the latter requires VLMs to reason about precise geometry and contact. In this paper, we propose WorldSketch, a new paradigm for agentic robo
Open source S2
l is self-improving: execution feedback verifies outcomes, updates uncertain hypotheses, and enables the agent to revise the model when its predictions are contradicted. We demonstrate the generality of this paradigm across RoboCasa, LIBERO-PRO, and R2R-CE navigation, achieving zero-shot success rates of 91%, 97%, and 76%, respectively. On RoboCasa, WorldSketch outperforms the next-best zero-shot agent baseline by 58
Open source S4
percentage points while using 10 times fewer VLM input tokens; on LIBERO-PRO, it exceeds CaP-X by 82 percentage points. In navigation, it matches the success rate of step-by-step VLM agents while using 6 times fewer input tokens.
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org