Library of Hands: Fix the Hands, Not the World Model
Library of Hands: Fix the Hands, Not the World Model
This research presents Library of Hands (LoH), a training-free approach that corrects command-to-action mappings without retraining the world model. By identifying actions that best explain changes between camera images, LoH stores identified hands to recognize returning setups rather than re-learn them. In closed-loop tests on three simulated tasks, LoH more than doubles success rates from 28.6% to 62.3%, with minimal performance loss when conditions remain unchanged. The method works unchanged with two other world models and on two of three models pretrained on real robot data, with gains increasing as reading quality improves.
The paper introduces Library of Hands (LoH), a training-free method that corrects command-to-action mappings without retraining the world model, achieving a closed-loop success rate increase from 28.6% to 62.3% in simulated tasks.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
arXiv:2611.12678v1 Announce Type: new Abstract: A world model predicts how a scene changes after an action, and a planner uses it to search for actions that reach a goal. If the camera moves or the controller changes, the same command moves the robot differently: when its commands are turned by 90 degrees, a frozen planner with the LeWM world model that succeeds in 77.5% of episodes succeeds in only 7.8%. Current fix
Open source S2
every hand it identifies, so a returning set-up is recognised rather than re-learnt. In closed-loop tests, LoH more than doubles success on three simulated tasks, from 28.6% to 62.3%, and loses only 0.6 percentage points when nothing changes. It also works, unchanged, with two other world models and on two of three models pretrained on real robot data, and its gain grows with the quality of its readings.
Open source S4
es retrain the model, need training data, or forget a set-up when it returns. We propose Library of Hands (LoH), a training-free method that keeps the world model frozen and corrects only the map from commands to applied actions, which we call the hand. LoH reads the hand through the frozen model, with no extra sensor: it finds the action that best explains the change between two camera images (a reading). It stores
Open source S3
Source：arXiv Robotics — research abstracts · arxiv.org