Pretraining Knows What, Simulation Teaches How: Compositional Generalization in a Sim-Trained Robot Agent
Pretraining Knows What, Simulation Teaches How: Compositional Generalization in a Sim-Trained Robot Agent
This research introduces ZAP, a robot policy that uses a pretrained 27B parameter vision-language model to control a robot via a simple tool harness. Trained on simulated tasks, ZAP achieves 96.7% success on real-world tasks, outperforming existing models like MolmoAct2 and GPT-6 Astra in efficiency and cost. It also generalizes to new objects, goals, and strategies, and transfers to an unseen embodiment with 55.9% performance on RoboLab-120.
The paper introduces ZAP, a robot policy that leverages a pretrained vision-language model to achieve high success rates in real-world tasks without real-world training data. It demonstrates generalization to novel objects and tasks, and cross-embodiment transfer to a Franka arm.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
performance comparison
3
View original evidence
standing and reasoning, followed by supervised fine-tuning and GRPO on simple simulated tasks to teach it how to move the robot. Without any real-world robot data during training, ZAP reaches 96.7% success on real-world tasks, matching GPT-6 Astra in our harness and outperforming the MolmoAct2 VLA (33.3%), while running 3x faster and 17x cheaper than GPT-6 Astra. We demonstrate that ZAP also generalizes to completely
Open source S3performance comparison
17
View original evidence
standing and reasoning, followed by supervised fine-tuning and GRPO on simple simulated tasks to teach it how to move the robot. Without any real-world robot data during training, ZAP reaches 96.7% success on real-world tasks, matching GPT-6 Astra in our harness and outperforming the MolmoAct2 VLA (33.3%), while running 3x faster and 17x cheaper than GPT-6 Astra. We demonstrate that ZAP also generalizes to completely
Open source S3
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
standing and reasoning, followed by supervised fine-tuning and GRPO on simple simulated tasks to teach it how to move the robot. Without any real-world robot data during training, ZAP reaches 96.7% success on real-world tasks, matching GPT-6 Astra in our harness and outperforming the MolmoAct2 VLA (33.3%), while running 3x faster and 17x cheaper than GPT-6 Astra. We demonstrate that ZAP also generalizes to completely
Open source S3
new objects, goals, strategies, and language by seating a test tube in a rack (and choosing a different hole in the rack when prompted), handing a test tube between arms mid-air, drawing an X on a tic-tac-toe board, and playing chess. These goals and types of manipulation don't appear in our training data. Furthermore, ZAP also transfers to an unseen embodiment. On a single-arm Franka, ZAP reaches 55.9% on RoboLab-1
Open source S4
Source：arXiv Robotics — research abstracts · arxiv.org