RoboWits: Unexpected Challenges for Robotic Creative Problem Solving
RoboWits: Unexpected Challenges for Robotic Creative Problem Solving
This paper introduces RoboWits, a bi-manual robotic benchmark designed to evaluate cognitive reasoning, creative tool use, and robustness to unexpected conditions.
The authors introduce RoboWits, a benchmark using an automated multi-agent pipeline to generate 30 seed tasks and 200 mutated scenarios evaluating cognitive reasoning and robustness in bi-manual robots.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
seed tasks curated
30
View original evidence
eline, we curated 30 diverse seed tasks and 200 tasks with mutations and graded difficulty across geometry, material, and assembly-based reasoning. We benchmark popular robot policies, pre-trained VLAs, a zero-shot policy built on a frontier VLM, and oracle-state planners. Our results reveal a significant performance gap: while pre-trained VLAs exhibit preliminary success on seed tasks after fine-tuning, they struggl
Open source S4mutated tasks curated
200
View original evidence
eline, we curated 30 diverse seed tasks and 200 tasks with mutations and graded difficulty across geometry, material, and assembly-based reasoning. We benchmark popular robot policies, pre-trained VLAs, a zero-shot policy built on a frontier VLM, and oracle-state planners. Our results reveal a significant performance gap: while pre-trained VLAs exhibit preliminary success on seed tasks after fine-tuning, they struggl
Open source S4seed tasks with mutations
200
View original evidence
eline, we curated 30 diverse seed tasks and 200 tasks with mutations and graded difficulty across geometry, material, and assembly-based reasoning. We benchmark popular robot policies, pre-trained VLAs, a zero-shot policy built on a frontier VLM, and oracle-state planners. Our results reveal a significant performance gap: while pre-trained VLAs exhibit preliminary success on seed tasks after fine-tuning, they struggl
Open source S4
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
eline, we curated 30 diverse seed tasks and 200 tasks with mutations and graded difficulty across geometry, material, and assembly-based reasoning. We benchmark popular robot policies, pre-trained VLAs, a zero-shot policy built on a frontier VLM, and oracle-state planners. Our results reveal a significant performance gap: while pre-trained VLAs exhibit preliminary success on seed tasks after fine-tuning, they struggl
Open source S4
e to perform on mutated tasks, implying their brittleness in manipulation tasks requiring reasoning, strategy adaptation, and robustness to deceptive or constrained environments. A zero-shot policy built on GPT-6 Astra outperforms all fine-tuned VLAs, yet still loses about a fifth of its success on mutated tasks, showing that stronger foundation models alone do not close the robustness gap. See our project page https
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org