RoboARCade: Benchmarking Agentic Robot Control
RoboARCade: Benchmarking Agentic Robot Control
This research introduces RoboARCade, a benchmark designed to evaluate agentic robot control across long-horizon manipulation tasks in three different embodiments.
The paper introduces RoboARCade, a benchmark for agentic robot control that evaluates long-horizon manipulation tasks across three embodiments using the MuJoCo simulation backend. It reports specific performance metrics for agents like GPT-6 Astra and Claude Fable 5.1, noting failure modes related to manipulation strategy precision and self-reported success rates.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- Environment
- SimulationOpen source S3
- paper: not_reported
- code: Reported pending Open source S5
- weights: not_reported
- dataset: not_reported
- code: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
nchmarks, which we make harder by widening the range of initial conditions. RoboARCade is easy to use and extend: all tasks run on a single simulation backend, MuJoCo, each task is a self-contained data pack, and agents connect through interchangeable interfaces, so new tasks, agents, and interfaces can be added independently. The benchmark separates leading agents: GPT-6 Astra achieves 49.8% compared with 43.6% for
Open source S3
arXiv:2611.08209v1 Announce Type: new Abstract: Improvements in multi-modal capabilities have enabled agents to act directly as robot policies, a setting we refer to as agentic robot control (ARC). We introduce RoboARCade, a benchmark of long-horizon manipulation tasks across three embodiments designed to test in-context adaptation, memory, and reasoning for precise control. The tasks build on existing open-source be
Open source S2
e without human input: one successful demonstration in context raises Astra's score from 53.5% to 73.0% on a five-task subset, and a prompt the agent writes and revises itself reaches 72.0%. Code, videos and full traces are available at roboarcade.org.
Open source S5
Claude Fable 5.1, leaving substantial room for improvement. Comparing two interfaces, we find that a richer visual interface adds only 1.5 to 6.8 points at 1.7 to 2.3 times the cost. Failures stem mostly from choosing a poor manipulation strategy or executing a good one imprecisely, rarely from misunderstanding the scene, and agents incorrectly claim success on 24% to 39% of attempted subtasks. Agents can also improv
Open source S4
Source：arXiv Robotics — research abstracts · arxiv.org