A General Agent That Can Read Images and Write Code: Can It Perform Robot Operations? CodeActionBench Released
会看图、会写代码的通用 Agent，能完成机器人操作吗？CodeActionBench 发布
CodeActionBench is a study that evaluates the performance of general multimodal agents in robot operation tasks, testing the strategy generation and execution capabilities of multiple models through a dual-arm simulation environment. The study highlights Astra's advantages in task success rate and code error rate, and analyzes differences in task coverage, execution trajectories, and completion judgment across different configurations.
CodeActionBench is a study focused on evaluating the performance of general multimodal agents in robot operation tasks, testing the strategy generation and execution capabilities of multiple models through a dual-arm simulation environment. The study emphasizes the models' comprehensive performance in visual understanding, code generation, and feedback adjustment, and conducts multidimensional analysis through dimensions such as task success rate, execution trajectories, and completion judgment.
Source: Machine Heart — Robotics on WeChat · Read original article ↗
English translation unavailable. Choose the original article to read it.
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
Reported success rate
2.7%
What remains unknown
Not established in the collected evidence: Environment, Control, Data origin.
Reported performance applies to the described task. It does not establish general autonomy or deployment readiness.
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
CodeActionBench 基于 RoboTwin 2.0 和 SAPIEN，采用 ALOHA-AgileX 双臂机器人。
Open source S11
在双臂仿真评测中， GPT-6 Astra (Codex CLI) 的尝试成功 率达到 73.3% ，Claude Opus 5 在 Reference harness 和 Claude Code 下分别取得 49.3% 和 45.3% 。
Open source S3
在共享 Reference harness 的七个配置中，Opus 领先，Gemini 3.6 Flash 以 20.0% 的尝试成功率位居其后。
Open source S14
各配置的尝试成功率从 2.7% 到 73.3%，执行轨迹则揭示了总体分数背后的能力差异。
Open source S9
Astra 的占比最高，代码错误率也最低，仅 0.46%（10/2,167） 。
Open source S18
成功率最高的四个配置，该占比为 85.6% 至 9 2.6% ，其余五个为 15.6% 至 35.2% 。
Open source S17
Source：Machine Heart — Robotics on WeChat · mp.weixin.qq.com