Rethinking Self-Repair in Code-as-Policy Agents: What Drives Retry Success?
Rethinking Self-Repair in Code-as-Policy Agents: What Drives Retry Success?
This research investigates the drivers of retry success in code-as-policy agents for simulated robot manipulation. The authors compare execution feedback templates against resampling and static task guidance baselines on a dataset of 252 initial failures. Using Qwen2.5-Coder-7B, the study finds that fixed lookup feedback templates recover 85.3% of tasks within five retries, outperforming resampling (68.3%) under the same retry cap and token budgets. The research also notes that mismatched feedback can hinder performance on specific tasks like nut-insertion requiring a rim grasp, suggesting that evaluations should include diverse baselines to properly assess self-repair capabilities.
The study evaluates self-repair mechanisms in code-as-policy agents for simulated robot manipulation, comparing execution feedback templates against resampling and static task guidance baselines.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- Environment
- SimulationOpen source S2
Reported numbers
retry attempts
5
View original evidence
s a fresh program after each scene reset. On the same 252 initial failures, Qwen2.5-Coder-7B recovers 85.3% within five retries, compared with 68.3% for resampling with the same retry cap. The advantage persists under fixed total-token budgets. Feedback from other episodes and fixed task guidance yield similar recovery point estimates, with no detected benefit from matching the coarse failure label. In a separate stu
Open source S3
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
arXiv:2611.06863v1 Announce Type: new Abstract: Code-as-policy agents often use execution feedback to retry failed programs. Additional sampling and task guidance can also improve success, leaving the contribution of failure-specific feedback unclear. We study this question in simulated robot manipulation. A fixed lookup selects feedback templates using failure labels and the API variant. A frozen code model generate
Open source S2
s a fresh program after each scene reset. On the same 252 initial failures, Qwen2.5-Coder-7B recovers 85.3% within five retries, compared with 68.3% for resampling with the same retry cap. The advantage persists under fixed total-token budgets. Feedback from other episodes and fixed task guidance yield similar recovery point estimates, with no detected benefit from matching the coarse failure label. In a separate stu
Open source S3
dy, a fixed guided prompt used from the first attempt achieves an observed overall success rate similar to that of template-based feedback. Template-based feedback also lowers recovery on a nut-insertion task requiring a rim grasp, consistent with an incompatible center-grasp instruction. Evaluations of robot self-repair should include resampling, mismatched feedback, and concise, task-compatible static guidance as b
Open source S4
Source：arXiv Robotics — research abstracts · arxiv.org