SimRetract: Zero-Shot Sim-to-Real Robotic Assistance for Retractor Take-Over in Open Surgery
SimRetract: Zero-Shot Sim-to-Real Robotic Assistance for Retractor Take-Over in Open Surgery
This research introduces SimRetract, the first zero-shot sim-to-real transfer system for robotic assistance in open surgery, specifically for retractor take-over. The robot arm learns from simulated demonstrations to grasp retractors from a surgeon's hand while preserving retraction, achieving 90% success on phantom presentations and 78% retention of retraction without real-world fine-tuning or human demonstrations.
The study presents a zero-shot sim-to-real transfer framework for robotic retractor take-over in open surgery, achieving 90% grasp success on phantom presentations and 78% retention of retraction at handoff without real-world fine-tuning or human demonstrations.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
simulated trials count
1,200
View original evidence
efined phantom presentations and preserves retraction at handoff in 78%. Results on unseen retractors suggest potential for generalization to instruments with similar grasp geometry. Across ten policy configurations and 1,200 simulated and real trials, we localize the sim-to-real gap to components: in simulation, grasp rates across ablations span 80-96%, while on the real robot removing the instrument mask alone drop
Open source S4real trials count
1,200
View original evidence
efined phantom presentations and preserves retraction at handoff in 78%. Results on unseen retractors suggest potential for generalization to instruments with similar grasp geometry. Across ten policy configurations and 1,200 simulated and real trials, we localize the sim-to-real gap to components: in simulation, grasp rates across ablations span 80-96%, while on the real robot removing the instrument mask alone drop
Open source S4policy configurations count
10
View original evidence
efined phantom presentations and preserves retraction at handoff in 78%. Results on unseen retractors suggest potential for generalization to instruments with similar grasp geometry. Across ten policy configurations and 1,200 simulated and real trials, we localize the sim-to-real gap to components: in simulation, grasp rates across ablations span 80-96%, while on the real robot removing the instrument mask alone drop
Open source S4
- code: Available Artifact link Open source S5
- paper: not_reported
- weights: not_reported
- dataset: not_reported
- code: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
monstrations is limited by collection and labeling costs, and by performance degradation when the environment changes. Our framework, SimRetract, generates simulated demonstrations designed for transfer: the policy observes masked instrument images and grasp geometry, learns to recover from off-nominal states, and acts with margin for execution drift. Deployed zero-shot, it grasps the retractor in 90% of 50 surgeon-d
Open source S3
arXiv:2610.38234v2 Announce Type: replace Abstract: We demonstrate the first zero-shot sim-to-real transfer for a surgeon-robot collaboration task in open surgery: retractor take-over. A robot arm takes a retractor from the surgeon's hand and preserves retraction at handoff, using a policy trained only on simulated demonstrations, without human demonstrations or real-world fine-tuning. Imitation learning from real de
Open source S2
efined phantom presentations and preserves retraction at handoff in 78%. Results on unseen retractors suggest potential for generalization to instruments with similar grasp geometry. Across ten policy configurations and 1,200 simulated and real trials, we localize the sim-to-real gap to components: in simulation, grasp rates across ablations span 80-96%, while on the real robot removing the instrument mask alone drop
Open source S4
s performance to 22% grasped and 12% held. Transfer-oriented design thus makes zero-shot sim-to-real transfer feasible for a surgical assistance task without human demonstrations. Code is available here: https://anonymous.4open.science/r/simretract-58D2.
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org