SketchMate: Agentic Geometric Reasoning for Zero-Shot Precision Insertion
SketchMate: Agentic Geometric Reasoning for Zero-Shot Precision Insertion
This research introduces SketchMate, a system that uses vision-language models to create geometric sketches of insertion tasks. By decoupling part description from placement and using sketch memory with point-cloud registration, the method achieves sub-millimetre precision in simulated environments. On 69 episodes, it successfully inserted 88% of unseen parts at the correct pose, significantly outperforming existing zero-shot agentic manipulation methods which failed on the same task.
The study presents a method combining vision-language models with geometric sketching to enable zero-shot precision insertion, reporting 88% success on simulated episodes compared to baseline methods that achieved none.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- Environment
- SimulationOpen source S4
Reported numbers
sketch generation time
3.4
baseline sketch generation time
≈90
simulated episodes
69
Reported trials
View original evidence
asping the part, the robot inspects both mating faces at close range and corrects the sketch by shape matching. On 69 simulated insertion episodes, our method inserts 88% of the parts at the assembled pose in a strict kinematic test, while ReKep and MOKA with the same VLM insert none. On new placements of eight tasks, reusing one stored run inserts as many parts as calling the agent in each episode, and it produces t
Open source S4number of tasks
8
View original evidence
asping the part, the robot inspects both mating faces at close range and corrects the sketch by shape matching. On 69 simulated insertion episodes, our method inserts 88% of the parts at the assembled pose in a strict kinematic test, while ReKep and MOKA with the same VLM insert none. On new placements of eight tasks, reusing one stored run inserts as many parts as calling the agent in each episode, and it produces t
Open source S4robots
1
View original evidence
asping the part, the robot inspects both mating faces at close range and corrects the sketch by shape matching. On 69 simulated insertion episodes, our method inserts 88% of the parts at the assembled pose in a strict kinematic test, while ReKep and MOKA with the same VLM insert none. On new placements of eight tasks, reusing one stored run inserts as many parts as calling the agent in each episode, and it produces t
Open source S4
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
asping the part, the robot inspects both mating faces at close range and corrects the sketch by shape matching. On 69 simulated insertion episodes, our method inserts 88% of the parts at the assembled pose in a strict kinematic test, while ReKep and MOKA with the same VLM insert none. On new placements of eight tasks, reusing one stored run inserts as many parts as calling the agent in each episode, and it produces t
Open source S4
he sketch in 3.4 s instead of about 90 s.
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org