Semantics-to-Contact: A Stagewise Framework for Robust Contact-Rich Manipulation
Overview
This research introduces Semantics-to-Contact (S2C), a stagewise framework designed to enhance visually robust contact-rich manipulation. The approach decomposes the problem into semantic focusing, where a vision-language model localizes task-relevant regions in diverse visual scenes, and contact refinement, where a human-in-the-loop residual reinforcement learning policy learns fine-grained behaviors using force feedback within those localized regions. To further boost visual robustness, an object-centric visual augmentation strategy randomizes background appearance while preserving the robot and relevant objects. Experiments across six real-world manipulation tasks demonstrate that S2C consistently outperforms strong baselines in success rates and maintains reliable performance under significant visual and positional variations.
Generated from attributed reports · Updated 2 days ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Control: shared control
“e the problem into two stages: semantic focusing and contact refinement. In the first stage, a vision-language model localizes task-relevant regions across visually diverse scenes, providing coarse spatial guidance and reducing the burden of exploration. In the second stage, a real-world human-in-the-loop residual reinforcement learning policy learns fine-grained contact behaviors within the localized region using fo”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalSemantics-to-Contact: A Stagewise Framework for Robust Contact-Rich Manipulation
This research introduces Semantics-to-Contact (S2C), a stagewise framework designed to enhance visually robust contact-rich manipulation. The approach decomposes the problem into semantic focusing, where a vision-language model localizes task-relevant regions in diverse visual scenes, and contact refinement, where a human-in-the-loop residual reinforcement learning policy learns fine-grained behaviors using force feedback within those localized regions. To further boost visual robustness, an object-centric visual augmentation strategy randomizes background appearance while preserving the robot and relevant objects. Experiments across six real-world manipulation tasks demonstrate that S2C consistently outperforms strong baselines in success rates and maintains reliable performance under significant visual and positional variations.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).