Sketch-MoMa: Teleoperation for Mobile Manipulator via Interpretation of Hand-Drawn Sketches
Overview
This research paper introduces Sketch-MoMa, a teleoperation system that interprets hand-drawn sketches superimposed on observation images to control mobile manipulators. By leveraging Vision-Language Models (VLMs), the system infers drawn shapes and low-level tasks, enabling precise motion planning for operations like grasping and rotation. The approach is validated with seven tasks and five sketch shapes, showing competitive usability compared to existing 2D interfaces in a user study involving fourteen participants.
Generated from attributed reports · Updated 2 days ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Control: teleoperated
“modalities to set the sketches' semantics. This requires complex operations for users and leads to decreasing usability. In this paper, we propose Sketch-MoMa, a teleoperation system using the user-given hand-drawn sketches as instructions to control a robot. We use Vision-Language Models (VLMs) to understand the user-given sketches superimposed on an observation image and infer drawn shapes and low-level tasks of t”
Exact source · revision 1Source owner not reported
Reported quantity · tasks: 7 other · Basis not reported
“he robot. We utilize the sketches and the generated shapes for recognition and motion planning of the generated low-level tasks for precise and intuitive operations. We validate our approach using state-of-the-art VLMs with 7 tasks and 5 sketch shapes. We also demonstrate that our approach effectively specifies the detailed motions, such as how to grasp and how much to rotate. Moreover, we show the competitive usabil”
Exact source · revision 1Source owner not reported
Reported quantity · sketch shapes: 5 other · Basis not reported
“he robot. We utilize the sketches and the generated shapes for recognition and motion planning of the generated low-level tasks for precise and intuitive operations. We validate our approach using state-of-the-art VLMs with 7 tasks and 5 sketch shapes. We also demonstrate that our approach effectively specifies the detailed motions, such as how to grasp and how much to rotate. Moreover, we show the competitive usabil”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalSketch-MoMa: Teleoperation for Mobile Manipulator via Interpretation of Hand-Drawn Sketches
This research paper introduces Sketch-MoMa, a teleoperation system that interprets hand-drawn sketches superimposed on observation images to control mobile manipulators. By leveraging Vision-Language Models (VLMs), the system infers drawn shapes and low-level tasks, enabling precise motion planning for operations like grasping and rotation. The approach is validated with seven tasks and five sketch shapes, showing competitive usability compared to existing 2D interfaces in a user study involving fourteen participants.
Event coverage history
Current attention 3·Peak within the comparable range 4（2026-11-10 13:00 UTC）·Change within the comparable range over 24 hours –
The trend compares participants observed continuously; its scope may be narrower than current attention totals. Hover or click to explore hourly observations, or use the left and right arrow keys.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).