HITCH: Teaching a Vision-Language Model to Operate a Humanoid through a Whole-Body Harness
Overview
This paper introduces HITCH, a harness interface that allows vision-language models (VLMs) to control humanoid robots by providing 3D position feedback for any selected point in the image and executing commands for whole-body motion. The authors fine-tune a VLM on trajectories from teacher policies solving five base tasks, achieving 95% success on SIMPLE episodes with added trajectories and demonstrating the ability to carry a box and walk to a goal on a real G1 robot without real training data.
Generated from attributed reports · Updated 2 days ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Reported quantity · trajectories: 5 other · Basis not reported
“e pelvis, torso, hands, walking, and gaze into whole-body motion with a learned tracker. We find that both measuring the scene and executing commands accurately raise task success. We then fine-tune a VLM on trajectories in which teacher policies solve five base tasks through HITCH, so that it learns how to operate the humanoid. Without task-specific trajectories, the fine-tuned model completes 51% of SIMPLE episodes”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalHITCH: Teaching a Vision-Language Model to Operate a Humanoid through a Whole-Body Harness
This paper introduces HITCH, a harness interface that allows vision-language models (VLMs) to control humanoid robots by providing 3D position feedback for any selected point in the image and executing commands for whole-body motion. The authors fine-tune a VLM on trajectories from teacher policies solving five base tasks, achieving 95% success on SIMPLE episodes with added trajectories and demonstrating the ability to carry a box and walk to a goal on a real G1 robot without real training data.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).