HITCH: Teaching a Vision-Language Model to Operate a Humanoid through a Whole-Body Harness
HITCH: Teaching a Vision-Language Model to Operate a Humanoid through a Whole-Body Harness
This paper introduces HITCH, a harness interface that allows vision-language models (VLMs) to control humanoid robots by providing 3D position feedback for any selected point in the image and executing commands for whole-body motion. The authors fine-tune a VLM on trajectories from teacher policies solving five base tasks, achieving 95% success on SIMPLE episodes with added trajectories and demonstrating the ability to carry a box and walk to a goal on a real G1 robot without real training data.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
trajectories
5
View original evidence
e pelvis, torso, hands, walking, and gaze into whole-body motion with a learned tracker. We find that both measuring the scene and executing commands accurately raise task success. We then fine-tune a VLM on trajectories in which teacher policies solve five base tasks through HITCH, so that it learns how to operate the humanoid. Without task-specific trajectories, the fine-tuned model completes 51% of SIMPLE episodes
Open source S3
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
e pelvis, torso, hands, walking, and gaze into whole-body motion with a learned tracker. We find that both measuring the scene and executing commands accurately raise task success. We then fine-tune a VLM on trajectories in which teacher policies solve five base tasks through HITCH, so that it learns how to operate the humanoid. Without task-specific trajectories, the fine-tuned model completes 51% of SIMPLE episodes
Open source S3
and 42.5% of HumanoidArena episodes, and adding five trajectories per SIMPLE task raises its SIMPLE success to 95%. On a real G1, it carries a box and walks to a goal without real training data.
Open source S4
Source：arXiv Robotics — research abstracts · arxiv.org