Should the Action Expert Train the VLM? Action-Token Supervision and Gradient Routing in Mixture-of-Transformer VLAs
Should the Action Expert Train the VLM? Action-Token Supervision and Gradient Routing in Mixture-of-Transformer VLAs
This research investigates whether gradients from an action expert should update a pre-trained vision-language model (VLM) in Vision-Language-Action (VLA) systems.
The study evaluates gradient coupling versus insulation strategies in Vision-Language-Action models, finding that action-token supervision significantly alters the performance benefits of insulation in both autonomous driving and robot manipulation tasks.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
dependently varying action-token supervision and whether the expert's loss updates the VLM. In open-loop driving, insulation is competitive with coupling when action tokens are supervised; without them, coupling reduces trajectory-prediction error by roughly 25%. When both components are trained jointly from the outset, coupling increases closed-loop driving route progress by approximately 17% with action-token super
Open source S5
vision and 40% without it, and similarly improves robot manipulation performance under both settings. These findings show that the value of gradient insulation depends on the VLM's action supervision and the evaluation setting, challenging insulation as a default VLA training recipe.
Open source S6
Source：arXiv Robotics — research abstracts · arxiv.org