Say the Same, Act Differently: Diagnosing and Steering Text-Action Divergence in Reasoning VLAs
Say the Same, Act Differently: Diagnosing and Steering Text-Action Divergence in Reasoning VLAs
This research abstract introduces a diagnostic framework called TNAS that reveals how visual perturbations can alter the physical actions of reasoning vision-language-action (VLA) models while keeping their natural-language rationales identical.
The paper presents a diagnostic method (TNAS) and attack demonstrations on Alpamayo and InstructVLA models showing that visual perturbations can decouple physical actions from text rationales, with specific metrics on action shift, text drift, and collision rates under full-image and localized patch perturbations.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
hift actions 4.64x to 47.8x more than text-gradient perturbations, whereas text-gradient perturbations induce 21.2x to 277x more text drift. Steering visual perturbations along this direction reaches the same decoupling from the image. Across 154 obstacle-annotated driving scenes in Alpamayo, TNAS-guided attacks induce a new collision under a word-for-word identical rationale in 52.6% of scenes under full-image pertu
Open source S4
rbations and 35.1% under a localized forward-camera patch, compared with 4.55% for unguided action attacks. Relative to a text-preserving baseline, TNAS guidance raises exact rationale preservation from 83.0% to 89.5% and cuts teacher-forced text drift by 68.4% at a comparable collision rate. In InstructVLA, TNAS guidance cuts text drift by 40.5% and preserves the exact response on 85.0% of examples alongside a 1.73-
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org