When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAs
When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAs
This research addresses counterfactual failures in Vision-Language-Action models (VLAs), where models rely on vision shortcuts instead of following language instructions. The authors introduce LIBERO-CF, a benchmark for evaluating these failures, and propose Counterfactual Action Guidance (CAG), a training-free method that improves language grounding and task success across diverse VLAs without modifying base architectures.
The paper introduces LIBERO-CF, a benchmark for evaluating counterfactual failures in Vision-Language-Action models (VLAs), and proposes Counterfactual Action Guidance (CAG) to mitigate these issues by strengthening language conditioning without modifying base architectures.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
on vision shortcuts, improves robustness on counterfactual tasks, and requires neither additional demonstrations nor modifications to the base VLA architecture or its pretrained weights. Extensive experiments demonstrate consistent improvements across diverse VLAs. On LIBERO-CF, our training-free variant, TF-CAG, improves $\pi_{0.5}$ by 9.7% in language grounding and 3.6% in task success. When using a separate Vision
Open source S5
-Action (VA) module, VA-CAG achieves larger gains of 15.5% and 8.5%, respectively. In real-world evaluations, VA-CAG improves language grounding by 9.4% and task success by 17.2% on average.
Open source S6
Source：arXiv Robotics — research abstracts · arxiv.org