Failure Prediction and Classification in Vision-Language-Action Models Via Calibrated Latent Probing
Overview
This paper introduces VLA-Tug, a lightweight probe for detecting and classifying failures in Vision-Language-Action models (VLAs) under distribution shifts. The method improves early failure detection by up to 30% at matched false-positive rates compared to learned baselines and maintains valid bounds on missed-detection and false-alarm rates in up to 86% of runs under unseen tasks, versus 21% for baselines. VLA-Tug also predicts failure modes with up to 44% higher top-3 accuracy on unseen tasks, enabling corrective actions. The authors release a dataset of 3,180 annotated manipulation episodes to support future research.
Generated from attributed reports · Updated 2 days ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Reported quantity · annotated manipulation episodes: 3180 other · Basis not reported
“ve probe supports lightweight steering that improves task success. We release a dataset of 3,180 annotated manipulation episodes to support future work on latent failure detection in VLAs at https://vla-tug.github.io.”
Exact source · revision 1Source owner not reported
Reported quantity · Reported accuracy: 44 percent · reported trials
“he strongest learned baseline. Under shift to unseen tasks, the bounds on missed-detection and false-alarm rates remain valid in up to 86% of runs, versus at most 21% for baselines. Unlike existing methods, VLA-Tug predicts the failure mode to enable corrective actions, raising top-3 accuracy on unseen tasks by up to 44% over a majority baseline. VLA-Tug enables real-time monitoring where the well-timed and informati”
Exact source · revision 1Source owner not reported
Artifact availability · dataset: available
“ve probe supports lightweight steering that improves task success. We release a dataset of 3,180 annotated manipulation episodes to support future work on latent failure detection in VLAs at https://vla-tug.github.io.”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalFailure Prediction and Classification in Vision-Language-Action Models Via Calibrated Latent Probing
This paper introduces VLA-Tug, a lightweight probe for detecting and classifying failures in Vision-Language-Action models (VLAs) under distribution shifts. The method improves early failure detection by up to 30% at matched false-positive rates compared to learned baselines and maintains valid bounds on missed-detection and false-alarm rates in up to 86% of runs under unseen tasks, versus 21% for baselines. VLA-Tug also predicts failure modes with up to 44% higher top-3 accuracy on unseen tasks, enabling corrective actions. The authors release a dataset of 3,180 annotated manipulation episodes to support future research.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).