Failure Prediction and Classification in Vision-Language-Action Models Via Calibrated Latent Probing
Failure Prediction and Classification in Vision-Language-Action Models Via Calibrated Latent Probing
This paper introduces VLA-Tug, a lightweight probe for detecting and classifying failures in Vision-Language-Action models (VLAs) under distribution shifts. The method improves early failure detection by up to 30% at matched false-positive rates compared to learned baselines and maintains valid bounds on missed-detection and false-alarm rates in up to 86% of runs under unseen tasks, versus 21% for baselines. VLA-Tug also predicts failure modes with up to 44% higher top-3 accuracy on unseen tasks, enabling corrective actions. The authors release a dataset of 3,180 annotated manipulation episodes to support future research.
The authors present VLA-Tug, a lightweight latent probing method for failure prediction in Vision-Language-Action models, reporting up to 30% improvement in early failure detection and 44% increase in top-3 failure mode accuracy on unseen tasks based on experiments described in the abstract.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
annotated manipulation episodes
3,180
View original evidence
ve probe supports lightweight steering that improves task success. We release a dataset of 3,180 annotated manipulation episodes to support future work on latent failure detection in VLAs at https://vla-tug.github.io.
Open source S5Reported accuracy
≤44%
Reported trials
View original evidence
he strongest learned baseline. Under shift to unseen tasks, the bounds on missed-detection and false-alarm rates remain valid in up to 86% of runs, versus at most 21% for baselines. Unlike existing methods, VLA-Tug predicts the failure mode to enable corrective actions, raising top-3 accuracy on unseen tasks by up to 44% over a majority baseline. VLA-Tug enables real-time monitoring where the well-timed and informati
Open source S4
- paper: not_reported
- dataset: Available Artifact link Open source S5
- code: not_reported
- weights: not_reported
- dataset: not_reported
What remains unknown
Not established in the collected evidence: Environment, Control, Data origin.
Reported performance applies to the described task. It does not establish general autonomy or deployment readiness.
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
ve probe supports lightweight steering that improves task success. We release a dataset of 3,180 annotated manipulation episodes to support future work on latent failure detection in VLAs at https://vla-tug.github.io.
Open source S5
he strongest learned baseline. Under shift to unseen tasks, the bounds on missed-detection and false-alarm rates remain valid in up to 86% of runs, versus at most 21% for baselines. Unlike existing methods, VLA-Tug predicts the failure mode to enable corrective actions, raising top-3 accuracy on unseen tasks by up to 44% over a majority baseline. VLA-Tug enables real-time monitoring where the well-timed and informati
Open source S4
edge. Existing methods fail to fulfill these requirements. We present VLA-Tug, a lightweight probe over the latent layers of a VLA backbone that improves early failure detection and bounds the missed-detection and false-alarm rates, with minimal inference overhead. We show that supervising on the pre-failure segment of a trajectory increases early failure detection at matched false-positive rates by up to 30% over t
Open source S3
Failure Prediction and Classification in Vision-Language-Action Models Via Calibrated Latent Probing
Open source S1
arXiv:2611.08425v1 Announce Type: new Abstract: Vision-Language-Action models (VLAs) enable general-purpose embodied intelligence, but their performance is brittle under distribution shift, where unanticipated failures can damage equipment or endanger people. Deploying VLAs in real-world systems therefore requires a runtime monitor that is timely, calibrated, and informative while being efficient enough to run on the
Open source S2
Source：arXiv Robotics — research abstracts · arxiv.org