CalVLA: Shaping Native Confidence in Vision-Language-Action Policies with Outcome Supervision
Overview
This research presents CalVLA, a method that enhances the ability of vision-language-action (VLA) policies to predict task failure by shaping native action-token confidence through outcome supervision. While adaptation improves confidence calibration (AUROC rising from 0.4485 to 0.7327), it results in a slight decrease in mean task success from 58% to 55%, highlighting that the primary benefit is improved failure discrimination rather than improved control or established safety gains.
Generated from attributed reports · Updated 2 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Reported quantity · AUROC: 0.7392 other · Basis not reported · Differing source assertions
“aired within tasks at similar normalized elapsed times, not necessarily equivalent physical progress. On a held-out test split from 500 LIBERO-10 configurations, adapted confidence achieves a joint failure-discrimination AUROC of $0.7392\pm0.0480$ over three seeds, averaging scores at 130 and 260 steps. Frozen-policy probes reach 0.6146--0.6436, but superiority over the linear probe is statistically inconclusive. For”
Exact source · revision 1Source owner not reported
Reported quantity · AUROC: 0.6436 other · Basis not reported · Differing source assertions
“aired within tasks at similar normalized elapsed times, not necessarily equivalent physical progress. On a held-out test split from 500 LIBERO-10 configurations, adapted confidence achieves a joint failure-discrimination AUROC of $0.7392\pm0.0480$ over three seeds, averaging scores at 130 and 260 steps. Frozen-policy probes reach 0.6146--0.6436, but superiority over the linear probe is statistically inconclusive. For”
Exact source · revision 1Source owner not reported
Reported quantity · AUROC: 0.4485 other · Basis not reported · Differing source assertions
“the same confidence summary, AUROC rises from 0.4485 before adaptation to 0.7327 after adaptation. Mean task success decreases from 58% to 55%. These results show improved failure discrimination, not improved control or established safety benefits.”
Exact source · revision 1Source owner not reported
Reported quantity · AUROC: 0.7327 other · Basis not reported · Differing source assertions
“the same confidence summary, AUROC rises from 0.4485 before adaptation to 0.7327 after adaptation. Mean task success decreases from 58% to 55%. These results show improved failure discrimination, not improved control or established safety benefits.”
Exact source · revision 1Source owner not reported
Reported quantity · steps: 130 other · Basis not reported
“aired within tasks at similar normalized elapsed times, not necessarily equivalent physical progress. On a held-out test split from 500 LIBERO-10 configurations, adapted confidence achieves a joint failure-discrimination AUROC of $0.7392\pm0.0480$ over three seeds, averaging scores at 130 and 260 steps. Frozen-policy probes reach 0.6146--0.6436, but superiority over the linear probe is statistically inconclusive. For”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalCalVLA: Shaping Native Confidence in Vision-Language-Action Policies with Outcome Supervision
This research presents CalVLA, a method that enhances the ability of vision-language-action (VLA) policies to predict task failure by shaping native action-token confidence through outcome supervision. While adaptation improves confidence calibration (AUROC rising from 0.4485 to 0.7327), it results in a slight decrease in mean task success from 58% to 55%, highlighting that the primary benefit is improved failure discrimination rather than improved control or established safety gains.
Event coverage history
There is not enough continuous observation data to show a trend.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).