VLALagBench: Understanding When and Why VLA Inference Latency Matters
VLALagBench: Understanding When and Why VLA Inference Latency Matters
This paper introduces VLALagBench, a benchmark for evaluating Vision-Language-Action (VLA) models under varying inference latencies. The study reveals that even a 100 ms latency can nearly halve task success on moving targets, with sensitivity dependent on task type and policy. The authors also develop an empirical model to predict task success from limited calibration data, achieving prediction errors between 5.7% and 7.8%, which can guide system-specific configuration choices for maximizing performance.
The authors introduce VLALagBench to quantify how inference latency impacts VLA model performance, demonstrating that a 100 ms delay can significantly reduce success rates on moving targets and proposing an empirical model for predicting task success based on system latency.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
inference latency
100
View original evidence
cy sensitivity across task types, object speeds, execution modes (synchronous and asynchronous), and VLA policies. Using VLALagBench, we find that even a modest inference latency of 100 ms can nearly halve task success on moving targets, with sensitivity varying substantially across tasks and policies. These differences reflect task-specific requirements for precision and continuous motion. By overlapping inference w
Open source S3
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
cy sensitivity across task types, object speeds, execution modes (synchronous and asynchronous), and VLA policies. Using VLALagBench, we find that even a modest inference latency of 100 ms can nearly halve task success on moving targets, with sensitivity varying substantially across tasks and policies. These differences reflect task-specific requirements for precision and continuous motion. By overlapping inference w
Open source S3
ith action execution, asynchronous execution can recover some of the lost success at the cost of more frequent inference requests. Building on these findings, we develop an empirical model that predicts task success from limited calibration data, achieving root-mean-square prediction errors of only 5.7-7.8% across various VLA policies. Given an inference system's latency and per-robot throughput, these predictions gu
Open source S4
Source：arXiv Robotics — research abstracts · arxiv.org