Efficient VLA Inference via Retrieval-Based Speculation and Hardware Acceleration
Overview
This paper introduces Zora, a hardware-software co-design framework for accelerating autoregressive Vision-Language-Action (VLA) model inference on resource-constrained robotic platforms. By leveraging a lightweight retrieval-augmented generation (RAG)-based draft model that exploits robotic dynamics without additional neural networks, and employing kinematics-guided action acceptance with KV-cache sharing, Zora reduces latency. Dedicated hardware support (Zora core) addresses memory irregularities in RAG drafting. Evaluated on four VLA models using the LIBERO benchmark, Zora achieves up to 3.4x speedup with software-only optimization and up to 7.6x speedup with hardware acceleration while retaining comparable accuracy.
Generated from attributed reports · Updated 4 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Reported quantity · speedup factor: 3.4 other · Basis not reported · Differing source assertions
“-acceptance strategy and a KV-cache-sharing attention mechanism. To address the retrieval overhead introduced by RAG-based drafting, we co-design dedicated hardware support, Zora core, to address the memory irregularity in RAG-based drafting. We evaluate Zora on four VLA models with the LIBERO benchmark. Zora retains comparable accuracy while achieving up to 3.4$\times$ speedup with software-only optimization and up”
Exact source · revision 1Source owner not reported
Reported quantity · speedup factor: 7.6 other · Basis not reported · Differing source assertions
“to 7.6$\times$ speedup with hardware support over GPUs.”
Exact source · revision 1Source owner not reported
Reported quantity · speedup factor (software): 3.4 other · Basis not reported
“-acceptance strategy and a KV-cache-sharing attention mechanism. To address the retrieval overhead introduced by RAG-based drafting, we co-design dedicated hardware support, Zora core, to address the memory irregularity in RAG-based drafting. We evaluate Zora on four VLA models with the LIBERO benchmark. Zora retains comparable accuracy while achieving up to 3.4$\times$ speedup with software-only optimization and up”
Exact source · revision 1Source owner not reported
Reported quantity · speedup factor (hardware): 7.6 other · Basis not reported
“to 7.6$\times$ speedup with hardware support over GPUs.”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalEfficient VLA Inference via Retrieval-Based Speculation and Hardware Acceleration
This paper introduces Zora, a hardware-software co-design framework for accelerating autoregressive Vision-Language-Action (VLA) model inference on resource-constrained robotic platforms. By leveraging a lightweight retrieval-augmented generation (RAG)-based draft model that exploits robotic dynamics without additional neural networks, and employing kinematics-guided action acceptance with KV-cache sharing, Zora reduces latency. Dedicated hardware support (Zora core) addresses memory irregularities in RAG drafting. Evaluated on four VLA models using the LIBERO benchmark, Zora achieves up to 3.4x speedup with software-only optimization and up to 7.6x speedup with hardware acceleration while retaining comparable accuracy.
Event coverage history
There is not enough continuous observation data to show a trend.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).