Efficient VLA Inference via Retrieval-Based Speculation and Hardware Acceleration
Efficient VLA Inference via Retrieval-Based Speculation and Hardware Acceleration
This paper introduces Zora, a hardware-software co-design framework for accelerating autoregressive Vision-Language-Action (VLA) model inference on resource-constrained robotic platforms. By leveraging a lightweight retrieval-augmented generation (RAG)-based draft model that exploits robotic dynamics without additional neural networks, and employing kinematics-guided action acceptance with KV-cache sharing, Zora reduces latency. Dedicated hardware support (Zora core) addresses memory irregularities in RAG drafting. Evaluated on four VLA models using the LIBERO benchmark, Zora achieves up to 3.4x speedup with software-only optimization and up to 7.6x speedup with hardware acceleration while retaining comparable accuracy.
The authors present Zora, a hardware-software co-design framework that achieves up to 7.6x speedup over GPUs on four VLA models using the LIBERO benchmark while maintaining comparable accuracy.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
speedup factor
≤3.4
View original evidence
-acceptance strategy and a KV-cache-sharing attention mechanism. To address the retrieval overhead introduced by RAG-based drafting, we co-design dedicated hardware support, Zora core, to address the memory irregularity in RAG-based drafting. We evaluate Zora on four VLA models with the LIBERO benchmark. Zora retains comparable accuracy while achieving up to 3.4$\times$ speedup with software-only optimization and up
Open source S4speedup factor
7.6
speedup factor (software)
≤3.4
View original evidence
-acceptance strategy and a KV-cache-sharing attention mechanism. To address the retrieval overhead introduced by RAG-based drafting, we co-design dedicated hardware support, Zora core, to address the memory irregularity in RAG-based drafting. We evaluate Zora on four VLA models with the LIBERO benchmark. Zora retains comparable accuracy while achieving up to 3.4$\times$ speedup with software-only optimization and up
Open source S4speedup factor (hardware)
7.6
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
-acceptance strategy and a KV-cache-sharing attention mechanism. To address the retrieval overhead introduced by RAG-based drafting, we co-design dedicated hardware support, Zora core, to address the memory irregularity in RAG-based drafting. We evaluate Zora on four VLA models with the LIBERO benchmark. Zora retains comparable accuracy while achieving up to 3.4$\times$ speedup with software-only optimization and up
Open source S4
to 7.6$\times$ speedup with hardware support over GPUs.
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org