Zero-Shot VLA Adaptation with Retrieved Guidance
Zero-Shot VLA Adaptation with Retrieved Guidance
This paper presents RAGu, a method for adapting Vision-Language-Action (VLA) models to new domains by retrieving and guiding behavior from domain-mismatched demonstration datasets. The approach incorporates geometric calibration to align source demonstrations with the target setup and employs a coarse-to-fine retrieval strategy to select compatible trajectories. Instead of retraining the base model or expanding its context window, RAGu guides the VLA's flow-matching inference procedure. Evaluations across real-world, simulation, sim-to-real transfer, and cross-embodiment humanoid manipulation show that RAGu improves real-world VLA success rates from 20% to 80% and outperforms related baselines in simulation.
The authors introduce RAGu, a retrieval-augmented framework that uses geometric calibration and coarse-to-fine selection to guide zero-shot VLA adaptation in new domains without retraining the base model.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
Reported success rate
20%
View original evidence
eal-world, simulation, sim-to-real transfer, and cross-embodiment humanoid manipulation. We find it improves real-world VLA success rates from 20% to 80% and outperforms the closest-related baselines in sim. Project website: raguvla.github.io
Open source S5Reported success rate
80%
View original evidence
eal-world, simulation, sim-to-real transfer, and cross-embodiment humanoid manipulation. We find it improves real-world VLA success rates from 20% to 80% and outperforms the closest-related baselines in sim. Project website: raguvla.github.io
Open source S5
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
What remains unknown
Not established in the collected evidence: Environment, Control, Data origin.
Reported performance applies to the described task. It does not establish general autonomy or deployment readiness.
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
eal-world, simulation, sim-to-real transfer, and cross-embodiment humanoid manipulation. We find it improves real-world VLA success rates from 20% to 80% and outperforms the closest-related baselines in sim. Project website: raguvla.github.io
Open source S5
arXiv:2611.10894v1 Announce Type: new Abstract: Pre-training large parametric "Vision-Language-Action" (VLA) models on large demonstration datasets has recently yielded promising results for general-purpose manipulation. However, these models still struggle under even small domain shifts relative to the training data, such as changes in camera viewpoint, background, object identity, or robot morphology. Motivated by
Open source S2
the success of Retrieval-Augmented Generation (RAG) in language models, we pursue retrieval to boost VLA performance. Specifically, we seek to leverage retrieval from domain-mismatched, "in-the-wild" demonstration datasets like DROID to guide a VLA's behavior in a new target domain. To achieve this, we introduce RAGu, which overcomes robotics-specific challenges for RAG: (1) It incorporates simple geometric calibrati
Open source S3
on to transform source demonstrations into the target setup and reduce cross-domain differences. (2) It uses a coarse-to-fine retrieval strategy that selects, from a large pool of pre-warped candidates, the trajectory most compatible with the base VLA. (3) It guides the VLA's flow-matching inference procedure rather than seeking to retrain the base model or expand its limited context window. We evaluate RAGu across r
Open source S4
Zero-Shot VLA Adaptation with Retrieved Guidance
Open source S1
Source：arXiv Robotics — research abstracts · arxiv.org