VLA-GAUGE: Systematically Evaluating Generalization and Robustness in Vision-Language-Action Models

This research presents VLA-GA, a framework for evaluating the generalization and robustness of Vision-Language-Action models. The study finds that models perform poorly on zero-shot tasks and combinatorial perturbations, with mean Zero-Shot Scores of 22.3% and R-Scores of 9.3. Diagnosis-driven fine-tuning significantly improves performance, showing the importance of targeted evaluation and refinement.