Full-Weight VLA Fine-Tuning Without an fp32 Master Copy Silently Erases Most Updates
Full-Weight VLA Fine-Tuning Without an fp32 Master Copy Silently Erases Most Updates
This paper reveals that fine-tuning VLA models with bfloat16 weights without an fp32 master copy leads to significant loss of updates due to rounding. The study shows that retaining fp32 weights during training improves performance on tasks like LIBERO-Object, with statistically significant results across multiple models and training configurations.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- code: not_reported
- weights: not_reported
- dataset: not_reported
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
uccess over LeRobot's storage from 4.3% to 49.3% for SmolVLA and from 5.1% to 80.9% for $\pi_{0.5}$ at a ten-times lower learning rate, where both models are undertrained, and from 89.0% to 97.7% for $\pi_{0.5}$ at its default rate. For SmolVLA at its default rate it adds 10.2 points with open-loop 50-action chunks but only 4.2 when re-planning every 10 actions. In these five comparisons and two more against SmolVLA
Open source S4
Source：arXiv Robotics — research abstracts · arxiv.org