Behavioral Steering of Vision-Language-Action Models in the Low-Parameter Regime
Behavioral Steering of Vision-Language-Action Models in the Low-Parameter Regime
This research investigates adapting pretrained vision-language-action models (VLAs) to specific behavioral preferences, such as carrying objects higher, by freezing a 7B parameter OpenVLA-OFT policy and optimizing TinyLoRA adapters with only 8 to 384 trainable parameters. Using reinforcement learning methods like GRPO or evolutionary search (CMA-ES), the authors achieve significant improvements in task performance—for instance, raising carry height by 120 mm on trained tasks at 92% success—while retaining most of the model's pretrained capabilities. The study highlights that adapter capacity and optimizer choice critically influence the trade-off between style gains and capability retention, showing that even minimal parameter updates can effectively steer VLA behavior.
The study demonstrates that behavioral steering of pretrained Vision-Language-Action Models (VLAs) can be achieved with extremely low-dimensional updates (as few as 256 trainable parameters) using TinyLoRA adapters, effectively balancing style adaptation with capability retention without full retraining.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
carry height increase on trained tasks
120
View original evidence
OpenVLA-OFT 7B policy and optimize TinyLoRA adapters with 8 to 384 trainable parameters using either policy-gradient RL (GRPO) or evolutionary search (CMA-ES). With just 256 trainable parameters, we raise carry height by 120 mm on trained tasks at 92% success and by 74 mm on a held-out task at 80% success. By rescaling the learned update, we can tune the trade-off between style gains and capability retention without
Open source S3carry height increase on held-out task
74
View original evidence
OpenVLA-OFT 7B policy and optimize TinyLoRA adapters with 8 to 384 trainable parameters using either policy-gradient RL (GRPO) or evolutionary search (CMA-ES). With just 256 trainable parameters, we raise carry height by 120 mm on trained tasks at 92% success and by 74 mm on a held-out task at 80% success. By rescaling the learned update, we can tune the trade-off between style gains and capability retention without
Open source S3trainable parameters in TinyLoRA adapters
256
View original evidence
OpenVLA-OFT 7B policy and optimize TinyLoRA adapters with 8 to 384 trainable parameters using either policy-gradient RL (GRPO) or evolutionary search (CMA-ES). With just 256 trainable parameters, we raise carry height by 120 mm on trained tasks at 92% success and by 74 mm on a held-out task at 80% success. By rescaling the learned update, we can tune the trade-off between style gains and capability retention without
Open source S3trainable parameters range
8–384
View original evidence
OpenVLA-OFT 7B policy and optimize TinyLoRA adapters with 8 to 384 trainable parameters using either policy-gradient RL (GRPO) or evolutionary search (CMA-ES). With just 256 trainable parameters, we raise carry height by 120 mm on trained tasks at 92% success and by 74 mm on a held-out task at 80% success. By rescaling the learned update, we can tune the trade-off between style gains and capability retention without
Open source S3
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
OpenVLA-OFT 7B policy and optimize TinyLoRA adapters with 8 to 384 trainable parameters using either policy-gradient RL (GRPO) or evolutionary search (CMA-ES). With just 256 trainable parameters, we raise carry height by 120 mm on trained tasks at 92% success and by 74 mm on a held-out task at 80% success. By rescaling the learned update, we can tune the trade-off between style gains and capability retention without
Open source S3
Behavioral Steering of Vision-Language-Action Models in the Low-Parameter Regime
Open source S1
Source：arXiv Robotics — research abstracts · arxiv.org