VLA Adaptation with Real and Simulated Data: A Case Study on Robotic Chess
VLA Adaptation with Real and Simulated Data: A Case Study on Robotic Chess
This research examines the adaptation of Vision-language-action (VLA) models for a robotic chess task, specifically moving a king between named squares. The authors fine-tune the \pi_{0.5} model using varying amounts of teleoperated and simulated data. Key findings include that success rates in simulation scale with data volume, a learning-rate cool-down improves checkpoint comparison by up to 31 points, and pretraining on simulated data significantly aids performance when real data is limited. Conversely, experiments show that advanced methods like filtered behavior cloning and residual reinforcement learning did not yield measurable improvements over standard behavior cloning in this context. The paper releases the simulator, expert script, and real demonstration data.
The study investigates VLA model fine-tuning for robotic chess using both teleoperated real data and simulated scripted expert data, finding that pretraining on simulation helps when real data is scarce but noting that certain advanced training techniques like RECAP-style conditioning did not improve upon plain behavior cloning in this specific setting.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- Control
- TeleoperatedOpen source S3
Reported numbers
trials
10
Reported trials
View original evidence
imitates the variability of human demonstrations. In simulation, the success rate scales with both data and updates. Importantly, we find that a short learning-rate cool-down adds up to 31 points and is necessary for comparing checkpoints, and that validation loss and action error are unreliable for model selection. On the real arm, we fine-tune on up to 256 teleoperated demonstrations and evaluate ten policies in 2
Open source S4points
≤31
View original evidence
imitates the variability of human demonstrations. In simulation, the success rate scales with both data and updates. Importantly, we find that a short learning-rate cool-down adds up to 31 points and is necessary for comparing checkpoints, and that validation loss and action error are unreliable for model selection. On the real arm, we fine-tune on up to 256 teleoperated demonstrations and evaluate ten policies in 2
Open source S4demonstrations
0.5
View original evidence
n. We study these questions on a task that requires both precise manipulation and spatial understanding of the scene. In this task, an SO-101 arm moves a chess king between two squares that are given only by their names, such as e4 and a5. We fine-tune $\pi_{0.5}$ with various dataset sizes and update budgets, using teleoperated demonstrations as well as simulated demonstrations from a randomized scripted expert that
Open source S3
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
arXiv:2611.12676v1 Announce Type: new Abstract: Vision-language-action (VLA) models are trained as general-purpose foundation models for robot learning, and require fine-tuning for each downstream task and robot. This raises practical questions such as how much data and training updates are needed, how checkpoints should be selected, and whether simulation can be leveraged to alleviate the cost of real data collectio
Open source S2
n. We study these questions on a task that requires both precise manipulation and spatial understanding of the scene. In this task, an SO-101 arm moves a chess king between two squares that are given only by their names, such as e4 and a5. We fine-tune $\pi_{0.5}$ with various dataset sizes and update budgets, using teleoperated demonstrations as well as simulated demonstrations from a randomized scripted expert that
Open source S3
imitates the variability of human demonstrations. In simulation, the success rate scales with both data and updates. Importantly, we find that a short learning-rate cool-down adds up to 31 points and is necessary for comparing checkpoints, and that validation loss and action error are unreliable for model selection. On the real arm, we fine-tune on up to 256 teleoperated demonstrations and evaluate ten policies in 2
Open source S4
5 trials each. Partial results indicate that pretraining on the simulated demonstrations helps substantially, most clearly when real data is scarce. We also present a negative result, in which training on the policy's own rollouts with filtered behavior cloning, RECAP-style advantage conditioning, corrections from an automatic coach, or residual reinforcement learning did not yield a measurable improvement over plain
Open source S5
behavior cloning in our setting. We release the simulator, the expert and the real demonstrations.
Open source S6
Source：arXiv Robotics — research abstracts · arxiv.org