Swim2Real: Vision-Language Models for Simulator Calibration of a Soft Robotic Fish
Swim2Real: Vision-Language Models for Simulator Calibration of a Soft Robotic Fish
This research introduces Swim2Real, a method that uses vision-language models (VLMs) to calibrate simulators for soft aquatic robots. By comparing real and simulated swimming videos, the system proposes updates to 16 simulator parameters using backtracking line search. In an independent preregistered replication with fifty seeds, Swim2Real reduced the mean best-so-far error over 40 simulator evaluations by 63.4 mm compared to random directions and 53.2 mm compared to signed-permuted directions, with both paired comparisons passing Holm correction (p<10^-5). A second VLM in a 24-seed experiment supported the same direction effect. The method averaged 16.1 VLM proposals per run versus 39 for Vid2Sid while attaining lower error at the same simulator budget. Ten hardware trials demonstrated open-loop execution of simulator-generated motor commands.
The study presents a method using vision-language models to calibrate soft robotic fish simulators by comparing real and simulated video data, achieving statistically significant error reduction in preregistered replication experiments.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
simulator evaluations
>40
View original evidence
hether the proposals carry useful directions, we compare them with random and signed-permuted directions under the same line search and simulator budget. In an independent, preregistered fifty-seed replication, Swim2Real reduces the mean best-so-far error over 40 simulator evaluations by 63.4 mm against random directions and 53.2 mm against permuted directions. Both paired comparisons pass Holm correction ($p<10^{-5}
Open source S3VLM proposals per run
16.1
View original evidence
$), conditional on run completion. A 24-seed experiment with a second VLM supports the same direction effect. Backtracking averages 16.1 VLM proposals per run versus 39 for Vid2Sid, which attains lower error at the same simulator budget. Ten hardware trials demonstrate open-loop execution of simulator-generated motor commands. Project page and videos: https://swim2real.github.io.
Open source S4random direction error reduction
63.4
View original evidence
hether the proposals carry useful directions, we compare them with random and signed-permuted directions under the same line search and simulator budget. In an independent, preregistered fifty-seed replication, Swim2Real reduces the mean best-so-far error over 40 simulator evaluations by 63.4 mm against random directions and 53.2 mm against permuted directions. Both paired comparisons pass Holm correction ($p<10^{-5}
Open source S3permuted direction error reduction
53.2
View original evidence
hether the proposals carry useful directions, we compare them with random and signed-permuted directions under the same line search and simulator budget. In an independent, preregistered fifty-seed replication, Swim2Real reduces the mean best-so-far error over 40 simulator evaluations by 63.4 mm against random directions and 53.2 mm against permuted directions. Both paired comparisons pass Holm correction ($p<10^{-5}
Open source S3VLM proposals per run (Vid2Sid)
39
View original evidence
$), conditional on run completion. A 24-seed experiment with a second VLM supports the same direction effect. Backtracking averages 16.1 VLM proposals per run versus 39 for Vid2Sid, which attains lower error at the same simulator budget. Ten hardware trials demonstrate open-loop execution of simulator-generated motor commands. Project page and videos: https://swim2real.github.io.
Open source S4error reduction (paired comparison)
63.4
View original evidence
hether the proposals carry useful directions, we compare them with random and signed-permuted directions under the same line search and simulator budget. In an independent, preregistered fifty-seed replication, Swim2Real reduces the mean best-so-far error over 40 simulator evaluations by 63.4 mm against random directions and 53.2 mm against permuted directions. Both paired comparisons pass Holm correction ($p<10^{-5}
Open source S3error reduction (paired comparison)
53.2
View original evidence
hether the proposals carry useful directions, we compare them with random and signed-permuted directions under the same line search and simulator budget. In an independent, preregistered fifty-seed replication, Swim2Real reduces the mean best-so-far error over 40 simulator evaluations by 63.4 mm against random directions and 53.2 mm against permuted directions. Both paired comparisons pass Holm correction ($p<10^{-5}
Open source S3
- code: not_reported
- weights: not_reported
- dataset: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
hether the proposals carry useful directions, we compare them with random and signed-permuted directions under the same line search and simulator budget. In an independent, preregistered fifty-seed replication, Swim2Real reduces the mean best-so-far error over 40 simulator evaluations by 63.4 mm against random directions and 53.2 mm against permuted directions. Both paired comparisons pass Holm correction ($p<10^{-5}
Open source S3
$), conditional on run completion. A 24-seed experiment with a second VLM supports the same direction effect. Backtracking averages 16.1 VLM proposals per run versus 39 for Vid2Sid, which attains lower error at the same simulator budget. Ten hardware trials demonstrate open-loop execution of simulator-generated motor commands. Project page and videos: https://swim2real.github.io.
Open source S4
Source：arXiv Robotics — research abstracts · arxiv.org