Swim2Real: Vision-Language Models for Simulator Calibration of a Soft Robotic Fish
Overview
This research introduces Swim2Real, a method that uses vision-language models (VLMs) to calibrate simulators for soft aquatic robots. By comparing real and simulated swimming videos, the system proposes updates to 16 simulator parameters using backtracking line search. In an independent preregistered replication with fifty seeds, Swim2Real reduced the mean best-so-far error over 40 simulator evaluations by 63.4 mm compared to random directions and 53.2 mm compared to signed-permuted directions, with both paired comparisons passing Holm correction (p<10^-5). A second VLM in a 24-seed experiment supported the same direction effect. The method averaged 16.1 VLM proposals per run versus 39 for Vid2Sid while attaining lower error at the same simulator budget. Ten hardware trials demonstrated open-loop execution of simulator-generated motor commands.
Generated from attributed reports · Updated 1 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Reported quantity · simulator evaluations: 40 other · Basis not reported
“hether the proposals carry useful directions, we compare them with random and signed-permuted directions under the same line search and simulator budget. In an independent, preregistered fifty-seed replication, Swim2Real reduces the mean best-so-far error over 40 simulator evaluations by 63.4 mm against random directions and 53.2 mm against permuted directions. Both paired comparisons pass Holm correction ($p<10^{-5}”
Exact source · revision 1Source owner not reported
Reported quantity · VLM proposals per run: 16.1 other · Basis not reported
“$), conditional on run completion. A 24-seed experiment with a second VLM supports the same direction effect. Backtracking averages 16.1 VLM proposals per run versus 39 for Vid2Sid, which attains lower error at the same simulator budget. Ten hardware trials demonstrate open-loop execution of simulator-generated motor commands. Project page and videos: https://swim2real.github.io.”
Exact source · revision 1Source owner not reported
Reported quantity · random direction error reduction: 63.4 other · Basis not reported
“hether the proposals carry useful directions, we compare them with random and signed-permuted directions under the same line search and simulator budget. In an independent, preregistered fifty-seed replication, Swim2Real reduces the mean best-so-far error over 40 simulator evaluations by 63.4 mm against random directions and 53.2 mm against permuted directions. Both paired comparisons pass Holm correction ($p<10^{-5}”
Exact source · revision 1Source owner not reported
Reported quantity · permuted direction error reduction: 53.2 other · Basis not reported
“hether the proposals carry useful directions, we compare them with random and signed-permuted directions under the same line search and simulator budget. In an independent, preregistered fifty-seed replication, Swim2Real reduces the mean best-so-far error over 40 simulator evaluations by 63.4 mm against random directions and 53.2 mm against permuted directions. Both paired comparisons pass Holm correction ($p<10^{-5}”
Exact source · revision 1Source owner not reported
Reported quantity · VLM proposals per run (Vid2Sid): 39 other · Basis not reported
“$), conditional on run completion. A 24-seed experiment with a second VLM supports the same direction effect. Backtracking averages 16.1 VLM proposals per run versus 39 for Vid2Sid, which attains lower error at the same simulator budget. Ten hardware trials demonstrate open-loop execution of simulator-generated motor commands. Project page and videos: https://swim2real.github.io.”
Exact source · revision 1Source owner not reported
Reported quantity · error reduction (paired comparison): 63.4 other · Basis not reported · Differing source assertions
“hether the proposals carry useful directions, we compare them with random and signed-permuted directions under the same line search and simulator budget. In an independent, preregistered fifty-seed replication, Swim2Real reduces the mean best-so-far error over 40 simulator evaluations by 63.4 mm against random directions and 53.2 mm against permuted directions. Both paired comparisons pass Holm correction ($p<10^{-5}”
Exact source · revision 1Source owner not reported
Reported quantity · error reduction (paired comparison): 53.2 other · Basis not reported · Differing source assertions
“hether the proposals carry useful directions, we compare them with random and signed-permuted directions under the same line search and simulator budget. In an independent, preregistered fifty-seed replication, Swim2Real reduces the mean best-so-far error over 40 simulator evaluations by 63.4 mm against random directions and 53.2 mm against permuted directions. Both paired comparisons pass Holm correction ($p<10^{-5}”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalSwim2Real: Vision-Language Models for Simulator Calibration of a Soft Robotic Fish
This research introduces Swim2Real, a method that uses vision-language models (VLMs) to calibrate simulators for soft aquatic robots. By comparing real and simulated swimming videos, the system proposes updates to 16 simulator parameters using backtracking line search. In an independent preregistered replication with fifty seeds, Swim2Real reduced the mean best-so-far error over 40 simulator evaluations by 63.4 mm compared to random directions and 53.2 mm compared to signed-permuted directions, with both paired comparisons passing Holm correction (p<10^-5). A second VLM in a 24-seed experiment supported the same direction effect. The method averaged 16.1 VLM proposals per run versus 39 for Vid2Sid while attaining lower error at the same simulator budget. Ten hardware trials demonstrated open-loop execution of simulator-generated motor commands.
Event coverage history
There is not enough continuous observation data to show a trend.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).