Splat2Act: From Captured Scenes to Personalized VLA Manipulation
Splat2Act: From Captured Scenes to Personalized VLA Manipulation
Splat2Act is an automated real-to-sim-to-real pipeline that personalizes a pretrained vision-language-action (VLA) policy to a specific scene and object set using only a handheld phone scan and images of the user's objects. The method constructs photorealistic digital twins with 3D Gaussian Splatting and collision geometry, automatically discovers receptacles, and generates successful demonstrations. Applied to 101 campus environments, the pipeline yields 852 receptacles and over 68,000 pickup demonstrations. The strongest personalized policy achieves 62.5% aggregate pickup success in the target scene, with 63.9% success on a real robot.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
navigable area
104.87
receptacles
852
tasks
12
scenes
101
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
atting and collision geometry, automatically discovers tabletop and non-tabletop receptacles, constructs feasible pickup tasks, and generates successful demonstrations with a privileged simulation oracle, with only scene and object capture required from a human. Applied to 101 scanned campus environments with a median navigable area of 104.87 m$^2$ and 7 receptacles, the pipeline yields 852 receptacles and over 68,00
Open source S4
0 pickup demonstrations. We fine-tune $\pi_{0.5}$ in two stages: Stage 1 adapts to hundreds of public object assets, drawn either from the deployment scene alone or from all 101 scenes; Stage 2 personalizes to the user's own objects on the deployment scene's receptacles. Our strongest personalized policy reaches 62.5% aggregate pickup success in the target scene. Deployed on a real Stretch 3, the same policy achieves
Open source S5
63.9% success on a 12-task subset, closely tracking its 66.7% simulated success on those tasks. These results suggest that automated real-to-sim-to-real training can personalize VLA manipulation policies using only simulation-oracle data.
Open source S6
Source：arXiv Robotics — research abstracts · arxiv.org