Visuo-tactile Diffusion Policy for Robotic Manipulation with Auxiliary Tactile Dynamics Prediction
Visuo-tactile Diffusion Policy for Robotic Manipulation with Auxiliary Tactile Dynamics Prediction
This research introduces a visuotactile diffusion policy that combines four-step histories from two camera views and two fingertip tactile images to condition action-sequence generation for long-horizon robotic assembly. The model uses a recurrent temporal representation integrating recent observations from both modalities and an auxiliary objective that predicts fingertip-image changes two steps ahead during training to capture how contact evolves under action. Across three independent training seeds in MuJoCo assembly experiments, the full model achieves an average success rate of 59.0% ± 2.6%, improving on the visual-only baseline (42.7% ± 1.5%) by 16.3 percentage points. The auxiliary objective adds no inference-time computation.
The study presents a visuotactile diffusion policy trained on MuJoCo assembly tasks, achieving an average success rate of 59.0% ± 2.6% across three independent seeds, which improves upon a visual-only baseline of 42.7% ± 1.5%; the auxiliary tactile dynamics prediction objective is used only during training and adds no inference-time computation.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
standard deviation (full model)
2.6
View original evidence
-time computation. Across three independent training seeds in MuJoCo assembly experiments, the full model achieves an average success rate of $59.0\% \pm 2.6\%$, improving on the visual-only baseline ($42.7\% \pm 1.5\%$) by 16.3 percentage points.
Open source S4standard deviation (baseline)
1.5
View original evidence
-time computation. Across three independent training seeds in MuJoCo assembly experiments, the full model achieves an average success rate of $59.0\% \pm 2.6\%$, improving on the visual-only baseline ($42.7\% \pm 1.5\%$) by 16.3 percentage points.
Open source S4
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
-time computation. Across three independent training seeds in MuJoCo assembly experiments, the full model achieves an average success rate of $59.0\% \pm 2.6\%$, improving on the visual-only baseline ($42.7\% \pm 1.5\%$) by 16.3 percentage points.
Open source S4
Source：arXiv Robotics — research abstracts · arxiv.org