VTAM: Efficient Visuo-Tactile World-Action Models for Contact-Rich Manipulation without Tactile Pretraining
VTAM: Efficient Visuo-Tactile World-Action Models for Contact-Rich Manipulation without Tactile Pretraining
This paper introduces VTAM, a visuo-tactile world-action model designed for contact-rich manipulation tasks that eliminates the need for tactile or action pretraining. By adapting a frozen video VAE backbone to jointly predict visual and tactile latents and fine-tuning an end-to-end action expert, VTAM outperforms baselines on four real-robot tasks across two platforms with different tactile sensors and achieves high success rates in UniVTAC simulations without full video denoising at inference. The model demonstrates zero-shot transferability between GelSight Mini and Xense Photon sensors while maintaining competitive latency compared to compiled versions of other models.
VTAM achieves comparable success rates to tactile pretraining baselines without requiring tactile or action pretraining, demonstrating efficient inference and zero-shot transfer across different tactile sensors.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
simulation tasks
8
Reported trials
View original evidence
rom tactile optical flow. Trained only on task demonstrations, VTAM outperforms all baselines on four real-robot tasks across two platforms with different tactile sensors, outperforms all baselines without tactile pretraining in average success on eight UniVTAC simulation tasks, and is comparable to tactile VLAs and WAMs pretrained on over 30K hours of largely visuo-tactile robot data (83.4% vs. 84.5% for N0-TWAM). W
Open source S4training hours
>30,000 hours
View original evidence
rom tactile optical flow. Trained only on task demonstrations, VTAM outperforms all baselines on four real-robot tasks across two platforms with different tactile sensors, outperforms all baselines without tactile pretraining in average success on eight UniVTAC simulation tasks, and is comparable to tactile VLAs and WAMs pretrained on over 30K hours of largely visuo-tactile robot data (83.4% vs. 84.5% for N0-TWAM). W
Open source S4speedup factor
≈6
View original evidence
ithout full video denoising at inference, VTAM is approximately 6x and 16x faster than N0-TWAM and LingBot-VA without compilation, with compiled latency comparable to Fast-WAM. Ablations show that feeding frozen-VAE tactile features directly to the action expert, bypassing tactile world modeling, fails on every tested task, that Stage I visuo-tactile world-model fine-tuning improves UniVTAC success on most tasks, and
Open source S5speedup factor
16
View original evidence
ithout full video denoising at inference, VTAM is approximately 6x and 16x faster than N0-TWAM and LingBot-VA without compilation, with compiled latency comparable to Fast-WAM. Ablations show that feeding frozen-VAE tactile features directly to the action expert, bypassing tactile world modeling, fails on every tested task, that Stage I visuo-tactile world-model fine-tuning improves UniVTAC success on most tasks, and
Open source S5
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
rom tactile optical flow. Trained only on task demonstrations, VTAM outperforms all baselines on four real-robot tasks across two platforms with different tactile sensors, outperforms all baselines without tactile pretraining in average success on eight UniVTAC simulation tasks, and is comparable to tactile VLAs and WAMs pretrained on over 30K hours of largely visuo-tactile robot data (83.4% vs. 84.5% for N0-TWAM). W
Open source S4
ithout full video denoising at inference, VTAM is approximately 6x and 16x faster than N0-TWAM and LingBot-VA without compilation, with compiled latency comparable to Fast-WAM. Ablations show that feeding frozen-VAE tactile features directly to the action expert, bypassing tactile world modeling, fails on every tested task, that Stage I visuo-tactile world-model fine-tuning improves UniVTAC success on most tasks, and
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org