VTAM: Efficient Visuo-Tactile World-Action Models for Contact-Rich Manipulation without Tactile Pretraining
Overview
This paper introduces VTAM, a visuo-tactile world-action model designed for contact-rich manipulation tasks that eliminates the need for tactile or action pretraining. By adapting a frozen video VAE backbone to jointly predict visual and tactile latents and fine-tuning an end-to-end action expert, VTAM outperforms baselines on four real-robot tasks across two platforms with different tactile sensors and achieves high success rates in UniVTAC simulations without full video denoising at inference. The model demonstrates zero-shot transferability between GelSight Mini and Xense Photon sensors while maintaining competitive latency compared to compiled versions of other models.
Generated from attributed reports · Updated 2 days ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Reported quantity · simulation tasks: 8 other · reported trials
“rom tactile optical flow. Trained only on task demonstrations, VTAM outperforms all baselines on four real-robot tasks across two platforms with different tactile sensors, outperforms all baselines without tactile pretraining in average success on eight UniVTAC simulation tasks, and is comparable to tactile VLAs and WAMs pretrained on over 30K hours of largely visuo-tactile robot data (83.4% vs. 84.5% for N0-TWAM). W”
Exact source · revision 1Source owner not reported
Reported quantity · training hours: 30000 hours · Basis not reported
“rom tactile optical flow. Trained only on task demonstrations, VTAM outperforms all baselines on four real-robot tasks across two platforms with different tactile sensors, outperforms all baselines without tactile pretraining in average success on eight UniVTAC simulation tasks, and is comparable to tactile VLAs and WAMs pretrained on over 30K hours of largely visuo-tactile robot data (83.4% vs. 84.5% for N0-TWAM). W”
Exact source · revision 1Source owner not reported
Reported quantity · speedup factor: 6 other · Basis not reported · Differing source assertions
“ithout full video denoising at inference, VTAM is approximately 6x and 16x faster than N0-TWAM and LingBot-VA without compilation, with compiled latency comparable to Fast-WAM. Ablations show that feeding frozen-VAE tactile features directly to the action expert, bypassing tactile world modeling, fails on every tested task, that Stage I visuo-tactile world-model fine-tuning improves UniVTAC success on most tasks, and”
Exact source · revision 1Source owner not reported
Reported quantity · speedup factor: 16 other · Basis not reported · Differing source assertions
“ithout full video denoising at inference, VTAM is approximately 6x and 16x faster than N0-TWAM and LingBot-VA without compilation, with compiled latency comparable to Fast-WAM. Ablations show that feeding frozen-VAE tactile features directly to the action expert, bypassing tactile world modeling, fails on every tested task, that Stage I visuo-tactile world-model fine-tuning improves UniVTAC success on most tasks, and”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalVTAM: Efficient Visuo-Tactile World-Action Models for Contact-Rich Manipulation without Tactile Pretraining
This paper introduces VTAM, a visuo-tactile world-action model designed for contact-rich manipulation tasks that eliminates the need for tactile or action pretraining. By adapting a frozen video VAE backbone to jointly predict visual and tactile latents and fine-tuning an end-to-end action expert, VTAM outperforms baselines on four real-robot tasks across two platforms with different tactile sensors and achieves high success rates in UniVTAC simulations without full video denoising at inference. The model demonstrates zero-shot transferability between GelSight Mini and Xense Photon sensors while maintaining competitive latency compared to compiled versions of other models.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).