SomaAlign: Reliability-Aware Multimodal Representation Learning for Soft Robotic Grasping
Overview
This paper introduces Soma, a multimodal dataset for soft robotic grasping with 100 objects and 74,914 samples across RGB, depth, tactile, IMU, and text modalities. The authors also release Soma-Success, an independent grasp-outcome dataset of 1,000 trials used to evaluate frozen-encoder transfer for post-hoc classification and early prediction. SomaAlign is a representation learning framework built on Soma that uses masked reconstruction, reliability-aware contrastive alignment, and conflict-aware semantic constraints. On 30 cross-modal retrieval tasks, SomaAlign achieves an average Recall@10 of 90.66%. When applied to Soma-Success with lightweight probes, frozen SomaAlign encoders reach 94.59% post-hoc classification accuracy and 84.07% early prediction accuracy. The work provides a foundation for robust multimodal perception in soft robotic manipulation.
Generated from attributed reports · Updated 2 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Reported quantity · trajectories: 1000 other · trajectory count
“arXiv:2611.02794v1 Announce Type: new Abstract: Robust multimodal perception for soft robotic grasping is challenged by modality-dependent noise, semantic ambiguity, and limited coordinated multisensory data. We introduce Soma, a dataset containing 100 objects, 1,000 continuous grasping trajectories, and 74,914 multimodal samples grouped by software acquisition cycle, spanning remote RGB, hand RGB, depth, tactile, IM”
Exact source · revision 1Source owner not reported
Reported quantity · samples: 74914 other · Basis not reported
“arXiv:2611.02794v1 Announce Type: new Abstract: Robust multimodal perception for soft robotic grasping is challenged by modality-dependent noise, semantic ambiguity, and limited coordinated multisensory data. We introduce Soma, a dataset containing 100 objects, 1,000 continuous grasping trajectories, and 74,914 multimodal samples grouped by software acquisition cycle, spanning remote RGB, hand RGB, depth, tactile, IM”
Exact source · revision 1Source owner not reported
Reported quantity · trials: 1000 other · reported trials
“U, and structured text observations. We also collect Soma-Success, an independent grasp-outcome dataset with 1,000 trials (998 retained after quality control), to evaluate frozen-encoder transfer for post-hoc grasp outcome classification and early grasp success prediction. Built on Soma, SomaAlign learns multimodal representations in three stages: masked reconstruction estimates relative modality reliability; reliabi”
Exact source · revision 1Source owner not reported
Reported quantity · Reported accuracy: 84.07 percent · Basis not reported
“on answering, cross-modal generation, and cross-dataset transfer to grasp outcome prediction. On Soma-Success, frozen SomaAlign encoders with lightweight probes achieve 94.59% post-hoc classification accuracy and 84.07% early prediction accuracy. Together, Soma, Soma-Success, and SomaAlign provide a data and representation-learning foundation for robust and transferable multimodal perception in soft robotic manipulat”
Exact source · revision 1Source owner not reported
Reported quantity · Reported recall: 90.66 percent · Basis not reported
“lity-aware contrastive alignment uses pair-specific adaptive temperatures to adjust alignment sharpness; and conflict-aware semantic constraints improve intra-sample cross-modal consistency while preserving global feature geometry. SomaAlign achieves an average Recall@10 of 90.66% across 30 directed cross-modal retrieval tasks. The learned representations also support physical attribute recognition, multimodal questi”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalSomaAlign: Reliability-Aware Multimodal Representation Learning for Soft Robotic Grasping
This paper introduces Soma, a multimodal dataset for soft robotic grasping with 100 objects and 74,914 samples across RGB, depth, tactile, IMU, and text modalities. The authors also release Soma-Success, an independent grasp-outcome dataset of 1,000 trials used to evaluate frozen-encoder transfer for post-hoc classification and early prediction. SomaAlign is a representation learning framework built on Soma that uses masked reconstruction, reliability-aware contrastive alignment, and conflict-aware semantic constraints. On 30 cross-modal retrieval tasks, SomaAlign achieves an average Recall@10 of 90.66%. When applied to Soma-Success with lightweight probes, frozen SomaAlign encoders reach 94.59% post-hoc classification accuracy and 84.07% early prediction accuracy. The work provides a foundation for robust multimodal perception in soft robotic manipulation.
Event coverage history
There is not enough continuous observation data to show a trend.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).