SomaAlign: Reliability-Aware Multimodal Representation Learning for Soft Robotic Grasping
SomaAlign: Reliability-Aware Multimodal Representation Learning for Soft Robotic Grasping
This paper introduces Soma, a multimodal dataset for soft robotic grasping with 100 objects and 74,914 samples across RGB, depth, tactile, IMU, and text modalities. The authors also release Soma-Success, an independent grasp-outcome dataset of 1,000 trials used to evaluate frozen-encoder transfer for post-hoc classification and early prediction. SomaAlign is a representation learning framework built on Soma that uses masked reconstruction, reliability-aware contrastive alignment, and conflict-aware semantic constraints. On 30 cross-modal retrieval tasks, SomaAlign achieves an average Recall@10 of 90.66%. When applied to Soma-Success with lightweight probes, frozen SomaAlign encoders reach 94.59% post-hoc classification accuracy and 84.07% early prediction accuracy. The work provides a foundation for robust multimodal perception in soft robotic manipulation.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
trajectories
1,000
Trajectory count
View original evidence
arXiv:2611.02794v1 Announce Type: new Abstract: Robust multimodal perception for soft robotic grasping is challenged by modality-dependent noise, semantic ambiguity, and limited coordinated multisensory data. We introduce Soma, a dataset containing 100 objects, 1,000 continuous grasping trajectories, and 74,914 multimodal samples grouped by software acquisition cycle, spanning remote RGB, hand RGB, depth, tactile, IM
Open source S2samples
74,914
View original evidence
arXiv:2611.02794v1 Announce Type: new Abstract: Robust multimodal perception for soft robotic grasping is challenged by modality-dependent noise, semantic ambiguity, and limited coordinated multisensory data. We introduce Soma, a dataset containing 100 objects, 1,000 continuous grasping trajectories, and 74,914 multimodal samples grouped by software acquisition cycle, spanning remote RGB, hand RGB, depth, tactile, IM
Open source S2trials
1,000 trials
Reported trials
View original evidence
U, and structured text observations. We also collect Soma-Success, an independent grasp-outcome dataset with 1,000 trials (998 retained after quality control), to evaluate frozen-encoder transfer for post-hoc grasp outcome classification and early grasp success prediction. Built on Soma, SomaAlign learns multimodal representations in three stages: masked reconstruction estimates relative modality reliability; reliabi
Open source S3Reported accuracy
84.07%
View original evidence
on answering, cross-modal generation, and cross-dataset transfer to grasp outcome prediction. On Soma-Success, frozen SomaAlign encoders with lightweight probes achieve 94.59% post-hoc classification accuracy and 84.07% early prediction accuracy. Together, Soma, Soma-Success, and SomaAlign provide a data and representation-learning foundation for robust and transferable multimodal perception in soft robotic manipulat
Open source S5Reported recall
90.66%
View original evidence
lity-aware contrastive alignment uses pair-specific adaptive temperatures to adjust alignment sharpness; and conflict-aware semantic constraints improve intra-sample cross-modal consistency while preserving global feature geometry. SomaAlign achieves an average Recall@10 of 90.66% across 30 directed cross-modal retrieval tasks. The learned representations also support physical attribute recognition, multimodal questi
Open source S4
- paper: not_reported
- dataset: not_reported
- code: not_reported
- weights: not_reported
What remains unknown
Not established in the collected evidence: Environment, Control, Data origin.
Dataset licensing and commercial-use rights are not established here.
Reported performance applies to the described task. It does not establish general autonomy or deployment readiness.
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
arXiv:2611.02794v1 Announce Type: new Abstract: Robust multimodal perception for soft robotic grasping is challenged by modality-dependent noise, semantic ambiguity, and limited coordinated multisensory data. We introduce Soma, a dataset containing 100 objects, 1,000 continuous grasping trajectories, and 74,914 multimodal samples grouped by software acquisition cycle, spanning remote RGB, hand RGB, depth, tactile, IM
Open source S2
U, and structured text observations. We also collect Soma-Success, an independent grasp-outcome dataset with 1,000 trials (998 retained after quality control), to evaluate frozen-encoder transfer for post-hoc grasp outcome classification and early grasp success prediction. Built on Soma, SomaAlign learns multimodal representations in three stages: masked reconstruction estimates relative modality reliability; reliabi
Open source S3
on answering, cross-modal generation, and cross-dataset transfer to grasp outcome prediction. On Soma-Success, frozen SomaAlign encoders with lightweight probes achieve 94.59% post-hoc classification accuracy and 84.07% early prediction accuracy. Together, Soma, Soma-Success, and SomaAlign provide a data and representation-learning foundation for robust and transferable multimodal perception in soft robotic manipulat
Open source S5
lity-aware contrastive alignment uses pair-specific adaptive temperatures to adjust alignment sharpness; and conflict-aware semantic constraints improve intra-sample cross-modal consistency while preserving global feature geometry. SomaAlign achieves an average Recall@10 of 90.66% across 30 directed cross-modal retrieval tasks. The learned representations also support physical attribute recognition, multimodal questi
Open source S4
Implications for data suppliers
RoboSignal interpretation and collection questions, not statements of buyer demand.
- Confirm the required data type and collection setting with the buyer; this source does not establish a complete collection specification.
- Compare the reported units and scope before using these quantities in a budget. Recording hours, sensor-hours and trajectories are different measures.
- Validate demand and acceptance criteria with a buyer before scaling. Publication, popularity and a research result do not establish a purchase commitment.
Source：arXiv Robotics — research abstracts · arxiv.org