Multimodal Socially Aware Framework for Human-Robot Interaction in Virtual Reality
Multimodal Socially Aware Framework for Human-Robot Interaction in Virtual Reality
This research presents a multimodal framework that integrates speech, vision, and touch to enable contextually appropriate robot responses. Evaluated in virtual reality with 18 participants across five social scenarios, the autonomous system was rated above neutral for naturalness and social appropriateness but received significantly lower scores than a human-teleoperated baseline. Analysis identified conversational timing, clarification of ambiguous requests, and spatial coordination as key factors contributing to the performance gap, highlighting challenges in developing autonomous robots capable of natural multimodal social interaction.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
scenarios
5
View original evidence
arXiv:2610.34534v2 Announce Type: replace Abstract: Social robots must coordinate speech and movement while interpreting multimodal human input. We present a multimodal framework that integrates speech, vision, and touch to generate contextually appropriate robot speech and movement. We evaluated the framework in virtual reality with 18 participants across five social interaction scenarios, comparing its performance
Open source S2
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
arXiv:2610.34534v2 Announce Type: replace Abstract: Social robots must coordinate speech and movement while interpreting multimodal human input. We present a multimodal framework that integrates speech, vision, and touch to generate contextually appropriate robot speech and movement. We evaluated the framework in virtual reality with 18 participants across five social interaction scenarios, comparing its performance
Open source S2
against a human-teleoperated baseline. Although participants rated the autonomous system above the neutral midpoint for both naturalness and social appropriateness, the teleoperated baseline received significantly higher ratings on both measures. Behavioral coding and qualitative feedback revealed that conversational timing, clarification of ambiguous requests, and spatial coordination were important factors contribu
Open source S3
Source：arXiv Robotics — research abstracts · arxiv.org