Rhythmic-Semantic Co-Speech Gestures for Emotional Dialogue in Robots
Rhythmic-Semantic Co-Speech Gestures for Emotional Dialogue in Robots
This paper introduces an emotion-aware system that generates both iconic and beat co-speech gestures in real time from text and the robot's internal emotional state. The authors formalize beat-iconic gesture generation using emotion, propose lightweight models to predict timing and intensity, and fill remaining spans with emotion-conditioned beats. Trained on the BEAT2 dataset and deployed on the social robot Haru, the system was evaluated in an online user study with 60 participants, showing stronger speech-gesture alignment and higher preference compared to baseline methods.
The authors present a system generating rhythmic-semantic co-speech gestures conditioned on text and internal emotional state, trained on the BEAT2 dataset and evaluated via an online user study with 60 participants; claims of improved speech-gesture alignment and preference over baselines are attributed to this specific study.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
emotional state. We formalize the task of beat-iconic gesture generation using emotion and propose lightweight models that predict iconic timing and intensity, then fill the remaining spans with emotion-conditioned beats at an appropriate intensity level. We train on the BEAT2 dataset and deploy the system on the social robot Haru. In an online user study with 60 participants, our method shows stronger speech-gestur
Open source S3
e alignment and is preferred over baseline methods.
Open source S4
Rhythmic-Semantic Co-Speech Gestures for Emotional Dialogue in Robots
Open source S1
arXiv:2611.13472v1 Announce Type: new Abstract: Co-speech gestures can increase engagement and improve speech understanding. Yet, most data-driven robot systems generate either rhythmic beat-like motion or semantically meaningful iconic gestures, but not both. We present an emotion-aware rhythmic-semantic co-speech gesture system that generates iconic and beat behaviors in real time from text and the robot's internal
Open source S2
Source：arXiv Robotics — research abstracts · arxiv.org