538 Hours of 'Extremely Dirty' Human Data: Hong Kong University Teaches Robots to Unbox, Use Hot Glue Guns, and Make Cold Drinks
538小时「极脏」人类数据：港大让机器人学会拆快递、打胶枪、做冷饮
A team from the MMLab at the University of Hong Kong has introduced EgoWild2Dex, a dataset and framework that uses raw, uncurated first-person human operation data to train robots on complex tasks such as unboxing, using a hot glue gun to fix figurines, and making cold drinks. The method achieves an average success rate of 96.7% by employing GeoFormer perspective alignment and action retargeting to transfer natural human data into the robot's action space, reducing reliance on laboratory environments.
The study presents EgoWild2Dex, a dataset and framework that leverages raw, uncurated human first-person video data to train robots on complex tasks. By addressing the mismatch between human and robot perspectives through GeoFormer and action retargeting, the approach enables efficient training with minimal real robot data, achieving 96.7% success in multi-step tasks.
Source: Chinese robotics — Dataset and collection discovery · Read original article ↗
English translation unavailable. Choose the original article to read it.
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
hours_of_data
538.9 hours
Reported success rate
96.7%
View original evidence
在每个下游任务使用不到 1 小时真机数据的情况下，机器人能够连续完成拆快递、使用热熔胶枪固定手办、制作冷饮等复杂双臂灵巧长时序任务，平均成功率达到 96.7%。
Open source S5head_rotation_per_second
15.93
object_instances_per_frame
34.78
Reported duration
<1 hours
View original evidence
在每个下游任务使用不到 1 小时真机数据的情况下，机器人能够连续完成拆快递、使用热熔胶枪固定手办、制作冷饮等复杂双臂灵巧长时序任务，平均成功率达到 96.7%。
Open source S5
- code: not_reported
- dataset: not_reported
- paper: not_reported
- weights: not_reported
- code: Reported pending Open source S8
- dataset: Reported pending Open source S8
What remains unknown
Not established in the collected evidence: Environment, Control, Data origin.
Reported performance applies to the described task. It does not establish general autonomy or deployment readiness.
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
近日，团队提出 EgoWild2Dex，直接利用 538.9 小时未经刻意编排的真实人类第一视角操作数据训练机器人。
Open source S4
在每个下游任务使用不到 1 小时真机数据的情况下，机器人能够连续完成拆快递、使用热熔胶枪固定手办、制作冷饮等复杂双臂灵巧长时序任务，平均成功率达到 96.7%。
Open source S5
数据显示，EgoWild 中，每秒累计头部旋转角度平均达到 15.93° ，每帧图像平均包含 34.78 个不同的物体实例。
Open source S16
按传统机器人数据采集的标准来看，这批数据确实相当 「脏」 。
Open source S17
数据集、模型与代码计划后续开放。
Open source S8
Source：Chinese robotics — Dataset and collection discovery · cj.sina.com.cn