DexRoam: Learning Mobile Bimanual Dexterous Manipulation from Egocentric Whole-Body Human Demonstrations
CoRL 2026｜移动双臂灵巧操作，也能从人类示范中学？
Accepted at CoRL 2026, DexRoam is a system developed by researchers from HKUST, the Beijing Institute for Brain Intelligence, and Peking University. It enables robots to learn complex mobile bimanual dexterous manipulation tasks from first-person human demonstrations using a lightweight capture setup with Meta Quest 3 headsets and ZED Mini cameras. The framework employs three-stage alignment (body, semantic, and temporal) to bridge human-robot differences without requiring external motion capture equipment. Experiments show that combining human demonstrations with robot data improves task success rates on GR00T N1.7 and π₀.₅ models across five tasks, allowing robots to learn new manipulation strategies not present in their original training data. The project's code for data collection, alignment, and policy training is open-sourced.
Source: Machine Heart — Robotics on WeChat · Read original article ↗
English translation unavailable. Choose the original article to read it.
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- Data origin
- Human egocentric dataOpen source S10
Reported numbers
Reported success rate
29%
Reported trials
View original evidence
其中，「人类预训练＋人机联合微调」表现最佳，GR00T N1.7 的平均成功率从 29%提升至 56% ，π₀.₅从 32%提升至 57% 。
Open source S23Reported success rate
56%
Reported trials
View original evidence
其中，「人类预训练＋人机联合微调」表现最佳，GR00T N1.7 的平均成功率从 29%提升至 56% ，π₀.₅从 32%提升至 57% 。
Open source S23
Dataset details reported by the source
- Embodiments
- GR00T N1.7 Open source S19π₀.₅ Open source S19
- paper: not_reported
- code: Available Artifact link Open source S10
- code: Available Artifact link Open source S10
- code: not_reported
- dataset: not_reported
What remains unknown
Not established in the collected evidence: Environment, Control.
Dataset licensing and commercial-use rights are not established here.
Reported performance applies to the described task. It does not establish general autonomy or deployment readiness.
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
论文标题：DexRoam: Learning Mobile Bimanual Dexterous Manipulation from Egocentric Whole-Body Human Demonstrations arxiv 链接：https://arxiv.org/abs/2609.35761 项目主页：https://dexroam.github.io/ 采集对齐代码 链接：https://github.com/zhourui9813/DexRoam 策略训练代码链接：https://github.com/zhourui9813/DexRoam-Policy-Training 方法框架：从人类全身示范到机器人策略学习 轻量化采集人类与机器人全身示范 图 1 DexRoam 数据采集系统 DexRoam 构建了一套无需外部动捕设备的便携式第一视角全身采集系统。
Open source S10
两套系统分别提供人类与机器人示范，为后续联合学习建立数据基础。
Open source S12
围绕这一问题，香港科技大学、北京智源研究院、北京大学等机构的联合研究团队提出了 DexRoam ，构建了一套从第一视角人类示范采集、全身动作对齐，到机器人策略学习与真机执行的完整系统。
Open source S8
团队采用 GR00T N1.7 和π₀.₅ 两个 VLA 模型，并探索三种训练策略： 人机联合训练、人类预训练后使用机器人数据微调，以及人类预训练后进行人机联合微调 ，验证这一框架在不同模型与训练方式下的有效性。
Open source S19
其中，「人类预训练＋人机联合微调」表现最佳，GR00T N1.7 的平均成功率从 29%提升至 56% ，π₀.₅从 32%提升至 57% 。
Open source S23
图 4 对齐前后人机动作分布差异的 MMD 与 SWD 指标对比 定量评估中，MMD 和 SWD 指标分别平均降低了 41.7%和 31.1% ，且转换后的人类轨迹能够在真机上成功回放。
Open source S21
此外，团队还搭建了一套单人全身遥操作系统，通过 VR 头显、控制器与数据手套，即可同步控制机器人底盘、躯干、双臂、头部和灵巧手。
Open source S11
Implications for data suppliers
RoboSignal interpretation and collection questions, not statements of buyer demand.
- Check whether the intended buyer needs human observations, hand or object annotations, or aligned robot actions. Human video alone does not provide robot commands.
- Compare the reported units and scope before using these quantities in a budget. Recording hours, sensor-hours and trajectories are different measures.
- Check the exact dataset version, license, modalities and task coverage against the intended use.
- Validate demand and acceptance criteria with a buyer before scaling. Publication, popularity and a research result do not establish a purchase commitment.
Source：Machine Heart — Robotics on WeChat · mp.weixin.qq.com