DexCanvas: Bridging Human Demonstrations and Robot Learning for Dexterous Manipulation
DexCanvas: Bridging Human Demonstrations and Robot Learning for Dexterous Manipulation
This paper presents DexCanvas, a hybrid real-synthetic dataset containing 7,000 hours of dexterous hand-object interaction with per-frame contact points and forces. The dataset is generated from 70 hours of optical motion capture spanning 21 manipulation types using a real-to-sim pipeline where an RL policy drives an actuated MANO hand in physics simulation to reproduce captured trajectories. Key results include an 87.5% reproduction rate of demonstrations, 80.9% success on perturbed rollouts, and 72.5% success on six unseen YCB objects. A Diffusion Policy using DexCanvas contact features achieves 82.2% success compared to 67.5% without them over 18,522 trials. The dataset is available on Hugging Face.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
Reported duration
7,000 hours
View original evidence
arXiv:2510.15786v3 Announce Type: replace Abstract: A dexterous hand succeeds or fails through contact, yet every scalable record of human manipulation (video, vision-based hand tracking, motion capture) observes how the hand moves and misses how it pushes. We present DexCanvas, a hybrid real-synthetic dataset of 7,000 hours of dexterous hand-object interaction with per-frame contact points and contact forces, grown
Open source S2Reported duration
70 hours
View original evidence
from 70 hours of optical motion capture that spans 21 manipulation types, from power and precision grasps to in-hand manipulation. The forces come from a real-to-sim pipeline: one reinforcement learning (RL) policy, conditioned on object geometry and manipulation type, drives an actuated MANO hand in physics simulation to reproduce each captured object trajectory, and the simulator measures the contacts that produce
Open source S3trajectories
111,857
Trajectory count
View original evidence
e time, against 67.5% without them, over 18,522 trials. Contact-driven retargeting carries the same demonstrations to two dexterous robot hands, where the RL stage again produces stable rollouts. A first release of 111,857 force-annotated trajectories is publicly available on Hugging Face.
Open source S5
Dataset details reported by the source
- Modalities
- video Open source S2vision-based hand tracking Open source S2motion capture Open source S2
- Tasks
- dexterous hand-object interaction Open source S2precision grasps Open source S3in-hand manipulation Open source S3
- Embodiments
- MANO hand Open source S3
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
What remains unknown
Dataset licensing and commercial-use rights are not established here.
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
arXiv:2510.15786v3 Announce Type: replace Abstract: A dexterous hand succeeds or fails through contact, yet every scalable record of human manipulation (video, vision-based hand tracking, motion capture) observes how the hand moves and misses how it pushes. We present DexCanvas, a hybrid real-synthetic dataset of 7,000 hours of dexterous hand-object interaction with per-frame contact points and contact forces, grown
Open source S2
from 70 hours of optical motion capture that spans 21 manipulation types, from power and precision grasps to in-hand manipulation. The forces come from a real-to-sim pipeline: one reinforcement learning (RL) policy, conditioned on object geometry and manipulation type, drives an actuated MANO hand in physics simulation to reproduce each captured object trajectory, and the simulator measures the contacts that produce
Open source S3
the reproduced motion. The policy reproduces 87.5% of the demonstrations and still succeeds on 80.9% of rollouts from perturbed initial object poses; these perturbed rollouts expand the 70 captured hours a hundredfold. On six YCB objects absent from the capture set it succeeds 72.5% of the time. The contact channels make better policies: a Diffusion Policy that observes DexCanvas contact features succeeds 82.2% of th
Open source S4
e time, against 67.5% without them, over 18,522 trials. Contact-driven retargeting carries the same demonstrations to two dexterous robot hands, where the RL stage again produces stable rollouts. A first release of 111,857 force-annotated trajectories is publicly available on Hugging Face.
Open source S5
Implications for data suppliers
RoboSignal interpretation and collection questions, not statements of buyer demand.
- Confirm the required data type and collection setting with the buyer; this source does not establish a complete collection specification.
- Compare the reported units and scope before using these quantities in a budget. Recording hours, sensor-hours and trajectories are different measures.
- Check the exact dataset version, license, modalities and task coverage against the intended use.
- Validate demand and acceptance criteria with a buyer before scaling. Publication, popularity and a research result do not establish a purchase commitment.
Source：arXiv Robotics — research abstracts · arxiv.org