EgoMatrix: Physically Faithful Dexterous Robot Demonstrations from Egocentric Human Videos
EgoMatrix: Physically Faithful Dexterous Robot Demonstrations from Egocentric Human Videos
EgoMatrix is an end-to-end data engine that converts monocular egocentric human videos into bimanual dexterous trajectories and paired robot observation videos. The method focuses on relative geometry and contact reconstruction, and uses reinforcement learning to reproduce tasks in simulation.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- Environment
- SimulationOpen source S4
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
bot hands and objects from these simulated trajectories to produce physically faithful observations synchronized with state and control records. Rendering the same trajectories in diverse, task-compatible backgrounds expands paired training data while preserving robot-object motion and contact. Across TACO, OakInk2, and self-collected data, EgoMatrix achieves 54.84% end-to-end success in simulation versus 14.52% for
Open source S4
the strongest baseline. Reconstruction reaches state-of-the-art performance on most metrics of the two public benchmarks. Qualitative results show more faithful hand-object interactions than prior methods, while feature-space analysis shows closer alignment with real-robot observations than GPT-6 Astra. Our project page is available at https://spikew726.github.io/EgoMatrix/.
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org