Pixel-UMI: Learning Environment-Aware Whole-Body Manipulation from Sensorized Human Demonstrations
Pixel-UMI: Learning Environment-Aware Whole-Body Manipulation from Sensorized Human Demonstrations
This paper introduces Pixel-UMI, a framework for learning environment-aware whole-body manipulation policies from sensorized human demonstrations. The method employs Environment-Aware Retargeting (EAR) to align robot posture with demonstrated trajectories using reconstructed scene geometry, and Retargeting-Error-Aware Masking (REAM) to mitigate viewpoint mismatches during training. Additionally, on-policy human corrections via DAgger are collected to handle contact-rich interactions. Evaluated across five challenging tasks including articulated-object manipulation, deformable-object handling, constrained-space reaching, whole-body collision avoidance, and bimanual coordination, Pixel-UMI achieves an average task progress of 87.0%.
The authors present Pixel-UMI, a pipeline for learning whole-body manipulation policies from sensorized human demonstrations, incorporating Environment-Aware Retargeting (EAR), Retargeting-Error-Aware Masking (REAM), and DAgger-based corrections; evaluation on five tasks reports an average task progress of 87.0%.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
nation, achieving 87.0% average task progress. Video results are available at https://pixel-robot.com/pixel-umi
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org