Pixel-UMI: Learning Environment-Aware Whole-Body Manipulation from Sensorized Human Demonstrations
Overview
This paper introduces Pixel-UMI, a framework for learning environment-aware whole-body manipulation policies from sensorized human demonstrations. The method employs Environment-Aware Retargeting (EAR) to align robot posture with demonstrated trajectories using reconstructed scene geometry, and Retargeting-Error-Aware Masking (REAM) to mitigate viewpoint mismatches during training. Additionally, on-policy human corrections via DAgger are collected to handle contact-rich interactions. Evaluated across five challenging tasks including articulated-object manipulation, deformable-object handling, constrained-space reaching, whole-body collision avoidance, and bimanual coordination, Pixel-UMI achieves an average task progress of 87.0%.
Generated from attributed reports · Updated 2 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
No current evidence-backed claims. Missing information remains not reported.
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalPixel-UMI: Learning Environment-Aware Whole-Body Manipulation from Sensorized Human Demonstrations
This paper introduces Pixel-UMI, a framework for learning environment-aware whole-body manipulation policies from sensorized human demonstrations. The method employs Environment-Aware Retargeting (EAR) to align robot posture with demonstrated trajectories using reconstructed scene geometry, and Retargeting-Error-Aware Masking (REAM) to mitigate viewpoint mismatches during training. Additionally, on-policy human corrections via DAgger are collected to handle contact-rich interactions. Evaluated across five challenging tasks including articulated-object manipulation, deformable-object handling, constrained-space reaching, whole-body collision avoidance, and bimanual coordination, Pixel-UMI achieves an average task progress of 87.0%.
Event coverage history
There is not enough continuous observation data to show a trend.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).