ReGATE: Recurrent Gated Visual Fusion for Reactive Humanoid-Object Interaction
Overview
This paper introduces ReGATE, a whole-body controller combining egocentric depth, proprioception, and online locomotion commands for humanoids. Using a teacher-student framework with recurrent adapters, it estimates object geometry and contact without external pose estimators. Head-wise and global gates regulate visual updates, while curriculum training and phase-aware initialization improve performance. In simulation, ReGATE achieves 85.2% success in static-object acquisition (330 trials) versus 48.5% for VisualMimic, and 85.9% in moving-object trials (440). Deployment on a Unitree G1 shows successful grasp-transport-release sequences and box acquisition despite camera occlusion. This is the first work to extend recurrent gated fusion from perceptive locomotion to humanoid loco-manipulation under interrupted visual sensing.
Generated from attributed reports · Updated 2 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Environment: simulation
“s interaction-state estimation through a recurrent adapter that predicts object geometry and physical contact without an external object-pose estimator. Head-wise and global gates regulate visual updates to a proprioceptive pathway. A visual-grounding and corruption curriculum trains the adapter, while phase-aware initialization increases coverage of brief contact transitions. In simulation, ReGATE achieves 85.2% suc”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalReGATE: Recurrent Gated Visual Fusion for Reactive Humanoid-Object Interaction
This paper introduces ReGATE, a whole-body controller combining egocentric depth, proprioception, and online locomotion commands for humanoids. Using a teacher-student framework with recurrent adapters, it estimates object geometry and contact without external pose estimators. Head-wise and global gates regulate visual updates, while curriculum training and phase-aware initialization improve performance. In simulation, ReGATE achieves 85.2% success in static-object acquisition (330 trials) versus 48.5% for VisualMimic, and 85.9% in moving-object trials (440). Deployment on a Unitree G1 shows successful grasp-transport-release sequences and box acquisition despite camera occlusion. This is the first work to extend recurrent gated fusion from perceptive locomotion to humanoid loco-manipulation under interrupted visual sensing.
Event coverage history
There is not enough continuous observation data to show a trend.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).