ReGATE: Recurrent Gated Visual Fusion for Reactive Humanoid-Object Interaction
ReGATE: Recurrent Gated Visual Fusion for Reactive Humanoid-Object Interaction
This paper introduces ReGATE, a whole-body controller combining egocentric depth, proprioception, and online locomotion commands for humanoids. Using a teacher-student framework with recurrent adapters, it estimates object geometry and contact without external pose estimators. Head-wise and global gates regulate visual updates, while curriculum training and phase-aware initialization improve performance. In simulation, ReGATE achieves 85.2% success in static-object acquisition (330 trials) versus 48.5% for VisualMimic, and 85.9% in moving-object trials (440). Deployment on a Unitree G1 shows successful grasp-transport-release sequences and box acquisition despite camera occlusion. This is the first work to extend recurrent gated fusion from perceptive locomotion to humanoid loco-manipulation under interrupted visual sensing.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- Environment
- SimulationOpen source S3
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
s interaction-state estimation through a recurrent adapter that predicts object geometry and physical contact without an external object-pose estimator. Head-wise and global gates regulate visual updates to a proprioceptive pathway. A visual-grounding and corruption curriculum trains the adapter, while phase-aware initialization increases coverage of brief contact transitions. In simulation, ReGATE achieves 85.2% suc
Open source S3
cess across 330 static-object acquisition trials, compared with 48.5% for VisualMimic under the same protocol, and 85.9% across 440 moving-object acquisition trials. Deployment on a Unitree G1 demonstrates commanded grasp-transport-release sequences and acquisition of laterally displaced boxes. A qualitative hardware experiment further demonstrates successful box acquisition when the camera is occluded after an initi
Open source S4
al observation and before the contact command. To the best of our knowledge, this is the first work to extend recurrent gated fusion of onboard depth and proprioception from perceptive locomotion to humanoid loco-manipulation under interrupted visual sensing.
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org