One Agent to Guide Them All: Empowering MLLMs for Vision-and-Language Navigation via Explicit World Representation
Overview
This research introduces GTA, a decoupled navigation framework that separates low-level spatial state estimation from high-level semantic planning. It utilizes an explicit metric world representation combining accumulated geometry with navigation history and employs counterfactual branch reasoning to revise decisions based on discrepancies between anticipated outcomes and observations. The method grounds waypoint selection in reconstructed metric geometry and demonstrates transferability across wheeled, quadrupedal, and aerial robots. GTA achieves a 48.8% Success Rate (SR) on the full R2R-CE val-unseen split and 46.2% on a 260-episode English RxR-CE val-unseen subset.
Generated from attributed reports · Updated 2 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Reported quantity · Reported success rate: 48.8 percent · reported trials · Differing source assertions
“structed metric geometry. GTA achieves 48.8% Success Rate (SR) on the full R2R-CE val-unseen split and 46.2% on a 260-episode English RxR-CE val-unseen subset. Additional comparisons on shared R2R-CE episodes and mechanism ablations support the framework's effectiveness. We also demonstrate transfer to wheeled, quadrupedal, and aerial robots. Project page: https://zeruili22.github.io/GTA/”
Exact source · revision 1Source owner not reported
Reported quantity · Reported success rate: 46.2 percent · reported trials · Differing source assertions
“structed metric geometry. GTA achieves 48.8% Success Rate (SR) on the full R2R-CE val-unseen split and 46.2% on a 260-episode English RxR-CE val-unseen subset. Additional comparisons on shared R2R-CE episodes and mechanism ablations support the framework's effectiveness. We also demonstrate transfer to wheeled, quadrupedal, and aerial robots. Project page: https://zeruili22.github.io/GTA/”
Exact source · revision 1Source owner not reported
Reported quantity · Number of episodes in RxR-CE val-unseen subset: 260 other · Basis not reported
“structed metric geometry. GTA achieves 48.8% Success Rate (SR) on the full R2R-CE val-unseen split and 46.2% on a 260-episode English RxR-CE val-unseen subset. Additional comparisons on shared R2R-CE episodes and mechanism ablations support the framework's effectiveness. We also demonstrate transfer to wheeled, quadrupedal, and aerial robots. Project page: https://zeruili22.github.io/GTA/”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalOne Agent to Guide Them All: Empowering MLLMs for Vision-and-Language Navigation via Explicit World Representation
This research introduces GTA, a decoupled navigation framework that separates low-level spatial state estimation from high-level semantic planning. It utilizes an explicit metric world representation combining accumulated geometry with navigation history and employs counterfactual branch reasoning to revise decisions based on discrepancies between anticipated outcomes and observations. The method grounds waypoint selection in reconstructed metric geometry and demonstrates transferability across wheeled, quadrupedal, and aerial robots. GTA achieves a 48.8% Success Rate (SR) on the full R2R-CE val-unseen split and 46.2% on a 260-episode English RxR-CE val-unseen subset.
Event coverage history
There is not enough continuous observation data to show a trend.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).