One Agent to Guide Them All: Empowering MLLMs for Vision-and-Language Navigation via Explicit World Representation
One Agent to Guide Them All: Empowering MLLMs for Vision-and-Language Navigation via Explicit World Representation
This research introduces GTA, a decoupled navigation framework that separates low-level spatial state estimation from high-level semantic planning. It utilizes an explicit metric world representation combining accumulated geometry with navigation history and employs counterfactual branch reasoning to revise decisions based on discrepancies between anticipated outcomes and observations. The method grounds waypoint selection in reconstructed metric geometry and demonstrates transferability across wheeled, quadrupedal, and aerial robots. GTA achieves a 48.8% Success Rate (SR) on the full R2R-CE val-unseen split and 46.2% on a 260-episode English RxR-CE val-unseen subset.
The paper proposes GTA, a navigation framework that decouples spatial state estimation from semantic planning using an explicit metric world representation and counterfactual reasoning, achieving 48.8% success rate on the full R2R-CE val-unseen split.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
Reported success rate
48.8%
Reported trials
View original evidence
structed metric geometry. GTA achieves 48.8% Success Rate (SR) on the full R2R-CE val-unseen split and 46.2% on a 260-episode English RxR-CE val-unseen subset. Additional comparisons on shared R2R-CE episodes and mechanism ablations support the framework's effectiveness. We also demonstrate transfer to wheeled, quadrupedal, and aerial robots. Project page: https://zeruili22.github.io/GTA/
Open source S4Reported success rate
46.2%
Reported trials
View original evidence
structed metric geometry. GTA achieves 48.8% Success Rate (SR) on the full R2R-CE val-unseen split and 46.2% on a 260-episode English RxR-CE val-unseen subset. Additional comparisons on shared R2R-CE episodes and mechanism ablations support the framework's effectiveness. We also demonstrate transfer to wheeled, quadrupedal, and aerial robots. Project page: https://zeruili22.github.io/GTA/
Open source S4Number of episodes in RxR-CE val-unseen subset
260
View original evidence
structed metric geometry. GTA achieves 48.8% Success Rate (SR) on the full R2R-CE val-unseen split and 46.2% on a 260-episode English RxR-CE val-unseen subset. Additional comparisons on shared R2R-CE episodes and mechanism ablations support the framework's effectiveness. We also demonstrate transfer to wheeled, quadrupedal, and aerial robots. Project page: https://zeruili22.github.io/GTA/
Open source S4
- paper: not_reported
What remains unknown
Not established in the collected evidence: Environment, Control, Data origin.
Reported performance applies to the described task. It does not establish general autonomy or deployment readiness.
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
structed metric geometry. GTA achieves 48.8% Success Rate (SR) on the full R2R-CE val-unseen split and 46.2% on a 260-episode English RxR-CE val-unseen subset. Additional comparisons on shared R2R-CE episodes and mechanism ablations support the framework's effectiveness. We also demonstrate transfer to wheeled, quadrupedal, and aerial robots. Project page: https://zeruili22.github.io/GTA/
Open source S4
Source：arXiv Robotics — research abstracts · arxiv.org