NavGPT-3: Harnessing Context in a Hierarchical Navigation Runtime

NavGPT-3 is a new system that connects language models with action policies for autonomous navigation. It uses a hierarchical runtime architecture to enable reasoning, acting, and monitoring in parallel, with the action policy trained on 19.28M examples. The system achieves state-of-the-art results on R2R-CE and matches human performance on RxR-CE, demonstrating the potential of integrating language models with physical control.