See What I Need, Dream What I Want: A Navigation Foundation Model on a Self-Constructed Representation Space
See What I Need, Dream What I Want: A Navigation Foundation Model on a Self-Constructed Representation Space
This paper presents RepresentNav, a navigation foundation model that constructs a self-constructed representation space using a compact three-layer transformer. It learns to see what it needs for navigation and predicts future scene representations through a joint-embedding predictive architecture, achieving state-of-the-art results on multiple benchmarks and physical robot navigation tasks.
RepresentNav introduces a novel approach to navigation by constructing a self-constructed representation space using a policy's own objectives, leading to significant performance improvements on benchmark tasks and physical robot navigation.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
ment ahead. The dream is trained end-to-end with the policy as a joint-embedding predictive architecture (JEPA), with a next-representation prediction loss and a regularizer on its targets. RepresentNav sets a new state of the art on VLNVerse, VLN-CE R2R, EVT-Bench, and two point-goal benchmarks, and its gains carry over to a physical robot amid clutter and moving people. On VLNVerse fine-grained, RepresentNav achiev
Open source S4
es 83.80% success, outperforming the previous state of the art by 20.05 percentage points, and on EVT-Bench distracted tracking it reaches 85.5% with a single camera (+10.1). With the same encoders, the self-constructed space clearly outperforms concatenating their outputs and learns to weight their layers unevenly, and a dream in it cuts collisions by a quarter. A single dreamed latent decodes into the depth, semant
Open source S5
ics, and 3D occupancy of the scene a moment ahead, more accurately than pretrained encoders applied to the real future frame, although occupancy is never supervised, and the rollout tracks the robot's own motion over the next second: the dream emerges as a compact latent world model.
Open source S6
Source：arXiv Robotics — research abstracts · arxiv.org