LocalNav: Distilling Frontier VLMs and Embodied RL for On-Device Object Goal Navigation
LocalNav: Distilling Frontier VLMs and Embodied RL for On-Device Object Goal Navigation
This paper introduces LocalNav, a method that distills frontier Vision Language Models (VLMs) and applies Embodied Reinforcement Learning from Verifiable Rewards (E-RLVR) to enable low-latency object goal navigation on resource-constrained embedded GPUs like the Jetson Orin. While cloud-based models achieve 41.61% success on the HM3D OVON benchmark, a zero-shot local deployment of Qwen3.5-4B only reaches 25.97%. By distilling reasoning traces and applying E-RLVR, the authors improve local performance to 37.36% SR. Further reducing token generation latency through regularization yields a final success rate of 35.53% with a cumulative 80.21% reduction in overall inference latency when combined with quantization, offering a viable path for local VLM execution on mobile robots.
The authors propose a distillation and fine-tuning strategy to deploy a lightweight Vision Language Model on edge devices for object goal navigation, achieving a 35.53% success rate with a 69.29% latency reduction compared to cloud inference baselines.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
Reported success rate
41.61%
Reported zero-shot result; the excerpt does not specify an evaluation protocol.
Reported trials
View original evidence
on resource-constrained robots. To address this challenge, we present a distillation strategy that aligns a lightweight, 4B- parameter local VLM to our spatial-semantic navigation interface via frontier model traces for edge execution on embedded GPU devices (e.g., Jetson Orin). While cloud models establish a baseline achieving 41.61% Success Rate (SR) on the HM3D OVON benchmark, zero-shot deployment of our proposed
Open source S3
- paper: not_reported
What remains unknown
Not established in the collected evidence: Environment, Control, Data origin.
Reported performance applies to the described task. It does not establish general autonomy or deployment readiness.
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
on resource-constrained robots. To address this challenge, we present a distillation strategy that aligns a lightweight, 4B- parameter local VLM to our spatial-semantic navigation interface via frontier model traces for edge execution on embedded GPU devices (e.g., Jetson Orin). While cloud models establish a baseline achieving 41.61% Success Rate (SR) on the HM3D OVON benchmark, zero-shot deployment of our proposed
Open source S3
architecture with Qwen3.5-4B reaches only 25.97% SR. We demonstrate that distilling frontier reasoning traces and subsequent finetuning with Embodied Reinforcement Learning from Verifiable Rewards (E- RLVR) obtains a 37.36% SR, substantially narrowing the gap to large cloud models. However, while deployable on memory- constrained devices, the model cannot compete with round-trip cloud inference time: we identify Toke
Open source S4
n Generation (TG) as the main driver of latency and reduce it with Token Generation (TG)- regularized E-RLVR. The combined method reduces latency by 69.29% and retains a SR of 35.53%. Combined with quantization, this joint strategy yields a cumulative 80.21% reduction in overall inference latency without significantly sacrificing performance, presenting a viable paradigm for local, low-latency VLM execu- tion on mobi
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org