Surrogate Uncertainty for Black-box Visual Grounding
Surrogate Uncertainty for Black-box Visual Grounding
This research paper introduces a method to address uncertainty in visual grounding tasks where robots use black-box vision-language models (VLMs). The authors propose a formal Bayesian definition treating VLM predictions as observations, utilizing an external surrogate Bayesian model to quantify per-query uncertainty over local neighborhoods. This approach decomposes uncertainty into aleatoric and epistemic components and integrates with conformal prediction or model-based planning for active visual grounding. Evaluated across benchmarks and state-of-the-art VLMs, the method improves calibration and task performance. On a real Fetch robot, it enables principled trade-offs between confident action and information gathering, enhancing robustness in open-world scenarios.
The paper proposes a Bayesian surrogate model to quantify uncertainty in black-box vision-language models for visual grounding, decomposing errors into aleatoric and epistemic components to improve robot robustness on real hardware.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
tion and task performance. On a real Fetch robot under open-world instructions, it enables principled trade-offs between confident action and information gathering, substantially improving robustness.
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org