Semantically Adaptive Control Barrier Functions with Vision-Language Models for Real-Time Robot Navigation
Semantically Adaptive Control Barrier Functions with Vision-Language Models for Real-Time Robot Navigation
This research presents AlphaAdj, a navigation framework that combines egocentric RGB observations with a vision-language model to dynamically adjust control barrier function conservativeness based on semantic context. By extracting caution signals from token-level probabilities asynchronously, the system achieves real-time operation (0.0167 s control period) while allocating significantly more clearance to humans than benign objects in fixed-geometry scenarios. Evaluations on a physical Clearpath Jackal demonstrate that AlphaAdj remains collision-free in cluttered conditions where geometric baselines fail, proving that semantic visual reasoning can complement geometric safety control without entering the critical control path.
The paper introduces AlphaAdj, a framework that integrates vision-language models with control barrier functions to adapt safety margins based on semantic context while maintaining real-time performance on physical hardware.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
control period
0.0167
View original evidence
tes 0.405 m more clearance to a human than to a benign object, while geometry-only and non-visual adaptive CBF baselines produce no semantic differentiation. This additional behavioral reserve improves robustness to unexpected human motion, including cluttered conditions in which all evaluated geometric baselines collide while AlphaAdj remains collision-free. The system maintains a 0.0167 s control period with 0.158
Open source S5VLM round-trip latency
0.158
View original evidence
tes 0.405 m more clearance to a human than to a benign object, while geometry-only and non-visual adaptive CBF baselines produce no semantic differentiation. This additional behavioral reserve improves robustness to unexpected human motion, including cluttered conditions in which all evaluated geometric baselines collide while AlphaAdj remains collision-free. The system maintains a 0.0167 s control period with 0.158
Open source S5additional clearance to human
0.405
View original evidence
tes 0.405 m more clearance to a human than to a benign object, while geometry-only and non-visual adaptive CBF baselines produce no semantic differentiation. This additional behavioral reserve improves robustness to unexpected human motion, including cluttered conditions in which all evaluated geometric baselines collide while AlphaAdj remains collision-free. The system maintains a 0.0167 s control period with 0.158
Open source S5
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
real-time operation despite VLM latency, inference runs asynchronously with the control loop, and the semantic signal is extracted directly from token-level output probabilities rather than full autoregressive text generation. We evaluate AlphaAdj across matched-geometry, dynamic-human, and cluttered navigation scenarios, as well as on a physical Clearpath Jackal. When obstacle geometry is held fixed, AlphaAdj alloca
Open source S4
r physical configurations. We present AlphaAdj, a vision-to-control navigation framework that uses egocentric RGB observations and a vision-language model (VLM) to adapt CBF conservativeness according to visual semantic context. The VLM produces a scalar contextual caution signal that dynamically modulates the CBF parameter alpha, while a geometry- and speed-dependent cap bounds admissible aggressiveness. To support
Open source S3
tes 0.405 m more clearance to a human than to a benign object, while geometry-only and non-visual adaptive CBF baselines produce no semantic differentiation. This additional behavioral reserve improves robustness to unexpected human motion, including cluttered conditions in which all evaluated geometric baselines collide while AlphaAdj remains collision-free. The system maintains a 0.0167 s control period with 0.158
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org