Decoding Geometry, Not Tokens: Efficient VLMs for Robot Affordance and Trajectory Prediction
Decoding Geometry, Not Tokens: Efficient VLMs for Robot Affordance and Trajectory Prediction
This paper argues that predicting robot actions is fundamentally a geometry problem rather than a language generation task. The authors show that compact Vision-Language Models (VLMs) with 0.5-2B parameters already encode rich spatial information in their intermediate hidden states. By attaching a lightweight, geometry-compatible prediction head to a frozen backbone, the method regresses affordance boxes and trajectory waypoints directly as continuous values, bypassing the autoregressive decoding bottleneck. Evaluated on ShareRobot-Bench and AGD20K, the 2B variant reduces trajectory error by over 60% and improves affordance mAP by 85% relative to an autoregressive baseline, approaching RoboBrain performance at significantly higher throughput.
The authors demonstrate that frozen Vision-Language Model backbones contain latent spatial information usable for continuous trajectory and affordance prediction, achieving significant error reductions on ShareRobot-Bench and AGD20K without modifying the backbone architecture.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
hes to a frozen backbone. It reads features from an intermediate hidden layer, conditions a learned query on the task instruction and refines it via cross-attention over the visual tokens to regress affordance boxes and trajectory waypoints directly as continuous values, without modifying the backbone. We evaluate on ShareRobot-Bench and AGD20K. Our 2B variant reduces trajectory error by over 60% and improves ShareRo
Open source S4
r, we show that compact off-the-shelf VLMs with 0.5-2B parameters already encode rich spatial information in their intermediate hidden states. Autoregressive decoding is the dominant bottleneck for trajectory prediction, while affordance prediction depends additionally on feature depth and decoder architecture. We expose the latent spatial information with a lightweight, geometry-compatible prediction head that attac
Open source S3
Source：arXiv Robotics — research abstracts · arxiv.org