Do Embodied VLMs Actually Use Their Scene-Graph Memory?
Overview
This research paper introduces a diagnostic 'graph-swap test' for embodied vision-language models (VLMs) to determine if they truly utilize scene-graph memory or ignore it in favor of textual information. The study finds that while text-based memory delivery significantly improves route question accuracy, standard graph encoders with pooled or late fusion fail to convey graph information effectively. However, assigning individual input tokens to each node recovers some graph utility. The proposed test offers a low-cost method to validate whether a VLM relies on its scene-graph memory.
Generated from attributed reports · Updated 2 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Reported quantity · number of rooms in benchmark routes: 2 other · Basis not reported
“introduce a graph-swap test for the last of these: each episode is given another episode's graph, and we count how many answers change. On an embodied question-answering benchmark with routes through two to seven rooms, we use it to compare ways of delivering scene-graph memory to a fine-tuned VLM. As text, the memory is used strongly, adding 30 points on route questions. As a graph, it is not, and we trace why: the”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalDo Embodied VLMs Actually Use Their Scene-Graph Memory?
This research paper introduces a diagnostic 'graph-swap test' for embodied vision-language models (VLMs) to determine if they truly utilize scene-graph memory or ignore it in favor of textual information. The study finds that while text-based memory delivery significantly improves route question accuracy, standard graph encoders with pooled or late fusion fail to convey graph information effectively. However, assigning individual input tokens to each node recovers some graph utility. The proposed test offers a low-cost method to validate whether a VLM relies on its scene-graph memory.
Event coverage history
There is not enough continuous observation data to show a trend.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).