GeoMind: A Benchmark for Spatial Understanding in Robotic Manipulation
GeoMind: A Benchmark for Spatial Understanding in Robotic Manipulation
This paper introduces GeoMind, a new benchmark for evaluating spatial understanding in robotic manipulation. It consists of 54 task variants focused on spatial reasoning and memory, addressing limitations in existing benchmarks that rely on strong visual cues or static scenes. The authors release an automated pipeline for generating and annotating tasks with verified targets and executable interactions. Evaluations across diverse embodied agents reveal consistent weaknesses in tracking identities after spatial changes and converting relational constraints into actions.
The authors introduce GeoMind, a benchmark consisting of 54 task variants designed to evaluate spatial reasoning and spatial memory in robotic manipulation, noting that existing benchmarks often rely on strong visual cues or static scenes.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
task variants
54
View original evidence
ng embodied benchmarks fall short in evaluating spatial understanding: they often introduce strong visual cues that bypass spatial reasoning, and focus on static scenes that offer little insight into an agent's capacity for spatial memory. To address this gap, we introduce GeoMind, a systematic benchmark for structured spatial understanding through tabletop manipulation. GeoMind comprises 54 evaluated task variants o
Open source S3
- paper: not_reported
- weights: not_reported
- dataset: not_reported
- code: not_reported
What remains unknown
Dataset licensing and commercial-use rights are not established here.
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
ng embodied benchmarks fall short in evaluating spatial understanding: they often introduce strong visual cues that bypass spatial reasoning, and focus on static scenes that offer little insight into an agent's capacity for spatial memory. To address this gap, we introduce GeoMind, a systematic benchmark for structured spatial understanding through tabletop manipulation. GeoMind comprises 54 evaluated task variants o
Open source S3
onvert relational or geometric constraints into executable actions. We release the GeoMind benchmark at https://github.com/Notfound-JH/GEOMIND-benchmark.git.
Open source S5
Implications for data suppliers
RoboSignal interpretation and collection questions, not statements of buyer demand.
- Confirm the required data type and collection setting with the buyer; this source does not establish a complete collection specification.
- Compare the reported units and scope before using these quantities in a budget. Recording hours, sensor-hours and trajectories are different measures.
- Validate demand and acceptance criteria with a buyer before scaling. Publication, popularity and a research result do not establish a purchase commitment.
Source：arXiv Robotics — research abstracts · arxiv.org