Release of MicroPlanBench and MicroAnnotationBench benchmarks
Overview
Animesh Garg and @microagi introduce two benchmarks, MicroPlanBench and MicroAnnotationBench, to assess the performance of Vision-Language Models (VLMs) in robotics. Results show that even top models like GPT-6 Astra and Sol struggle with next-action prediction and temporal grounding, while specialized models like nanoannot-1 achieve better performance at a fraction of the cost.
From Animesh Garg — X Robotics
Event evidence and corrections
1 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Reported quantity · over-segmentation segments per clip: 22 other · Basis not reported
“- GPT-6 Astra/Sol over-segment (~22 segments/clip vs. 13.8 human mean), collapsing precision (~39%) and strict Boundary F1 (<20%).”
Exact source · revision 1animesh-garg
Evidence dependency changes (2)
- 2026-10-30T22:08:45.608Z · evidence updated · source revision 1. Evidence extraction was updated; current source attributions are shown above.
- 2026-10-30T22:08:23.884Z · evidence updated · source revision 1. Evidence extraction was updated; current source attributions are shown above.
Report timeline
Follow attributed reports and material updates.
- Animesh Garg — X RoboticsSignalTwo New Egocentric Benchmarks Evaluate VLM Readiness for Robotics
Animesh Garg and @microagi introduce two benchmarks, MicroPlanBench and MicroAnnotationBench, to assess the performance of Vision-Language Models (VLMs) in robotics. Results show that even top models like GPT-6 Astra and Sol struggle with next-action prediction and temporal grounding, while specialized models like nanoannot-1 achieve better performance at a fraction of the cost.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).