Evaluating Long-Horizon Robot Planning under Uncertainty
Evaluating Long-Horizon Robot Planning under Uncertainty
This research introduces a benchmark for long-horizon household robot tasks within the Habitat simulation environment. The authors evaluate adaptations of ReAct, VLM-TAMP, and Tru-POMDP planners across 1,350 episodes involving seven tasks with varying initial memory and environment configurations. Results show that ReAct achieves the highest overall task success rate of 45.3% and completion rate of 71.7%. The study also identifies recurring errors such as placement mistakes, precondition failures, and redundant actions through trace analysis, highlighting how different planners utilize initial knowledge and respond to uncertainty.
The study evaluates three robot planning algorithms (ReAct, VLM-TAMP, and Tru-POMDP) across a benchmark of seven household tasks in the Habitat simulator, analyzing task success, partial completion, and action efficiency under varying memory and environment configurations.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
episodes
1,350 episodes
Summed sensor-hours
View original evidence
ks-environment-memory combinations, totaling 1,350 episodes. Our evaluation measures task success, partial completion, and action efficiency, along with rule-based analysis of execution traces. ReAct achieves the highest overall task success (45.3%) and completion (71.7%), while having accurate initial memory does not consistently yield the highest performance for every planner. We identify recurring placement errors
Open source S3tasks-environment-memory combinations
90
View original evidence
arXiv:2610.17326v2 Announce Type: replace Abstract: Household robots must successfully navigate im-perfect circumstances, both in their internal models and the external world. We introduce a benchmark of seven long-horizon household tasks in Habitat that vary initial robot memory and environment configurations. We evaluate adaptations of ReAct, VLM-TAMP, and Tru-POMDP within a shared execution framework across 90 tas
Open source S2
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
arXiv:2610.17326v2 Announce Type: replace Abstract: Household robots must successfully navigate im-perfect circumstances, both in their internal models and the external world. We introduce a benchmark of seven long-horizon household tasks in Habitat that vary initial robot memory and environment configurations. We evaluate adaptations of ReAct, VLM-TAMP, and Tru-POMDP within a shared execution framework across 90 tas
Open source S2
ks-environment-memory combinations, totaling 1,350 episodes. Our evaluation measures task success, partial completion, and action efficiency, along with rule-based analysis of execution traces. ReAct achieves the highest overall task success (45.3%) and completion (71.7%), while having accurate initial memory does not consistently yield the highest performance for every planner. We identify recurring placement errors
Open source S3
Source：arXiv Robotics — research abstracts · arxiv.org