LADDER: A Long-Horizon Robot Manipulation Benchmark
Overview
LADDER evaluates long-horizon robot manipulation, showing general-purpose policies degrade on extended tasks, while orchestration methods like GPT-Policy show better performance but face execution errors.
Generated from attributed reports · Updated 1 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Environment: simulation
“X2Real: an eXtensive simulation benchmark for real-world generalist policies”
Exact source · revision 1Source owner not reported
“ronment over three suites (SimplerEnv-Long, RobotArena-$\infty$-Long, and RoboLab-Long), 145 tasks, eight categories, two embodiments, and 2,024 simulated episodes per system. We evaluate two orchestration paradigms: text subgoals for learned low-level policies (VoLo) and tool selection orchestration (CaP-X, GPT-Policy, PSP-Agent, TiPToP). Given the full instruction directly, low-level policies achieve 85.4% on singl”
Exact source · revision 1Source owner not reported
Reported quantity · simulation trajectory dataset: 300 hours · Basis not reported
“training-evaluation pipelines to mitigate benchmark exploitation and ensure credible evaluation. Powered by a custom physical domain-specific language, the Mana simulation ecosystem supports modular task design and iterative performance analysis, alongside a nearly 300-hour annotated simulation trajectory dataset. X2Real offers a faithful, diverse, and fair evolving evaluation infrastructure, effectively bridging the”
Exact source · revision 1Source owner not reported
Reported quantity · unit-test scenarios: 330 other · Basis not reported
“world settings. We conduct a comprehensive evaluation of 9 visual navigation models on 330 unit-test scenarios, 800 pedestrian-reactive scenarios, and 105 long-horizon scenarios. Our findings highlight that pedestrian interaction and long-horizon robustness remain critical bottlenecks for existing models, and scaling up sidewalk training with synthetic data emerges as a promising solution.”
Exact source · revision 1Source owner not reported
Reported quantity · pedestrian-reactive scenarios: 800 other · Basis not reported
“world settings. We conduct a comprehensive evaluation of 9 visual navigation models on 330 unit-test scenarios, 800 pedestrian-reactive scenarios, and 105 long-horizon scenarios. Our findings highlight that pedestrian interaction and long-horizon robustness remain critical bottlenecks for existing models, and scaling up sidewalk training with synthetic data emerges as a promising solution.”
Exact source · revision 1Source owner not reported
Reported quantity · long-horizon scenarios: 105 other · Basis not reported
“world settings. We conduct a comprehensive evaluation of 9 visual navigation models on 330 unit-test scenarios, 800 pedestrian-reactive scenarios, and 105 long-horizon scenarios. Our findings highlight that pedestrian interaction and long-horizon robustness remain critical bottlenecks for existing models, and scaling up sidewalk training with synthetic data emerges as a promising solution.”
Exact source · revision 1Source owner not reported
Reported quantity · tasks: 145 other · Basis not reported
“ronment over three suites (SimplerEnv-Long, RobotArena-$\infty$-Long, and RoboLab-Long), 145 tasks, eight categories, two embodiments, and 2,024 simulated episodes per system. We evaluate two orchestration paradigms: text subgoals for learned low-level policies (VoLo) and tool selection orchestration (CaP-X, GPT-Policy, PSP-Agent, TiPToP). Given the full instruction directly, low-level policies achieve 85.4% on singl”
Exact source · revision 1Source owner not reported
Reported quantity · episodes: 2024 other · Basis not reported
“ronment over three suites (SimplerEnv-Long, RobotArena-$\infty$-Long, and RoboLab-Long), 145 tasks, eight categories, two embodiments, and 2,024 simulated episodes per system. We evaluate two orchestration paradigms: text subgoals for learned low-level policies (VoLo) and tool selection orchestration (CaP-X, GPT-Policy, PSP-Agent, TiPToP). Given the full instruction directly, low-level policies achieve 85.4% on singl”
Exact source · revision 1Source owner not reported
Reported quantity · subgoals: 1 other · Basis not reported
“arXiv:2610.18053v1 Announce Type: new Abstract: General-purpose robot manipulation policies learn short-horizon skills effectively, but degrade rapidly on long-horizon tasks. Existing benchmarks cannot isolate failures because task horizon is either too short or entangled with scene and object variations. We propose LADDER, a long-horizon robot manipulation benchmark scaling from 1 to 10 subgoals within the same envi”
Exact source · revision 1Source owner not reported
Reported quantity · embodiments: 2 other · Basis not reported
“ronment over three suites (SimplerEnv-Long, RobotArena-$\infty$-Long, and RoboLab-Long), 145 tasks, eight categories, two embodiments, and 2,024 simulated episodes per system. We evaluate two orchestration paradigms: text subgoals for learned low-level policies (VoLo) and tool selection orchestration (CaP-X, GPT-Policy, PSP-Agent, TiPToP). Given the full instruction directly, low-level policies achieve 85.4% on singl”
Exact source · revision 1Source owner not reported
Reported quantity · categories: 8 other · Basis not reported
“ronment over three suites (SimplerEnv-Long, RobotArena-$\infty$-Long, and RoboLab-Long), 145 tasks, eight categories, two embodiments, and 2,024 simulated episodes per system. We evaluate two orchestration paradigms: text subgoals for learned low-level policies (VoLo) and tool selection orchestration (CaP-X, GPT-Policy, PSP-Agent, TiPToP). Given the full instruction directly, low-level policies achieve 85.4% on singl”
Exact source · revision 1Source owner not reported
Reported quantity · Reported success rate: 88.8 percent · Basis not reported · Differing source assertions
“affects each unexecuted action. The resulting belief revision enables selective repair of only the affected actions within a cached action chunk, preserving valid actions and reserving full replanning for severe deviations. On CALVIN, L-CALVIN, and RLBench, ReEnvision achieves 86.6% 5-step success, improves 10-step success from 56% to 63%, and reaches an average success rate of 88.8%, respectively. On five real-worl”
Exact source · revision 3Source owner not reported
Reported quantity · Reported success rate: 82.8 percent · Basis not reported · Differing source assertions
“d manipulation tasks, it achieves an average success rate of 82.8%. Controlled repair experiments further demonstrate the benefits of localizing affected actions and preserving valid ones during execution.”
Exact source · revision 3Source owner not reported
Evidence dependency changes (3)
- 2026-10-16T11:25:19.498Z · evidence updated · source revision 2. Evidence extraction was updated; current source attributions are shown above.
- 2026-10-15T20:22:44.868Z · evidence updated · source revision 3. Evidence extraction was updated; current source attributions are shown above.
- 2026-10-15T19:34:17.571Z · evidence updated · source revision 1. Evidence extraction was updated; current source attributions are shown above.
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsX2Real: an eXtensive simulation benchmark for real-world generalist policies
X2Real is a simulation benchmark that evaluates the real-world performance of robotic manipulation policies by aligning simulation visual and physical properties with real hardware. It features a comprehensive taxonomy with 10 capability dimensions and 44 hierarchical tasks, along with multi-axis domain randomization and disjoint training-evaluation pipelines to ensure fair and reliable evaluation.
- arXiv Robotics — research abstractsSidewalkBench: Benchmark for Visual Navigation on Urban Sidewalks
SidewalkBench evaluates visual navigation models on urban sidewalks using simulated environments. It identifies pedestrian interaction and long-horizon robustness as key challenges.
- arXiv Robotics — research abstractsLADDER: A Long-Horizon Robot Manipulation Benchmark
LADDER is a new benchmark for long-hor,izon robot manipulation, designed to evaluate the performance of policies across extended tasks. The study shows that general-purpose policies degrade rapidly on long-horizon tasks, while orchestration methods like tool selection with GPT-Policy show better performance but remain limited by low-level execution errors.
- arXiv Robotics — research abstractsRM-WAM: Reliable Memory-Augmented World Action Models for Long-Horizon Robot Manipulation
RM-WAM improves robot manipulation by using memory to predict actions and states. It addresses reliability issues in long tasks by selectively using historical data. Experiments show its effectiveness in complex scenarios.
- arXiv Robotics — research abstractsReEnvision: Future-Guided Selective Action Repair for Long-Horizon Manipulation
ReEnvision is a framework for long-horizon robotic manipulation that addresses execution deviations by selectively repairing affected actions using future-guided belief updates. It improves success rates on multiple benchmarks and real-world tasks, demonstrating the benefits of localized repair and preserving valid actions during execution.
- arXiv Robotics — research abstractsGoal2Skill: Long-Horizon Manipulation with Adaptive Planning and Reflection
This research proposes a dual-system framework for long-horizon embodied manipulation, combining high-level semantic reasoning with low-level visuomotor control. The framework addresses limitations in existing VLA systems by incorporating structured memory, adaptive task decomposition, and closed-loop recovery mechanisms, achieving a significant improvement in success rates on benchmark tasks.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).