Ai2 Releases ToyBlocks to Test How Robots Learn and Fail

Automatically extracted; no manual editorial approval recorded.

Ai2 released ToyBlocks on November 6, 2026, as the first environment in its FoundationWorlds framework, pairing an open robot simulation with generated demonstrations, two trained models and evaluation tools. The framework is designed to make robot-learning experiments repeatable and failures easier to trace. The setting puts a simulated arm and labeled blocks on a tabletop. Pushing, flipping and stacking are joined

Open source S2

execution. Failed states can also return to data generation for fresh demonstrations and recoveries. Ai2 reports that the initial release contains 85,336 long-horizon demonstrations, each comprising roughly 6–12 skill demonstrations depending on the set, covering nearly 1,600 hours of robot execution and 144 million frames. The public dataset uses Parquet-formatted LeRobot data under an ODC-BY license, with easy, med

Open source S7

ium and hard subsets. The main code is Apache-2.0 licensed; specified submodules and downloaded assets carry their own licenses. Ai2’s model cards divide the released policy into two levels: Molmo2-4B-tb , a roughly 4.5-billion-parameter model, reads a front-camera image and text goal to produce a short plan, next action and target point; ManiFlow-tb , with 322.5 million parameters, uses front and gripper images, rob

Open source S8

ot state, task information and a target heatmap to generate action chunks. Generalization and Agent Experiments The spatial experiments cover Stack, Unstack, Flip and Rotate, distinguishing interpolation within the training range from extrapolation beyond it. Examples include training with blocks in one workspace region and testing in another, or withholding particular stack heights. Ai2 reports poor initial generali

Open source S9

scriptions raised success from 24% with default descriptions to 63%, exceeding the 36% achieved with descriptions written by a human expert. Physical Robot Tests and Their Limits For physical evaluation, models trained entirely on ToyBlocks simulation data ran without real-world training examples on a Franka FR3 arm with a Robotiq gripper. The 30 scene configurations contained 4–10 blocks seen during training. An epi

Open source S12

reports ManiFlow atomic-skill success rates of 80.0% in simulation and 81.1% on the physical arm. The hierarchical Molmo 2–ManiFlow system succeeded in 55.5% of simulated episodes (111 of 200) and 50.0% of physical episodes (15 of 30). Ai2 qualifies those results on two fronts. The physical hierarchy evaluation used the easiest simulated configuration, with block characters visible and upright, and further trials are

Open source S14

zation for ManiFlow, π0.5 and MolmoAct 2 despite large-scale pretraining, and concludes that action-model generalization remains far from solved. The agentic tests expose a different weakness. Ai2 says its Claude Opus 5.5-based system solved all 30 classical block-stacking tasks from text descriptions and all 30 when choosing actions in a closed loop from repeated camera images. In search-and-spell scenes containing

Open source S10

needed to establish statistical confidence across tasks. Its ManiFlow card also records success rates from 54% to 59% across four runs of the same 128 simulated episodes, because simulation and batched inference are not bitwise deterministic. Ai2 plans a second release with a larger dataset, more skills and broader benchmarking, plus fully integrated reinforcement-learning training that is absent from the current pa

Open source S15