AGIBOT Open-Sources AGIB T WORLD Release 4: 'Physical Reasoning' Dataset Enabling Robots to Reason Before Acting

AGIBOT Open-Sources AGIBOT WORLD Release 4: "Physical Reasoning" Dataset Enabling Robots to Reason Before Acting

【SHANGHAI, China, October 22, 2026】AGIBOT today announced the open-source release of the "Physical Reasoning" dataset, the fourth release of AGIBOT WORLD 2026. For the first time in the AGIBOT WORLD 2026 initiative, the focus of task design shifts from manipulation difficulty to decision-making complexity, helping robots choose and execute actions as real-world conditions change.

AGIBOT WORLD 2026 is the first open-source dataset to cover the full range of embodied artificial intelligence (AI) research. It is organized around five research themes in embodied AI, each with dedicated data-collection methods and a fine-grained annotation system to support researchers in different subfields. Grounded in complete, real-world task requirements, Release 4 captures decision-making and execution processes for the same goal across different states and constraints. This approach is intended to help robots progress from isolated manipulations toward autonomously completing complex real-world tasks.

For example, when asked to "help me pack what I need for going out tomorrow," an agent first checks the next day's weather forecast using a tool, then determines which items are needed and directs a robot to pick and place them. A sunny forecast calls for sunscreen, while rain calls for an umbrella.

Release 4 connects goal comprehension, task-state tracking, and action execution in real-world tasks. Covering over 10 categories of logical reasoning scenarios, the dataset deliberately includes multiple real-world states and execution branches for the same task. By providing high-level Q&A and low-level action trajectories, it supports coordinated training of high-level decision-making models and low-level action policies, linking task decision-making to physical robot execution. The aim is to help robots both "reason clearly" and "execute successfully."

Project Page: https://agibot-world.com

Open Source Link: https://huggingface.co/datasets/agibot-world/AgiBotWorld2026

Real-World Tasks: The Right Action Depends on the Situation

Real-world tasks cannot be solved by simply repeating a fixed sequence of actions.

For example, to restock a hotel-room minibar to three full bottles, a robot must distinguish empty bottles from full ones and count the bottles already there. It must then decide how many bottles to remove and how many new ones to add. The required action sequence changes with the initial state.

Minibar restocking requires the robot to assess bottle states and quantities before deciding what to remove or add.

Searching for toiletries across multiple drawers presents a different problem. Even when the camera view is unchanged, the next decision depends on which drawers have already been checked. Task-state assessment, dependencies between steps, and external conditions all matter in everyday activities such as housekeeping, cooking, and getting ready to go out.

Guest-room cleaning requires the robot to make decisions based on scene conditions and task progress.

For robots to complete these tasks autonomously, training data must capture more than "how to execute actions." It must also capture "how to make decisions based on goals and conditions." Release 4 is designed to address this need.

Same Task, Multiple States: Nine Categories of Reasoning Capabilities

Driven by real-world task goals, the dataset collects decision-making and execution processes for the same task under different states and constraints. It focuses on four key factors: object quantity, occlusion relationships, spatial occupancy, and task progress.

Consider retrieving a key from behind a box. If there is enough room to reach it, the robot can grasp the key directly. If the key is blocked, the robot must first move the box. The goal remains the same, but the action path changes with the scene.

Retrieving a key requires different action sequences depending on available space and obstructions.

The dataset addresses nine core reasoning abilities: attribute-constrained selection, context-conditioned selection, manipulation feasibility assessment, action precondition reasoning, goal-directed search, state and quantity reasoning, long-horizon planning, memory and state maintenance, and subtask ordering reasoning.

Examples of tasks in the Physical Reasoning dataset, covering different reasoning requirements, scene conditions, and constraints.

In practice, these capabilities often overlap. Minibar restocking, for example, requires product discrimination, quantity judgment, and state maintenance, while finding an item involves search strategy, search history, and spatial conditions.

Tasks are organized into three difficulty levels—Easy, Medium, and Hard. Easy tasks test individual judgments, such as attribute-constrained selection. Medium tasks introduce quantity requirements and action preconditions. Hard tasks combine multi-subgoal planning, long-horizon task-state tracking, and multiple contextual constraints. This structure allows researchers to isolate specific abilities and evaluate their compositional performance.

Execution outcomes are also annotated, including whether objects were correctly selected or placed and whether the required quantity was met. Successful and erroneous samples are given equal weight, providing material for research into how systems can adjust after deviations during execution.

One Data Pipeline Linking "Decision – Process – Action"

Supporting varied task situations requires annotations that connect the basis for each decision to the execution process. In Release 4, the high-level instruction describes the complete goal, while the task frame, subtask segments, and action steps capture the task structure. Together, these annotations help researchers map task progress to the current subgoal and study task breakdown, stage recognition, and next-subgoal prediction.

At the action level, the Dense Prompt (detailed action description) specifies the target object, manipulation region, bimanual role assignment, motion process, gripper state, and expected outcome. These descriptions are aligned with real-robot actions. Spatial annotations, including bounding-box, point, and trajectory-line annotations, further support visual-language grounding by linking language instructions to specific visual locations and motion paths.

Annotations link instructions to robot actions in a task involving returning a tool to its toolbox.

Additional annotations, including "success segments" and "adversarial segments," provide information for assessing task completion, distinguishing targets from distractors, and checking execution results. In combination, the annotations link the task goal, current stage, manipulation requirements, and execution outcomes in a single pipeline, connecting high-level decision-making research with action-policy training.

From Task Training to Real-World Applications

Researchers can use this release for joint or end-to-end training aimed at improving models' situational understanding, process memory, and task reasoning. The dataset also supports decision–execution co-training of embodied agents: an agent selects subgoals based on the current situation and execution history; the action policy performs the manipulation; and the task state is updated so subsequent plans can be adjusted according to the results.

Grounded in real-world tasks, the dataset provides a training foundation for extending the capabilities of large models into the physical world. By connecting task decision-making, process memory, and real-robot execution, it is designed to help foundation models and embodied agents translate an understanding of user goals into physical actions.

In the longer term, this approach could help integrate robots into complete workflows in home services, hospitality, and commercial operations, where tasks require continuous judgment and execution. It could also lay the groundwork for the large-scale deployment of embodied AI in the real economy.

Table resetting illustrates a service workflow that requires coordinated task planning and physical execution.

Retail restocking involves identifying products, assessing quantities, and placing items in the appropriate locations.

The "Physical Reasoning" dataset has already been used to support AGIBOT's upcoming agent framework and to train its next-generation foundation model. Further progress will be announced in due course.

For more information, please visit AGIBOT.com and follow AGIBOT on:
X: https://x.com/AGIBOTofficial
LinkedIn: https://www.linkedin.com/company/agibot/
YouTube: https://www.youtube.com/@AGIBOTofficial
Instagram: https://www.instagram.com/agibotofficial/
TikTok: https://www.tiktok.com/@agibotofficial
Facebook: https://www.facebook.com/AGIBOTofficial/

About AGIBOT
AGIBOT is a pioneer in the global general-purpose AI robotics industry, with its core technology focused on embodied AI. AGIBOT's "Three Intelligences in One" architecture integrates Locomotion Intelligence, Interaction Intelligence, and Manipulation Intelligence into a unified embodied system. Its portfolio spans humanoid robots, quadrupeds, dexterous systems, and commercial cleaning solutions. In September 2026, AGIBOT announced that its 20,000th robot had rolled off the production line.