Let Robots Learn to 'Act Based on Circumstances': Zhiyuan Opens Fourth 'Physical Reasoning' Dataset

(Source: Shanghai Observer News)

Help me prepare the items for going out. Robot You need to first check the weather forecast and then determine which items to carry based on the weather conditions. If tomorrow is sunny, prepare sunscreen; if the weather turns rainy, you need to bring an umbrella. For humans, this sequence of judgments is almost effortless, but for robots, it involves multiple steps such as task understanding, information retrieval, conditional judgment, and action execution.

How can robots not only execute fixed actions according to instructions, but also autonomously judge and adjust their actions based on changes in the real environment, learning to “act according to the situation”? On October 22, AGIBOT officially open-sourced the fourth edition of the AGIBOT WORLD 2026 'Physical Reasoning' (Physical Reasoning) dataset.

Unlike data focused on operational skills, this dataset shifts the task design focus from 'operational difficulty' to 'decision difficulty', centering around the decision and execution processes under different states and constraints for the same task, helping robots transition from completing individual operations to autonomously handling complex tasks.

Tasks in the real world often lack a fixed set of procedures applicable to all situations. For example, to replenish the drinks in a hotel room mini-bar to 3 bottles, the robot needs to first distinguish between empty and full bottles, count the current number, and then decide how many to recover and replenish. Facing different initial states, even with the same task goal, the actions the robot needs to take may vary. For instance, when searching for toiletries in multiple drawers, the robot must not only identify the items but also remember which drawers have already been checked to avoid redundant searches.

To enable robots to autonomously perform these tasks, data on “how to execute actions” is insufficient—there must also be data on “how to make decisions based on goals and conditions.” The fourth dataset was precisely built around this gap.

According to the introduction, the fourth phase of AGIBOT WORLD 2026, the 'Embodied Reasoning' dataset, focuses on factors such as item count, occlusion relationships, spatial occupation, and task progress to construct decision-making scenarios of the same task under different states. It covers nine core reasoning capabilities including attribute selection, conditional judgment, operation feasibility assessment, goal search, long-term planning, process memory, and sets three difficulty levels: Easy, Medium, and Hard, covering tasks ranging from basic judgment to multi-goal planning and long-term state maintenance.

In addition to providing a variety of task scenarios, the dataset also links task goals, decision processes, and actual actions through fine-grained annotation. Data describes the complete goal with task instructions, and characterizes the task structure through information such as task phases, subtask intervals, and action steps; meanwhile, it provides detailed action descriptions and spatial annotations such as boxes, points, and lines, clearly defining the object of operation, contact positions, and motion paths, and recording the task execution results. These data can help researchers train robots' task understanding and reasoning capabilities, as well as support the coordinated training of high-level decision-making and low-level action strategies.

AGIBOT WORLD 2026 is an open-source dataset series launched by AGIBOT around embodied intelligence research, covering five major research themes in embodied intelligence. Each theme adopts corresponding data collection methods and annotation systems. It is reported that this dataset will be used for the training of AGIBOT's upcoming Agent framework and new-generation base large model, providing data foundation for robots to understand tasks, remember execution processes, and select actions.

Original Title: Let Robots Learn to “Act According to the Situation”, AGIBOT Opens Fourth Phase “Embodied Reasoning” Dataset

Column Chief Editor: Ren Quan Image Source: AGIBOT Robot

Source: Author: Wen Hui Daily Liu Qi

Massive information, precise analysis, all within the Sina Weibo APP

VIP Course Recommendation

Loading...

APP Exclusive Live Streaming

1/10

Popular Recommendations

Collapse

Sina Finance Official WeChat Account

24-hour rolling coverage of the latest financial news and videos, more fan benefits scan QR code to follow (sinafinance)