PrismWAM and RoboDoctor: Exploring the Scaling Path of Physical AI

Machine Heart releases

Recently, Li Yiming, professor from Tsinghua University's School of Artificial Intelligence and founder of Lixing Intelligence, released two Physical AI research achievements: the World Action Model PrismWAM, and the Agentic framework RoboDoctor aimed at robot execution diagnosis and continuous improvement.

PrismWAM explores at the model level how to better represent physical states, predict state changes, and generate actions, ranking first with a leading advantage in the RoboTwin 2.0 evaluation.

Leading in first place with a leading advantage in evaluations

; RoboDoctor explores from the system level how to utilize execution feedback for diagnosis, error correction, and experience accumulation, achieving 57.1 points in tests covering all 54 tasks of RoboDojo, ranking first in current comparisons.

In recent years, the progress of artificial intelligence has, to a large extent, come from the large-scale expansion of semantic intelligence—larger models, more data, stronger understanding and reasoning capabilities. However, as general-purpose foundational models such as GPT-6 Astra and Gemini continue to demonstrate stronger understanding, reasoning, and planning capabilities, a more fundamental question is emerging: enabling robots to understand the world and enabling robots to truly change the world are two different capabilities.

This is precisely the key question for the next phase of Physical AI: Scaling Semantics, or Scaling Physics?

The former allows models to have broader knowledge, stronger semantic understanding, and task reasoning; the latter allows models to truly understand physical states and action consequences, enabling robots to perceive changes, respond to uncertainty, and continuously correct and self-improve through failure.

The two works explore Scaling Physics from the model and system levels respectively: PrismWAM studies how physical states are represented, predicted, and converted into actions; Rob,Doctor further studies how the system uses physical feedback to diagnose deviations, correct actions, and accumulate experience after actions are executed in the real world.

The two projects explore Scaling Physics from the perspectives of models and systems: PrismWAM investigates how physical states are represented, predicted, and converted into actions; RoboDoctor further studies how the system utilizes physical feedback to diagnose deviations, correct actions, and accumulate experience after actions are executed in the real world.

Involving Physical State Changes in Action Generation

When semantic understanding gradually becomes a general capability, what kind of world model do robots still need?

As semantic understanding gradually becomes a general capability, what kind of world model do robots still need?

However, existing methods also face a question worth re-examining: should we model the appearance of the world as the world state itself?

An RGB video contains a large amount of information of different natures: the color, texture, lighting, and background of objects, the geometric shape, spatial position, and contact relationships of objects, as well as the changes they undergo under the action of forces. These pieces of information are intertwined in images but do not have the same level of control value.

These pieces of information are intertwined in images but do not have the same level of control value.

For a robot, understanding 'this is a cup' versus determining 'where is the rim, how should the bottle be tilted, and whether liquid will spill next,' are two different types of problems. The former mainly involves semantic recognition and task understanding, while the latter requires the system to accurately grasp geometric relationships, physical states, and the process of their evolution with actions.

The video model features include semantic and physical, among other types of information

As general foundational models become increasingly adept at understanding scenes and tasks, the research focus of Physical AI also needs to shift further towards the physical layer: instead of having a single representation handle all tasks, it is better to clearly distinguish semantic information from physical states, allowing different types of information to serve their most suitable targets.

PrismWAM: From 'What the World Looks Like' to 'How the World Changes'

Around this idea, Li Yiming's team proposed PrismWAM (Prismatic World Action Model).

The core insight of PrismWAM is to decouple semantic context (Semantic Context) from physical state (Physical State), no longer treating rich visual video features as a unified physical world state, but instead establishing a clear responsibility attention chain: Semantic Planning → Physical Rollout → Executable Action

Semantic Planning: Understanding Task Intent. The model learns task semantics and target scenarios from RGB observations, answering 'What should ultimately happen.'

Physical Rollout: Modeling Physical Evolution. The model predicts continuous geometric state changes through deep representation, characterizing how objects move, how spatial relationships change, and how tasks progressively approach their goals.

Executable Action: Generating Executable Actions. Action generation directly utilizes the physical evolution representation, converting state changes into continuous control that robots can execute.

PrismWAM Conceptual Illustration

Along this pipeline, information flows from RGB semantic representation to depth physical representation, then into action generation; the action module does not directly read RGB features. This allows the model to allocate more modeling capacity to changes in geometric structures and spatial states. To what extent contact and forces can be accurately predicted still needs to be validated through specific tasks and experimental results.

Traditional World Action Model vs. PrismWAM Architecture

This design explores a more fundamental question in Physical AI: what changes in the world should the model learn in order to more effectively control it?

If task semantics increasingly become capabilities that general foundational models can provide, then the relationship between physical states and actions becomes even more worth being explicitly modeled. PrismWAM does not deny the importance of semantic capabilities; rather, it attempts to let semantics no longer bear the entire modeling burden, making physical state evolution the core link between world understanding and action execution.

From this perspective, PrismWAM explores not simply adding a new modality to existing models, but rethinking the learning objective of world action models: not only learning how the world is presented, but also learning how the world changes under the influence of actions. This is the specific manifestation of Scaling Physics at the model level.

RoboTwin 2.0: Testing Model Capabilities in Complex Operations and Scenario Transfer

RoboTwin is an authoritative evaluation benchmark in the domain of dual-arm robot manipulation, covering 50 dual-arm manipulation tasks, and randomizing across five dimensions: background, lighting, table clutter, table height, and language instructions, to test the upper limit of the model's generalization capability. Among these, the Clean-to-Random (C2R) evaluation primarily examines whether the model can still stably complete operational tasks after migrating from a relatively clean, structured training environment to a test environment with randomized variations. This not only tests the model's mastery of known tasks, but also directly examines its cross-scenario generalization capability and environmental adaptability.

Without additional action data pre-training, PrismWAM achieves

an average success rate of 83.67%, ranking first on the official leaderboard

(as of October 23, 2026). In the high-difficulty Clean-to-Random evaluation, PrismWAM achieves

a success rate of 75.5%

further demonstrating its generalization capability in complex scene changes.

II. Without Additional Action Data Pre-training

Re-evaluating Data Efficiency in Physical AI

Stronger operational capabilities, can they only come from larger-scale action data?

In the paper's setup, PrismWAM uses a pre-trained video model as initialization and performs approximately 2,300 hours of pre-training on depth data for the depth encoder; however, it does not undergo additional action data pre-training. This setup reduces reliance on large-scale action pre-training. Action data typically requires specialized collection and annotation processes. If a model can reduce its dependence on such data through physical representation design, it could potentially lower the cost of training high-performance operational strategies.

Notably, PrismWAM outperformed several models pre-trained on large-scale action data, including OpenWAM, Xiaomi Robotics-0, and π0.5, on RoboTwin 2.0. It also surpassed a range of methods pre-trained on large-scale action data on RoboDojo, which evaluates capabilities such as coverage generalization, fine-grained manipulation, long-horizon tasks, memory, and open-ended tasks.

These results do not imply that action data pre-training is no longer important, nor do they mean that scaled training has reached its endpoint. They reveal another possibility:

what determines the value of data is not only its scale, but also its quality and modality, as well as how the model learns the world.

If a model's representation can more directly align physical states with action outcomes, the same training resources could potentially be converted into more effective control capabilities.

This is also an important goal of Scaling Physics: not only expanding data scale, but also improving the efficiency of converting data into physical capabilities.

PrismWAM is merely a Proof of Concept: Scaling is still ongoing, but the direction can change

It is important to emphasize that PrismWAM remains a proof of concept. It does not demonstrate that large-scale action data is no longer necessary; instead, it attempts to show that by rethinking the relationship between semantic understanding, physical states, and action outputs, models have the potential to build robust world action models with higher data efficiency.

Therefore, Scaling Physics is not rejecting Scaling, but rather rethinking what is being scaled. In addition to larger models and more data, physical state representation, state evolution modeling, and learning between actions and outcomes can also become core directions for capability expansion.

As robots transition from single-action generation to long-term task execution, Scaling Physics also needs to move beyond the model and into the system level: this is precisely the starting point of the second work, RoboDoctor.

III. From Policy to Agent

Allowing policies to diagnose, correct errors, and self-evolve during execution

With a stronger action model, why is diagnosis and error correction during execution still necessary?

PrismWAM explored how to convert changes in physical states into executable actions. However, in long-term tasks, the result of the previous step continuously alters the conditions for the next step: a slight offset in grasping may affect subsequent placement; a contact error may render an otherwise reasonable action ineffective. Even if a robot has strong local operational capabilities, it still needs to determine whether the entire task is progressing toward the goal. This brings up a new question:

When actual results deviate from expectations, can the robot identify the deviation, adjust its execution, and avoid repeating the same error in the next attempt?

Transitioning from Policy to Agent is precisely a response to this question. The underlying policy generates actions based on observations, while the system further manages the relationship between actions and task outcomes: assessing whether intervention is needed before execution, checking whether the expected outcome is achieved after execution, and connecting these assessments to subsequent steps. This extends the enhancement of physical capabilities from single-action generation to closed-loop execution of complete tasks.

RoboDoctor: Enabling a closed-loop of diagnosis, error correction, and experience accumulation

Building on this idea, the team led by Li Yiming proposed

RoboDoctor

to combine the continuous control capabilities of the underlying policy with the Agent's diagnosis, planning, and experience management. In the RoboDojo experiment, the system used a 5B parameter version of PrismWAM as the fixed underlying policy, conducting recursive self-improvement around three levels: control organization, intervention judgment, and specific operations.

Harness RSI: Improving the collaboration between the strategy and the Agent.

PrismWAM provides continuous and fine-grained action capabilities, while the Agent is responsible for reasoning, planning, and execution management. When repeated error correction fails to advance the task, the system needs to re-evaluate who should be in charge and how to organize tools and feedback. In the current experiment, a small number of Warmup rounds are used to assess the adaptability of the strategy, followed by a choice between two pre-set controllers: Hybrid retains strategy-led execution with Agent assistance; Direct allows the Agent to directly plan and organize actions. This is a specific implementation of control organization adapted to the task.

Intervention RSI: Improving the judgment of when and why intervention is needed.

In Hybrid mode, the Agent combines scene observations, action proposals, and robot kinematics preview to determine which effective behaviors should be retained and which deviations need correction. New observations after execution further validate this judgment: missed failures, unnecessary interventions, and new problems introduced by interventions all become the basis for revising diagnostic experience, enabling subsequent decisions to leverage previous execution feedback.

Operation RSI: Improving specific operations and error correction methods.

After determining the need for action, the Agent performs operations through scene localization and action tools, then revises the method based on actual contact, motion, and task results. Failures prompt the system to adjust the approach, contact position, or execution plan. Verified methods and their applicable conditions are retained as external experience and are repositioned and checked before reuse. This process serves both local corrections in Hybrid and task execution in Direct.

The weights of the policy and Agent models remain unchanged, with improvements occurring in control organization, decision-making, and experience utilization.

The three recursive self-improvement processes of RoboDoctor: execution evidence jointly supports updates to control organization, intervention judgment, and operational methods, while the weights of the policy and Agent models remain fixed.

This design further raises another core question in Physical AI:

How can a robot convert its experiences of interacting with the physical world into the basis for its next action?

When a general foundational model can provide task understanding and reasoning capabilities, and the underlying policy can generate continuous actions, the system still needs to connect both with actual execution results. Whether a grasp is stable, whether a correction is effective, all require evidence from physical feedback. RoboDoctor attempts to organize these evidences into checkable and reusable experiences, enabling the robot's subsequent decisions to be based on already occurred interactions.

From this perspective, the expansion of robotic capabilities can also occur during the continuous interaction between the model and the environment:

Let execution generate feedback, let feedback form experience, and then let experience guide action.

This is Scaling Physics' exploration at the system level—by more effectively utilizing physical interactions, converting existing model capabilities into task execution capabilities that can continuously improve.

RoboDojo: From Action Capabilities to Full Task Performance

RoboDojo evaluates robotic capabilities from five dimensions: Generalization, Precision, Long-Horizon, Memory, and Open Tasks. In the aggregated simulation experiments covering all

54 tasks

RoboDoctor v1.0 achieved

57.10 points, ranking first among current methods

. Compared to HexaShell-Preview (53.18 points) and WorldScape-R²SI (46.58 points), the top two in the official leaderboard, RoboDoctor led by

3.92 points and 10.52 points; compared to GPT-6-Astra and DeepSeek-Flash, it led by 28.13 points and 54.11 points.

The above is a percentage-based score and not the success rate. RoboDoctor results are updated on October 21, 2026, with reference methods from the official October 20 list.

The sub-results further demonstrate the value of physical models working in synergy with the Agent:

Precision: Converting physical representations into action advantages.

RoboDoctor combined with PrismWAM achieved

47.60 points in Precision, higher than all officially published Agent methods in the leaderboard

and higher than the highest-scoring reference method in this dimension, HexaShell-Preview (41.25 points), by

6.35 points

. Compared to WorldScape-R²SI's 23.51 points, EmbodiedEvo's 25.15 points, and GPT-6-Astra's 12.65 points, RoboDoctor also maintains a clear advantage. This performance indicates that introducing the Agent's memory and task management capabilities can coexist with leading precision capabilities, aligning with the design direction of using physical state modeling to support action execution.

Memory capability: enables action execution to leverage task context. RoboDoctor scores 77.50 on Memory, 11.72 points higher than WorldScape-R²SI, and 34.46 points higher than GPT-6-Astra.

Combined with the results of fine-grained manipulation, the system demonstrates complementary capabilities between action execution and task information utilization: robots need to complete the current action while also using previous information to determine how to proceed next.

Combined with the results from precision, the system demonstrates the complementary nature of action capabilities and task information utilization: robots need to complete current operations and also use previous information to determine how to act next.

Recursive Self-Improvement: Continuing to improve system performance under fixed weights.

Rises from 49.74 to 57.10, an increase of 7.36.

3.61 points, generalization score improved 9.48 points; open-ended task improvement was most significant, increased by 21.35 points. These stage results demonstrate the system's further improvement after utilizing execution feedback, adjusting control methods, and accumulating experience.

RoboDoctor outperformed GPT-6-Astra across all five dimensions, demonstrating the system value of integrating general reasoning with physical execution. Subsequently, the team will continue to focus on this closed-loop research: how to correct deviations with lower intervention costs, how to more accurately validate contact results and task completion, and whether accumulated experience can be reused across strategies and environments. From PrismWAM to RoboDoctor, the two works form a connection:

the former explores how to better model physical changes and generate actions, while the latter explores how to utilize execution feedback to transform action capabilities into sustainable improvements in task capabilities.

the former explores how to better model physical changes and generate actions, while the latter explores how to utilize execution feedback to transform action capabilities into sustainable improvements in task capabilities.

From Scaling Semantics to Scaling Physics

General intelligence is opening up new possibilities for robots. In the past, one of the important development paths for Physical AI was to combine stronger visual-language understanding capabilities with large-scale human motion data, enabling robots to understand scenes, understand instructions, and learn human operation methods.

However, as general-purpose foundational models such as GPT-6 Astra and Gemini continue to enhance semantic understanding, reasoning, and planning capabilities, a question worth reconsidering is: as general semantic capabilities can increasingly be obtained from foundational models, the next round of expansion for Physical AI may need to shift from purely Scaling Semantics to a greater emphasis on Scaling Physics.

This does not mean that semantic capabilities are no longer important. Quite the opposite, the stronger the foundational model, the more opportunity robots have to reuse mature semantic understanding and task knowledge, and to allocate more research resources to physical intelligence itself.

For industry, this means that the competition for Physical AI may no longer be limited to model parameter scale or pure video data scale, but will further extend to the acquisition and utilization of physical data, the effective representation of physical states, the accurate prediction of action consequences, and the closed-loop execution capability of robots in real environments.

PrismWAM explores at the model level how to convert semantic goals into physical states and executable actions; RoboDoctor explores at the system level how to let real execution generate feedback, and let feedback continue to improve task performance. These two efforts respectively advance the formation and execution of physical capabilities, and together constitute a stage-wise exploration of clarifying intelligence's focus on Scaling Physics.

Scaling Semantics, letting models understand more; Scaling Physics, letting models perform better.

Clarifying Intelligence bets on the latter:

It's not just about continuing to expand model and data scale, but also about improving the efficiency of converting data, computing power, and real interaction into physical capabilities.

© THE END

For republication, please contact this official account for authorization

For submissions or media inquiries: [email protected]