Robots Can Improve Themselves: Toward Physical Recursive Self-Improvement

In recent years, the development of robot learning has enabled machines to autonomously complete an increasing number of tasks. However, behind the autonomous execution, data collection, failure diagnosis, training plan adjustment, and deployment decisions still largely rely on researchers and engineers.

Robots are becoming increasingly capable of performing tasks, but how to enable them to continuously improve remains a task primarily driven by humans.

Research on self-improving agents provides a new approach to this challenge. Systems can modify code, accumulate experience, or update models based on feedback, and apply effective modifications to subsequent tasks. However, these efforts typically follow pre-set improvement processes: while models and skills are updated, the methods for generating and screening updates remain unchanged. Recursive self-improvement, however, further focuses on whether this improvement process itself can also be updated from experience, making subsequent learning more effective.

When recursive self-improvement enters the physical world, this goal faces more challenges. The observations robots receive are often incomplete and noisy, and it is difficult to accurately determine the cause of failures; validating a modification requires repeated trials, accompanied by environment reset, hardware wear, and even irreversible consequences. Therefore, self-improvement methods in digital environments need to be re-evaluated under the constraints of physical feedback, trial costs, and safety considerations.

Paper Information

Focusing on these issues, researchers from institutions such as Shanghai Jiao Tong University, University of California, Berkeley, Stanford University, Massachusetts Institute of Technology, Google, Microsoft, NVIDIA, and Xiaomi, in the paper 'Who Improves the Robot? Toward Physical Recursive Self-Improvement' will

update task capabilities and improve the improvement process

Incorporate into a unified framework, organizing how robots can continuously improve by leveraging physical experience, as well as how safety and verification mechanisms support this process. By distinguishing between ``robots performing better'' and ``robots improving their methods to become more efficient,'' the paper clarifies the progress and boundaries of existing work, and discusses key issues that need to be addressed to move toward Physical RSI. The paper's corresponding authors are Gu Shangding and Zhang Weinan from Shanghai Jiao Tong University, and Feng Qian from Technical University of Munich.

Paper link: https://arxiv.org/pdf/2610.18842

Project address: https://github.com/SAIL-Research-Lab/Physical-RSI

Core contributions

The paper presents three aspects of contributions around 'what was improved, through what path was it improved, and how was it proven effective'.

First, establish a unified framework for understanding Physical RSI. The paper provides the definition of Physical RSI and L0–L5 grading, from

whether the update is about task capability or process improvement

and the

whether the gains are limited to the same task or can be transferred across tasks

two dimensions, providing a common analytical framework for comparing different studies.

Second, outline the full-chain implementation path from capability update to process evolution

path

. The paper focuses on harness adaptation and model training, analyzing how memory, skills, workflows, and training methods affect task execution and subsequent improvements, and incorporates safety control and update verification into the same analysis, explaining what constraints and checks are required for continuous improvement.

Finally, clearly define the evidence required to test recursive gains. Under the same initial system, experience, and resource budget, compare the subsequent gains generated by the old and improved processes through independent cross-task evaluations, and identify the evidence gaps in existing research accordingly. This makes ``learning to learn'' a testable research proposition.

Physical RSI Fundamental Framework

Physical RSI focuses on whether robots can utilize their physical interaction experiences to continuously update their task execution systems and improve processes, and gain recursive benefits.

The paper divides the entire process into two major parts: one is the task execution system that interacts with the external environment, including the models used during interaction, as well as the harness, which includes prompts, memory, skills, and control code outside of the models. The second is the internal improvement process, which is responsible for improving the task execution system and the improvement process itself.

The main improvement loop begins after the task execution system completes one physical interaction, obtaining relevant interaction evidence and internal system component signals. The improvement process can propose candidate modifications based on these experiences and feedback, and the validator verifies and applies the candidates. An independent evaluator is responsible for assessing the gains of the updated system, proving that the new process can bring better subsequent improvements under the same conditions.

Our long-term vision is that robots can autonomously operate and continuously self-improve and evolve in real physical environments full of diverse and unpredictable challenges, ultimately achieving Physical RSI.

L0-L5 Grading

L0: No autonomous generation and retention of updates; only re-planning during each task execution, or improvements depend on human intervention

L1: Autonomous Update

Task Execution System, gaining benefits on the same task, with the improvement process remaining fixed L2: Autonomous Update Improvement Process Itself, gaining benefits on the same task L3: Autonomous Update of the Task Execution System, bringing Cross-Task Benefits, with the improvement process remaining fixed L4: Autonomous Update of the Improvement Process Itself, bringing Cross-Task Benefits L5: On the basis of L4, through independent, controlled comparisons, it is verified that the new process of continuous updates brings better subsequent cross-task benefits than the previous version Current Research Progress of Physical RSI Harness Adaptation: From Peripheral Systems to Evolution of Improvement Methods System resources outside the model can also become objects of improvement. Execution feedback is used to diagnose failures, update memory, skills, and workflows. Effective modifications are retained, and modification records support subsequent diagnosis and solution selection. Recent research has also begun to further adjust the methods for generating these updates, including diagnostic guidance, memory construction strategies, and search procedures, providing references for introducing the evolution of improvement methods themselves into physical scenarios.

Model Training: Physical Experience Begins to Enter the Training Mechanism Itself

Data collection, model updates, and re-execution form a training loop, continuously converting physical interaction data into subsequent improvements. Some work further treats the training mechanism itself as an object of improvement; in terms of data preparation, there have already been studies that adjust data ratios, simulation distributions, or sampling strategies based on learning effectiveness. These explorations are extending the role of physical feedback from model parameter updates to adjustments in data preparation and training methods.

Safety: Learning Process Must Also Be Safe

Passing a safety test does not mean that previous explorations were risk-free. Safety reinforcement learning has already provided constraints for the exploration process, and safety transfer research clearly ensures the conditions for continued use. These advances mainly target specific components and scenarios; how to maintain safety guarantees and assess the risks brought by repeated trials during the evolution of the improvement process itself remains a key issue for Physical RSI.

Verification: Are the Modifications Effective, Are the Old Skills Still There?

Code being able to run is just the first step. Whether the modifications fix the original failures must be judged through re-execution; if shared skills change, previous successful tasks need to be retested. Internal verification determines which updates are retained, while external independent evaluators assess the actual gains. If the validator itself is modified, then an external independent evaluator must judge the effectiveness of the new version to avoid the system gradually relaxing its acceptance standards.

Benchmark: From Testing Capabilities to Testing Improvement Methods

Embodied benchmarks such as RoboDojo provide task performance and generalization tests, while benchmarks such as AI4AI-Bench mainly examine method or data strategy updates in digital environments. However, the paper points out that a joint benchmark for evaluating Physical RSI is still lacking, especially the direct testing of recursive gains in the improvement process.

Challenges and Future Outlook

The paper proposes six issues that remain to be solved:

Learning from Uncertain Physical Experience

. How to extract reliable learning signals and avoid error feedback from being amplified continuously through the inheritance of the improvement mechanism?

Causal Attribution Between Harness Components

. How to identify the modification targets and distinguish the actual effects of each modification within limited physical experiments?

Non-Stationarity of the Improvement Loop

. When the system, environment, and existing capabilities change, are past effective modifications and improvement strategies still valid?

Reward Speculation and Evaluation Leakage

. How to prevent the system from exploiting vulnerabilities to gain high scores and maintain the independence of evaluation, ensuring the results reflect the true physical capabilities?

Safety Maintenance in Recursive Updates

How can we maintain safety guarantees and control the accumulated risk of multi-round trials when improving process changes?

Recur

sive gain benchmark evaluation

Why does the subsequent system perform better: because of more data, stronger starting point, or gains from an updated improvement mechanism?

Conclusion

Physical RSI extends the robot's autonomy from task execution to the continuous updating of capabilities and improvement methods. The paper organizes existing progress through the L0-L5 framework and clearly identifies a key gap: improved task performance alone is insufficient to prove that the improvement process itself has become more effective. Moving forward, reliable physical feedback, clear attribution of modifications, and safety guarantees that remain effective even after system and environmental changes are needed. The paper's target is to enable robots to safely accumulate capabilities and continuously improve their learning methods when facing new tasks and challenges.

For republication, please contact this official account for authorization

For submissions or media inquiries: [email protected]