Robots Begin to Self-Improve, But Still Fall Short of 'Learning to Learn'

Robots are now able to modify themselves, but they are still several levels away from being able to 'learn more and become better at learning'.

Lin Yue2026-10-231435Collaborative robot Embodied Intelligence

In recent years, the development of robot learning has enabled machines to autonomously complete an increasing number of tasks. However, behind this autonomy, data collection, failure diagnosis, training plan adjustment, and deployment decisions still largely rely on researchers and engineers. Robots are becoming increasingly capable of executing tasks, but how to enable them to continuously improve remains a task primarily driven by humans. Research on self-improving agents offers new perspectives on this issue. Systems can modify code based on feedback, accumulate experience, or update models, and apply effective modifications to subsequent tasks. However, these efforts typically follow pre-set improvement processes, with models and skills being updated, yet the methods for generating and selecting updates remain unchanged. Recursive self-improvement, on the other hand, further focuses on whether this improvement process itself can be updated from experience, making subsequent learning more effective.

In recent years, robot learning advances have enabled machines to autonomously complete more tasks. But behind autonomous execution, data collection, failure diagnosis, training plan adjustment, and deployment decisions still mainly rely on researchers and engineers. Robots are increasingly good at executing tasks, but how to make them continuously improve remains a human-led endeavor. Self-improving agent research offers new ideas—systems can modify code, accumulate experience, or update models based on feedback, applying effective modifications to subsequent tasks. However, these works usually follow preset improvement procedures: models and skills update, but the methods producing and selecting updates remain unchanged. Recursive self-improvement further asks whether this improvement procedure can also update from experience, making subsequent learning more effective.

When recursive self-improvement enters the physical world, this goal faces more challenges. The observations robots receive are often incomplete and noisy, making it difficult to accurately determine the cause of failures; validating a modification requires repeated trials, accompanied by environment resets, hardware wear, and even irreversible consequences. Therefore, self-improvement methods in digital environments need to be re-evaluated under the constraints of physical feedback, experimental costs, and safety considerations. Addressing these issues, researchers from institutions such as Shanghai Jiao Tong University, University of California, Berkeley, Stanford University, Massachusetts Institute of Technology, Google, Microsoft, NVIDIA, and Xiaomi, have published the paper 'Who Improves the Robot? Toward Physical Recursive Self-Improvement,' which integrates the update of task capabilities and the improvement process into a unified framework, exploring how robots can continuously improve using physical experience, and how safety and verification mechanisms support this process.

When recursive self-improvement enters the physical world, this goal faces more difficulties. Robot observations are often incomplete and noisy; failure causes are hard to accurately determine; validating a modification requires repeated trials with environment resets, hardware wear, and even irreversible consequences. Therefore, self-improvement methods from digital environments must be re-examined under physical feedback, trial costs, and safety constraints. Around these issues, researchers from Shanghai Jiao Tong University, UC Berkeley, Stanford University, MIT, Google, Microsoft, NVIDIA, and Xiaomi, in the paper "Who Improves the Robot? Toward Physical Recursive Self-Improvement," integrate task capability updates and improvement process updates into a unified framework, reviewing how robots continuously improve using physical experience and how safety and validation mechanisms support this process.

The paper provides three contributions regarding what is improved, through what path, and how to prove effectiveness. First, it establishes a unified framework for understanding Physical RSI. The paper defines Physical RSI and provides L0 to L5 grading, from whether the update is task capability or the improvement process, and whether the gain is limited to the same task or can be transferred across tasks, offering a common analytical framework for comparing different studies. Second, it outlines the full implementation path from capability updates to improvement process evolution. The paper focuses on harness adaptation and model training, analyzing how memory, skills, workflows, and training methods affect task execution and subsequent improvements, and integrating safety control and update verification into the same analysis. Finally, it clearly defines the evidence required to validate recursive gains. Under the same initial system, experience, and resource budget, through independent cross-task evaluations, it compares the subsequent gains produced by new and old improvement processes, and identifies the evidence gaps in existing research.

The paper offers three contributions around what was improved, through what path, and how to prove effectiveness. First, it establishes a unified framework for understanding Physical RSI, providing the definition and L0–L5 grading, using two dimensions—whether task capability or improvement procedure is updated, and whether gains are confined to the same task or transfer across tasks—to provide a common analytical framework for comparing different studies. Second, it reviews the full-chain implementation path from capability updates to improvement procedure evolution, analyzing how memory, skills, workflows, and training methods affect task execution and subsequent improvement around Harness adaptation and model training, incorporating safety control and update validation into the same analysis. Finally, it clarifies the evidence needed to test recursive gains—under the same initial system, experience, and resource budget, through independent cross-task evaluation, comparing subsequent gains produced by new versus old improvement procedures, identifying evidence gaps in existing research.

The grading from L0 to L5 is worth looking at separately. L0 is without the ability to autonomously generate and retain updates, only re-planning during each task execution, or improvements relying on human intervention. L1 is autonomous updating of the task execution system, gaining benefits within the same task, with the improvement process remaining fixed. L2 is autonomous updating of the improvement process itself, gaining benefits within the same task. L3 is autonomous updating of the task execution system, bringing cross-task benefits, with the improvement process remaining fixed. L4 is autonomous updating of the improvement process itself, bringing cross-task benefits. L5 is on the basis of L4, through independent, controlled comparisons, confirming that the new process of continuous updates brings better subsequent cross-task benefits than the previous version. The significance of this grading lies in distinguishing between making robots better and making their methods of improvement more effective. Previously, we discussed BetaMem from Beta Infinite, which performs recursive self-improvement through memory strategies, and Physical RSI from Chuguan Yuanchuan, as well as Real-World RSI from Euler Wanxiang. These works mostly remain at L1 to L2, capable of updating the task execution system or the improvement process itself, but have not yet confirmed that under the same conditions, the new process brings better subsequent gains than the old one.

The L0–L5 grading deserves separate attention. L0 is no autonomous generation and retention of updates—only re-planning during each task execution, or improvement relying on human intervention. L1 is autonomously updating the task execution system, gaining on the same task while the improvement procedure stays fixed. L2 is autonomously updating the improvement procedure itself, gaining on the same task. L3 is autonomously updating the task execution system, bringing cross-task gains while the improvement procedure stays fixed. L4 is autonomously updating the improvement procedure itself, bringing cross-task gains. L5 is on top of L4, through independent, controlled comparison, proving continuously updated new procedures bring better subsequent cross-task gains than the previous version. This grading's significance is distinguishing "the robot got better" from "the robot's method of improving itself got more effective." Earlier we discussed Beta Infinite's BetaMem doing memory strategy recursive self-improvement, Chiuguang Yueqian's Physical RSI, and Ola Universe's Real-World RSI. These works mostly stay at L1 to L2—updating task execution systems or improvement procedures themselves, but not yet proving that under the same conditions, new procedures bring better subsequent gains than old ones.

The implementation path outlined in the paper is divided into three parts. In terms of harness adaptation, the system resources outside the model can also become objects of improvement. Execution feedback is used for diagnosing failures, updating memory, skills, and workflows. Effective modifications are retained, and modification records support subsequent diagnosis and solution selection. Recent research has also begun to further adjust the methods for generating these updates, including diagnostic guidance, memory construction strategies, and search procedures, providing references for introducing the evolution of improvement methods into physical scenarios. In terms of model training, data collection, model updates, and re-execution are connected into a training loop, continuously converting physical interaction data into subsequent improvements. Some works further treat the training mechanism itself as an object of improvement. In terms of data preparation, there are also studies that adjust data ratios, simulation distributions, or sampling strategies based on learning effectiveness.

The paper's implementation path divides into three parts. On Harness adaptation, system resources outside the model can also become improvement objects. Execution feedback is used to diagnose failures, update memory, skills, and workflows. Effective modifications are retained; modification records support subsequent diagnosis and solution selection. Recent research also further adjusts the methods generating these updates, including diagnostic guidance, memory construction strategies, and search procedures, providing references for introducing improvement method evolution into physical scenarios. On model training, data collection, model updates, and re-execution connect into a training loop, continuously converting physical interaction data into subsequent improvements. Some works further treat the training mechanism itself as an improvement object; in data preparation, research has adjusted data ratios, simulation distributions, or sampling strategies based on learning effects.

In terms of safety, the final strategy passing safety tests does not imply that previous explorations were risk-free. Safety reinforcement learning has provided constraints for the exploration process, while safety transfer research has clarified the conditions for ensuring continuity. These advances mainly target specific components and scenarios. How to maintain safety assurance while improving the evolution of the process itself, and how to assess the risks brought by repeated trials during the learning process, remain key issues facing Physical RSI. In terms of validation, the fact that code can run is just the first step. Whether modifications have fixed previous failures needs to be judged by re-execution; when shared skills change, it also requires back-testing of previously successful tasks. Internal validation determines which updates are retained, while external independent assessors evaluate the actual benefits. If the validator itself is modified, then an external independent assessor must judge the effectiveness of the new version to avoid the system gradually relaxing its acceptance criteria.

On safety, a final policy passing safety tests does not mean earlier exploration had no risk. Safe reinforcement learning has provided constraints for exploration; safe transfer research clarifies conditions for continued guarantees. These advances mainly target specific components and scenarios; how to maintain safety guarantees when the improvement procedure itself evolves, and assessing risks from repeated trials during learning, remain key Physical RSI issues. On validation, code running is only the first hurdle. Whether a modification fixed the original failure still needs re-execution to judge; when shared skills change, previously successful tasks also need regression testing. Internal validation decides which updates are retained; external independent evaluators test actual gains. If the validator itself is modified, external independent evaluators must judge the new version's effectiveness, preventing the system from gradually loosening its own acceptance criteria.

The paper proposes six unresolved issues. Learning from uncertain physical experiences, how to extract reliable learning signals and avoid error feedback from being amplified through the inheritance of improvement mechanisms. Causal attribution between components, how to identify the target of modification within limited physical experiments and distinguish the actual impact of each modification. Non-stationarity in the improvement loop, whether past effective modifications and improvement strategies remain valid when the system, environment, and existing capabilities change. Reward speculation and evaluation leakage, how to prevent the system from exploiting vulnerabilities to gain high scores and maintain the independence of evaluation. Safety maintenance during recursive updates, how to maintain safety assurance and control the accumulated risks of multiple rounds of experiments when the improvement process changes. Benchmark evaluation of recursive gains, whether the improved performance of the subsequent system is due to more data, stronger starting points, or the benefits brought by updated improvement mechanisms.

The paper proposes six unresolved questions. Learning from uncertain physical experience—how to extract reliable learning signals and avoid error feedback amplifying through inherited improvement mechanisms. Causal attribution among Harness components—how to pinpoint modification targets in limited physical trials and distinguish each modification's actual effect. Non-stationarity of the improvement loop—whether previously effective modifications and improvement strategies remain effective when systems, environments, and existing capabilities change. Reward hacking and evaluation leakage—how to prevent systems from exploiting loopholes for high scores and maintain evaluation independence. Safety preservation in recursive updates—how to maintain safety guarantees and control cumulative risks of multiple trials when improvement procedures change. Benchmark evaluation of recursive gains—whether successor systems perform better because of more data, stronger starting points, or gains from updated improvement mechanisms.

This paper and the previous discussions on Unitree's H2-X, AGCO bringing Digit into combine factories, Mitsubishi Electric integrating physical AI into CNC machines, and Liangyuan New Creation stating that generalization ability is the largest barrier to scaling, represent different aspects of the same stage of industry development. Those discussions focus on whether robots can work, whether they can work stably, and whether they can scale up. This paper focuses on another issue: whether robots can continuously improve their working methods while performing tasks, and whether this improvement can be proven to be effective. Previous discussions have covered the two types of god-making movements in Silicon Valley and Wudaokou, as well as the cooling down of funding in the embodied intelligence industry, the postponement of IPOs, and the scrutiny of revenue eligibility. At this stage, the value of the Physical RSI direction does not lie in whether it can immediately make robots stronger, but in providing a testable framework that turns the slogan of 'learning more and becoming better' into a research proposition that can be graded, evaluated, and falsified.

This paper, alongside the previously discussed Unitree H2-X, AGCO bringing Digit into a combine factory, Mitsubishi Electric connecting physical AI to CNC machines, and Liangyuan Xinsheng saying generalization is the biggest scaling barrier, are different sides of the same industry stage. Those discussions focused on whether robots can work, work stably, and work at scale. This paper focuses on another question—whether robots can continuously improve their way of working during work, and whether this improvement can be proven effective. Earlier we discussed Silicon Valley and Wudaokou's two "god-making" movements, embodied intelligence industry financing cooling, IPO postponements, and revenue eligibility scrutiny. At this stage, Physical RSI's value lies not in whether it can immediately make robots stronger but in providing a testable framework, turning "the more they learn, the better they learn" from a slogan into a research proposition that can be graded, evaluated, and falsified.