RoboSkill: A Skill Acquisition and Reuse Loop for Embodied Agents

2026-10-24 12:49 ·Beijing · Official NetEase Account of Quantum Bit

RoboSkill Team Submission
Quantum Bit | Official WeChat Account QbitAI

GPT-6-Astra Gave a significant shock to embodied intelligence.

Now, general agents represented by Astra have overcome the previous bottleneck in robotics where tasks must first collect data.

Without pre-collecting demonstration data for the target task, nor doing targeted training, Robot they are already able to complete tasks by having the agent observe, reason, write code, and figure things out on-site.

However, although relying on on-site exploration saves the cost of data collection, the Agent still needs to spend a lot of time repeatedly performing the steps of 'observing - reasoning - executing - checking the results'.

From 'being able to complete' to 'efficiently completing,' it is still a hurdle that must be overcome.

Fudan University Trustworthy Embodied Intelligent Research Institute team thus proposed RoboSkill to store the effective methods and code as skills, allowing robots to take fewer detours in their next tasks.

When robots enter new scenarios, how much data do they need to collect?

△VLA/WAM learned to vertically hammer nails, but without adding new data, they would not hammer nails diagonally. Agents can solve this task through thinking, but the speed is very slow.

Visual-Language-Action Model (VLA) and World Action Model (WAM) usually first undergo pre-training with robot data, then adapt to specific tasks. However, when truly deployed, they often still need to collect a batch of data in the target scenario for fine-tuning. To a large extent, what tasks the model can complete still depends on what is in this batch of data.

For example, training with data for grasping blocks does not necessarily mean the model will open drawers. How much help pre-training can provide for new tasks remains a question that this approach still needs to answer.

The common practice currently is to continue collecting data in the deployment scenario: if a task is missing, then supplement it; if a new situation arises, add the corresponding demonstration. The hope is that the model sees enough to handle the deployment. However, this way, every time a new task is added or an environment is changed, a new batch of data may need to be collected.

Robot data is particularly difficult to collect. Taking autonomous driving as a comparison, people originally need to drive, and autonomous driving data can be continuously obtained during daily driving. General-purpose robots do not have such consumer demand yet. Now, they need to hire people to remotely operate robots, placing items one by one, demonstrating actions, and resetting the scene. To increase data, human resources, equipment, and facilities also need to be invested.

Moreover, spending more money to collect more data does not necessarily fill in the situations the model has not seen. Operators have their own habits, and the same task may always be completed in a few fixed ways; the items, arrangements, and tasks in the scene are also relatively limited. Even if the number of data collections increases, the model may still see the same similar operations. When it is truly deployed, changing an object or a different arrangement may again lead to problems.

Tesla benefits from its massive fleet of users, and data has already accumulated for about ten years. Robots do not have the same user base, yet they need to handle more tasks and operation methods.

If every new situation requires data collection to solve, how long would it take for robots to adapt to a new scenario they have never seen before?

Robot Agent is revolutionizing the embodied paradigm

The recent emergence of direct control capabilities for robots to complete tasks on agents like GPT-6-Astra has brought robot control research into a new stage.

Fudan University team and their collaborators in the paper《Explore, Execute, Evolve: A Skill Acquisition and Reuse Loop for Embodied Agents》even without pre-collecting demonstration data for the target task or conducting targeted model training agents can still control robots to complete various operational tasks through on-site observation, reasoning, and coding. Agent can still control robots to perform various operational tasks through on-site observation, reasoning, and coding.

In the LIBERO-10 simulation environment, allowing multiple rounds of attempts within a four-hour budget, the GPT-6-Astra and Opus 5, which do not use existing skills, both achieved final success rates of100%Fable 5.1 and GPT-5.6-Sol respectively reached97.5%and the82.5%. On real robots, GPT-6-Astra achieved success rates of91.7%and the80%in six simple tasks and six high-difficulty tasks respectively. These results show that agents can already solve various robot operation tasks through on-site reasoning and interactive exploration.

However, behind the high success rate is a long execution time. In simulation, GPT-5.6-Sol's average time reached94.5minutes, Opus 5 also needed80.9minutes; on real robots, GPT-6-Astra successfully completed simple and high-difficulty tasks, with average times of20.3minutes and27Minutes. Moving from 'being able to complete' to 'efficiently completing' remains a problem that robot agents must solve.

Why does it take so long for an Agent to complete a task?

The research team found that agents typically need to repeatedly go through the process of 'observing—reasoning—executing—checking results.' Each time encountering a new source of uncertainty, they may need to pause and reassess. A single movement of a robotic arm might be short, but the reasoning between actions could take several minutes, and with multiple rounds, time accumulates continuously.

For this, the team proposed RoboSkill, to reduce such expenses from two aspects: by Haptic Feedback, faster confirmation of the current physical state; by skill accumulation, reusing already explored effective methods.

The former helps the Agent quickly determine whether an action is properly executed, while the latter retains the executed strategies and code, reducing redundant exploration and reasoning in subsequent tasks.

Adding tactile feedback allows the Agent to quickly confirm the outcome of an action

Did the gripper touch the object? Mechanical arm moving downward movement, did it already contact the table? Based solely on images, these states may not be easy to determine. The Agent may need to observe repeatedly, make small adjustments, and then confirm whether the action is properly executed.

Tactile feedback provides direct basis for these judgments. Contact information such as pressure, force, and torque helps the Agent determine what actually happened, allowing timely decisions to continue or adjust.

In experiments, GPT-6-Astra relying solely on vision could also complete all simulation tasks, but with tactile feedback, the average time dropped from25.35minutes to15.93minutes; in the real machine task of extracting test tubes, the average time for successful trials also dropped from33.63minutes to20minutes to read. Tactile feedback allows the Agent to quickly confirm the state, thus completing the task faster.

Accumulating Skill, letting explored methods be retained

Beyond faster confirmation of the state, another time cost comes from repeated exploration. If the Agent finds an effective method in one task, but does not save it, it may need to re-derive and try again when encountering similar problems later.

RoboSkill allows the Agent to review the execution process after the task is completed, organize effective methods into Skills, and continue updating based on subsequent execution results. For example, after using contact detection to measure the table height, the Agent can save this set of measurement methods. When moving to a new scenario, the height needs to be remeasured, but the measurement steps already have methods to follow.

Skill accumulation brings actual efficiency gains: the average time for GPT-5.6-Sol in simulation dropped from94.5minutes to35.9minutes, and simple and high difficulty tasks on real machines also benefited from acceleration.

Saving code in Skill allows methods to be directly reused

What should be left in the Skill determines how much work the Agent can save. A sentence 'First measure the desk height' provides a strategy, but how to move, read feedback, and determine contact still requires the Agent to write a program.

Therefore, RoboSkill saves both the textual instructions and the actual executed actions: the textual explanation covers the conditions and operational strategies, while the code retains the specific implementation, allowing the Agent to modify and invoke based on the current scenario. Code Together, they are preserved: the textual explanation applies to the conditions and operational strategies, and the code retains the specific implementation, allowing the Agent to modify and invoke based on the current scenario.

During the experiment, after the skill changed from “only text” to “text plus code,” the average time taken by the four Agents all decreased. Among them, the time taken by GPT-5.6-Sol decreased from67.1minutes to39.9minutes, the success rate was increased from15%to67.5%. Keeping the code allows subsequent executions to have a starting point based on practice, also reducing the overhead of rederiving actions from textual strategies.

Other experiments also showed that skills can save execution time across tasks, improve performance across models, and continue updating through subsequent executions.

, using touch to improve exploration efficiency, using code to retain exploration results, and then continuously updating skills through execution. RoboSkill connects the efficiency issues of single-task execution with long-term skill accumulation, making exploration the starting point for subsequent capability growth, moving towards embodied RSI.

Author Introduction

Xie Siceng, Chen Yitong, doctoral students at the Trustworthy Embodied Intelligence Institute, Fudan University. Among them, Xie Siceng's research direction is robot agents, vision-action-language models, and cross-entity learning. This study was conducted in collaboration with Shanghai XinZhi Embodied Intelligence Technology Co., Ltd. The corresponding authors are Prof. Wu Zuxuan and Prof. Jiang Yugang from Fudan University.

Paper: arxiv.org/abs/2609.37810
Code: github.com/SII-dannyXSC/RoboSkill