GPT and Claude, Competing for Jobs at Robotics Brain Companies

Robotics Outlook (WeChat Official Account: robot_pro)

Author | Xu Lisi

Editor | Mo Ying

Robotics Outlook reported on October 16, that in the past two days, two robot companies have showcased new progress in AI Agent applications in the field of robotics.

On one side, MicroFactory used general-purpose large models such as Opus and Astra to let two robotic arms perform continuous autonomous assembly of electronic products for 1.5 hours, with zero errors, and even completed operations that the team originally thought could not be fully automated.

On the other side, Poke & Wiggle let Claude engage in embodied research, autonomously reading papers, writing algorithms, training models, and then calling real robots for testing. Their team has completed over 6,000 real robot tests, attempting to reproduce the core conclusions of 10 robot research papers.

These two advances are actually two important paths through which general-purpose large models are influencing the field of robotics: one is directly participating in the task decision-making and execution of robots, and the other is becoming embodied researchers, participating in the development, verification, and improvement of robot control methods.

General-purpose large models are beginning to show signs of changing the research and iteration rhythm of the embodied intelligence industry.

In the past, the robotics industry has been trying to create smarter AI brains. Now, a new possibility is emerging: perhaps it is the general-purpose large models themselves that will drive the evolution of the next generation of robot brains.

I. Abandoning VLA, Fully Switching to LLM, Robots Assemble Without Errors for 1.5 Hours

MicroFactory is an American company that develops modular robot manufacturing units, mainly targeting multi-product, small-batch manufacturing scenarios such as electronic product assembly.

Today, MicroFactory founder Igor Kulakov revealed that in the past month, the company has abandoned VLA and fully switched to using general-purpose large language models to control robots.

In the public video, two robotic arms work collaboratively within a closed work unit, picking up different parts and assembling them into the customer's electronic products.

Kulakov said that the robots performed continuous autonomous operation for 1.5 hours without any errors.

More surprisingly to the team was that they initially expected only 90% of the operations to be automated, but ultimately achieved full automation without replacing the existing dexterous grippers.

However, Kulakov also mentioned that using suction cups might make some operations faster and simpler.

1. Building an Enhanced System 1/System 2 Architecture

The key behind this is the division of labor between general-purpose large models and traditional robot control programs. Kulakov described this approach as an enhanced System 1/System 2 architecture.

In this architecture, high-level reasoning is mainly handled by the AI Agent based on Opus and Astra responsible for task understanding, algorithm selection, and code development; for the fast execution layer, it can be entrusted to computer vision programs written by the model these programs run quickly, are easy to modify, and have high reliability.

In addition, this architecture brings two additional capabilities:

Both aspects point to the question of whether Chinese precision transmission companies can address In-context Learning (In-context Learning), which is a fundamental capability of large models, allowing the system to adapt to tasks by utilizing information from the current context.

The second is Continual Learning (Continual Learning), whenever encountering new situations, the large model will intervene, adjusting and optimizing the entire processing flow according to the new situation, enabling the system to continuously adapt to new task requirements.

2. Driving Robots to Complete Tasks Such as Sorting Parts and High-Precision Insertion

In the demonstration, MicroFactory showcased several specific robot operations in the design of electronic product assembly.

First, it is extracting the target object from a randomly stacked pile of parts and adjusting it to the correct orientation. For such tasks, robots not only need to identify the target but also solve the problem of inconsistent part orientations.

MicroFactory's approach is to add an auxiliary platform in the work area. The robot first picks up the object from the parts pile, places it on the platform, flips the part if necessary, and then picks it up again with the correct orientation.

Regarding high-precision insertion during the assembly process, after the robotic arm picks up the part, it still needs to precisely adjust its position before inserting it into the corresponding location.

MicroFactory let the large model write code, collect a small dataset, and then use these data to train a linear regression model. This model uses visual feedback to adjust the position of the part, allowing the robotic arm to complete more precise alignment before insertion.

This means that general-purpose large models can not only generate control code but also organize simple data collection and model training, establishing auxiliary algorithms for specific operations.

Regarding Screw picking The operation, Kulakov said, robots could originally also use a similar part flipping method to pick up screws, but considering actual efficiency, the team ultimately chose to use an automatic screw feeder that costs only $300.

However, Kulakov also admitted that the current code-based control method is more suitable for repetitive tasks in constrained environments and is not yet applicable to non-repetitive, unstructured tasks such as home cleaning.

For tasks that require continuous re-decision based on environmental changes, directly calling a large model to control the robot still has limitations in terms of speed, among other aspects.

Therefore, the progress demonstrated by MicroFactory this time does not indicate that general large models already have full-scenario robot control capabilities, but rather that through code generation, simple model training, and integration with existing hardware, they begin to take on some robot system integration and task development work.

II. Claude begins to conduct robot research, 6,000+ real robot experiments validate 10 papers

In addition to controlling robots in practice, general large models have also begun to act like researchers, taking on robot research work.

On October 14, the robot company Poke & Wiggle announced its latest results: the company used Claude Opus 5.5 to let an AI Agent attempt to reproduce the core conclusions of 10 recent robot research papers, completing over 6,000 real robot tests in total.

According to the result disclosed by the company's co-founder Nicolas Keller, some of the core conclusions of the papers were successfully reproduced, some methods' actual effects were lower than those reported in the original papers, and some methods could not be reproduced on its robot platform.

1. From reading papers to real robot experiments, AI takes over the complete robot research process

In order to allow AI to conduct research independently, Poke & Wiggle built a robot experiment field in Munich, Germany, and through its self-developed Poker platform, encapsulated data collection, model training, and real robot evaluation functions as standardized APIs.

In this system, the research team only needs to provide the paper and the conclusions to be verified, and the AI Agent can autonomously read the paper, select experimental tasks, write algorithm code, and call GPUs to train models.

After training, the Agent can also apply for real robot testing, comparing the new method with the baseline model, and analyzing task success rate and progress.

The entire process is mainly driven by Claude Opus 5.5, while the experiment personnel are responsible for scenario preparation, environment reset, and quality checks, and researchers intervene for review when necessary.

To ensure the comparability of the experiment, the platform will also control the initial placement of objects, so that different models are tested under the same scenario.

2. Reproducing 10 papers, some methods' effects are validated

The 10 papers reproduced in this case cover multiple directions, including robot action representation, visual features, policy learning, and data screening.

Among them, the core conclusions of Patch Policy and DSD were successfully reproduced.

Patch Policy proposes using dense visual features to improve robot control. In Poke & Wiggle's experiment, compared to using a single CLS Token, dense visual features increased the task success rate by 22 percentage points.

DSD (Direction-Scale Decomposition) is an action representation method that encodes robot actions by decomposing them into direction and magnitude, improving the effectiveness of action learning.

In this experiment, Claude first ran continuously for about 25 hours autonomously, but concluded that DSD did not significantly improve the success rate.

After researchers checked, they found that the Agent had modified the baseline normalization parameters specified in the original paper, affecting the experimental results.

After manual correction and retraining, the average success rate of DSD increased from 32.0% with traditional methods to 45.3%, an increase of 13.3 percentage points.

In addition to paper reproduction, Poke & Wiggle also demonstrated the AI Agent's ability to autonomously debug.

In one experiment, the Agent found that the success rate of the robot's 10 consecutive tests was 0%, so it actively downloaded the runtime logs, located the problem in the code it had modified, then rolled back the code, updated the deployment image, and restarted the experiment.

From the discovery of the anomaly to the start of the experiment after the fix, it took less than 35 minutes.

The team also let the Agent autonomously connect to Microsoft's newly released VLA model Rho, and completed model integration, training, and real robot testing the next day, achieving an 87% success rate in the screw removal task.

However, Poke & Wiggle also mentioned that the AI Agent still misunderstands papers or ignores issues found during code reviews, and key research judgments still require human intervention.

Conclusion: Standing on the correct side of the Bitter Lesson

AI researcher Richard Sutton once pointed out in 'The Bitter Lesson' that In the long term, general methods that can fully leverage the growth of computational scale often have greater potential than specialized methods designed based on human experience.

MicroFactory founder also admitted that adopting cutting-edge general-purpose large models means standing on the correct side of 'The Bitter Lesson,' allowing direct benefits from the global investment on the order of several billion dollars in research and development of advanced models, without having to bear the cost of developing foundational models alone.

The exploration of Poke & Wiggle further indicates that the value brought by general-purpose large models is not only reflected in robot task execution, but may also extend to the research and improvement of robot control methods, allowing AI to participate in the programming, training, and experimental work that researchers used to repeatedly perform.

This also means that in the future, competition in the robotics industry may not only be about who can create a smarter robot brain, but also about who can more quickly convert advances in general AI into new capabilities for robots in the real world. Return to the previous page for more information