PyRUA-DataFarm: LLM Agent as a Cost-Efficient Teleoperator for Real-Ready Robot Data
PyRUA-DataFarm: LLM Agent as a Cost-Efficient Teleoperator for Real-Ready Robot Data
This research introduces PyRUA-DataFarm, a system where an LLM agent acts as a cost-efficient teleoperator to generate real-ready robot data. The method automatically rebuilds tasks in simulation by writing scenes, expert controllers, and success judges, then renders photographic videos conditioned on real images. Policies trained on this synthetic data match those trained on teleoperated episodes across five real-robot tasks, maintaining high success rates even under unseen conditions like different tablecloths and lighting. The approach reduces costs to approximately $107 per hour of three-view data compared to traditional teleoperation.
The authors present a method using LLM agents to generate synthetic robot demonstration data in simulation that matches real-robot performance on five tasks, with costs significantly lower than traditional teleoperation; however, the study relies on independent judges for certification and does not claim universal deployment readiness.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- Control
- TeleoperatedOpen source S1
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
What remains unknown
Dataset licensing and commercial-use rights are not established here.
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
PyRUA-DataFarm: LLM Agent as a Cost-Efficient Teleoperator for Real-Ready Robot Data
Open source S1
arXiv:2611.05320v1 Announce Type: new Abstract: Teleoperation remains the default way to collect robot demonstrations, but every episode costs robot and operator time, and every new task starts from scratch. We ask whether a coding agent can take over the teleoperator's job. In our DataFarm, the agent receives a data request and one calibration episode of the robot in the target scene and rebuilds the task in simulat
Open source S2
ion: it writes the scene, an expert controller, and a success judge. The simulator supplies exact actions, and a video model conditioned on real images renders each rollout as photographic video, generating all camera views together consistently. The output is real-ready data: synchronized camera views labelled with the robot's actions, on which a policy can be trained directly and deployed on the real robot. Indepen
Open source S3
dent judges certify the expert and every rendered episode before admission. On five real-robot tasks, policies trained only on farm data match policies trained on as many teleoperated episodes (93 vs. 84 successes in 100 trials), and under an unseen tablecloth and coloured light they keep 87 successes where the teleoperation-trained policies fall to 46. Adapting a task is fully automatic and costs \$21 of API usage o
Open source S4
Implications for data suppliers
RoboSignal interpretation and collection questions, not statements of buyer demand.
- Confirm the required data type and collection setting with the buyer; this source does not establish a complete collection specification.
- Validate demand and acceptance criteria with a buyer before scaling. Publication, popularity and a research result do not establish a purchase commitment.
Source：arXiv Robotics — research abstracts · arxiv.org