Berkeley Robotics Demonstrates Agentic Real-to-Sim Pipeline Combining Fable, Astra, and RSS for Rapid Environment Reconstruction

Summary

Berkeley Robotics presents a demonstration where an agentic system combines Claude Fable and GPT-6 Astra with RoboSimStudio (RSS) to reconstruct physical environments from online images or Amazon links within two hours. This workflow tunes simulation parameters—including geometry, dynamics, deformations, materials, lighting, texture, and camera extrinsics—to match input video, addressing prior limitations in scaling agentic robotics to new environments. The post notes that while combining AI models remains more art than science currently, this approach represents an evolution supported by recent results on agentic weight tuning and RSI.

Source: Ken Goldberg — X Robotics · Read original article ↗

Article text · Original source · English

Each AI model has different strengths. Combining and harnessing them is currently more art than science but this will evolve, esp given new results on agentic weight tuning and RSI.

Note how KEC combines Fable and Astra with RSS to rapidly tune complex simulation parameters for geometry, dynamics, deformations, materials, lighting, texture, and camera extrinsics to match a given input video. This fills a previous gap in Agentic Robotics ability to scale to new environments, objects, and tasks.

Quoted postKaiyuan Eric Chen@keplerccccc

We recently found a new way to do agentic real2sim: using Claude Fable commanding GPT-6 Astra. Give it an online image, or even an Amazon link, and they collaborate to recreate the world in RoboSimStudio, ready for robots under 2 hours

View the quoted post on X