SceneAgent: An Agentic Pipeline from 3D and 4D Captures to Simulation Environments
SceneAgent: An Agentic Pipeline from 3D and 4D Captures to Simulation Environments
SceneAgent is a research paper introducing an agentic pipeline that converts 3, 4D captures into simulation environments through object reconstruction, physical modeling, and error correction. It processes various capture types and supports policy learning with diverse object variants and scripted demonstrations. The method outperforms existing approaches in F1 scores and dynamic IoU, enabling simulated interactions and preliminary real-world policy transfer.
SceneAgent introduces an agentic pipeline that transforms 3D and 4D captures into simulation environments by integrating object reconstruction, physical modeling, and error correction. It demonstrates significant improvements in F1 scores and dynamic IoU compared to existing methods, supporting policy learning and sim-to-real transfer.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
cm
1
View original evidence
(3DGS), photogrammetry, LiDAR captures, AI-generated 3DGS scenes, and 4D videos of dynamic scenes, and combines object segmentation, physical parameter estimation, part decomposition, articulation, and deformable simulation. To support policy learning, SceneAgent generates diverse object variants ("digital sisters") and scripted demonstrations. On a cluttered bin scene from GraspClutter6D, SceneAgent raises F1 at 1
Open source S3
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
cm for every coding agent with scorable outputs both with and without its skills, and its reconstructions are better than SimFoundry: all agent configurations exceed SimFoundry's F1 score, with a best of 60.75% versus 6.9%. On three dynamic scenes from 4DCodeBench, SceneAgent raises the mean dynamic IoU of Gemini 4 Argon from 0.86% to 99.09%. We further demonstrate diverse simulated interactions and preliminary polic
Open source S4
(3DGS), photogrammetry, LiDAR captures, AI-generated 3DGS scenes, and 4D videos of dynamic scenes, and combines object segmentation, physical parameter estimation, part decomposition, articulation, and deformable simulation. To support policy learning, SceneAgent generates diverse object variants ("digital sisters") and scripted demonstrations. On a cluttered bin scene from GraspClutter6D, SceneAgent raises F1 at 1
Open source S3
Source：arXiv Robotics — research abstracts · arxiv.org