WeaveRL: Weaving Reconstruction into Scene-Aware Fabrics for Perceptive Reinforcement Learning
WeaveRL: Weaving Reconstruction into Scene-Aware Fabrics for Perceptive Reinforcement Learning
This paper introduces WeaveRL, a GPU-accelerated method that reconstructs scenes as surfels during active rollouts to enable reinforcement learning policies to operate over sensor-derived geometry rather than static, hand-specified representations. On collision-dense manipulation tasks, the approach enables policies to handle geometrically complex scenes where primitive-based baselines fail while maintaining sim-to-real transfer. Policies trained with scene-aware fabrics show improved robustness to novel geometry at test time, increasing collision-free task completion under unseen obstacles from 35% to 61%. The authors release their reconstruction system, training code, and test dataset.
The authors present a GPU-accelerated method that reconstructs scenes as surfels during active rollouts to enable reinforcement learning policies to operate over sensor-derived geometry rather than static, hand-specified representations.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- Environment
- SimulationOpen source S3
- paper: not_reported
- code: Reported pending Open source S4
- dataset: Reported pending Open source S4
- weights: not_reported
- code: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
3D perception into massively parallel RL training has so far been inaccessible. We introduce a GPU-accelerated method that reconstructs the scene as a collection of surfels across thousands of parallel simulation instances during active rollouts. This lets policies operate over sensor-derived, rather than hand-specified, geometry. On a suite of collision-dense manipulation tasks, our surfel fabrics enable policies to
Open source S3
tackle geometrically complex scenes where primitive-based baselines fail, while maintaining sim-to-real transfer. Furthermore, policies learned with a scene-aware fabric are more robust to the introduction of novel geometry at test time, improving collision-free task completion under unseen obstacles from 35% to 61%. We release our reconstruction system, training code and test dataset to spur research in this direct
Open source S4
arXiv:2609.18685v2 Announce Type: replace Abstract: Reinforcement learning allows robots to acquire complex skills, but producing policies for geometrically complex manipulation remains difficult. A promising approach is to learn on top of collision-avoidant controllers, such as geometric fabrics. However, these approaches have relied on static, hand-specified representations of the scene. Integrating active, online
Open source S2
WeaveRL: Weaving Reconstruction into Scene-Aware Fabrics for Perceptive Reinforcement Learning
Open source S1
Source：arXiv Robotics — research abstracts · arxiv.org