ExRL: Sample-Efficient Online Adaptation of Long-Chunk Policies via Execution Length Control
ExRL: Sample-Efficient Online Adaptation of Long-Chunk Policies via Execution Length Control
This research introduces ExRL, a method that shifts adaptation from optimizing actions to optimizing execution length for long-chunk robot policies. By using a lightweight adapter trained with off-policy RL algorithms, ExRL reduces the dimensionality of the adaptation space while maintaining expressivity. The approach enables sample-efficient adaptation across diverse long-chunk policies in both simulation and real-world robotic tasks with minimal task-specific hyperparameter tuning.
The paper proposes ExRL, a method that optimizes execution length rather than actions to enable sample-efficient adaptation of long-chunk robot policies, validated across simulation benchmarks and real-world robotic tasks with minimal hyperparameter tuning.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
arXiv:2611.13482v1 Announce Type: new Abstract: Predicting long action chunks, often spanning dozens of timesteps, has become a key ingredient in increasingly capable robot policies. In this paper, we find that adapting these long-chunk policies with existing reinforcement learning (RL) methods can require many online samples due to the high dimensionality of the action chunks being optimized, making such adaptation
Open source S2
impractical in real-world scenarios where interaction data are costly to collect. Our key insight is to shift adaptation from optimizing actions to optimizing how long to execute each predicted action chunk. This keeps the adaptation space low-dimensional while its expressivity grows with the chunk size. We thus introduce Execution at the Right Length (ExRL), a simple approach which augments the base policy with a li
Open source S3
ghtweight adapter that predicts the execution length of each chunk and is trained using off-policy RL algorithms. Across simulation benchmarks and real-world robotic tasks, ExRL enables sample-efficient adaptation of a diverse range of long-chunk policies with minimal task-specific hyperparameter tuning.
Open source S4
ExRL: Sample-Efficient Online Adaptation of Long-Chunk Policies via Execution Length Control
Open source S1
Source：arXiv Robotics — research abstracts · arxiv.org