Beyond Action Residuals: Real-World Robot Policy Steering via Bottleneck Latent Reinforcement Learning
Beyond Action Residuals: Real-World Robot Policy Steering via Bottleneck Latent Reinforcement Learning
This paper introduces ZPerturbation Reinforcement Learning (ZPRL), a method for adapting pretrained robot policies through online reinforcement learning in a compact bottleneck latent space. By applying reward-driven residual perturbations to a frozen pretrained policy, ZPRL achieves improved sample efficiency and higher success rates on real-world manipulation tasks compared to baseline imitation policies, while generating smooth actions without additional regularization.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- code: Available Artifact link Open source S5
- weights: not_reported
- dataset: not_reported
- paper: not_reported
- code: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
online RL control space through reward-driven residual perturbations while freezing the pretrained policy. We instantiate ZPRL on flow-matching policies and evaluate it on eight simulation tasks and four real-world tasks. Across diverse manipulation settings, ZPRL improves sample efficiency with competitive or better final performance. In the real world, ZPRL improves the average success rate on four tasks by 33.7 pe
Open source S4
Beyond Action Residuals: Real-World Robot Policy Steering via Bottleneck Latent Reinforcement Learning
Open source S1
arXiv:2605.19919v2 Announce Type: replace Abstract: Pretrained imitation policies have become a strong foundation for robot manipulation, but online adaptation can further improve policy performance within their pretrained behavioral support. A central question is therefore how reinforcement learning (RL) should adapt policies after offline pretraining. Existing lightweight methods commonly apply residual corrections
Open source S2
directly in action space. We investigate how the steering interface affects online RL. In this work, we propose Z-Perturbation Reinforcement Learning (ZPRL), an approach that steers pretrained policies through a compact bottleneck latent rather than through policy weights or output actions. Building on an offline-learned bottleneck policy interface introduced by previous work, ZPRL turns this representation into an
Open source S3
rcentage points over imitation base policies. Our qualitative and quantitative results also indicate that ZPRL combines task success with smooth actions without explicit temporal filtering or smoothness regularization. These results suggest that a compact, task-aligned bottleneck latent provides an effective interface for online RL adaptation. Videos and code can be found at https://manutdmoon.github.io/ZPRL/.
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org