Ordered Policy Optimization
Ordered Policy Optimization
This paper introduces Ordered Policy Optimization (OPO), an improvement over Proximal Policy Optimization (PPO) that adapts clipping radii for extreme importance ratios based on their statistical reliability. By sorting importance ratios within minibatches and assigning adaptive sample-wise radii derived from a prescribed tail profile, OPO aims to control harmful updates more effectively than fixed clipping. Experiments on MuJoCo and Atari benchmarks show that this modification achieves strong performance relative to established baselines.
The authors propose Ordered Policy Optimization (OPO), a modification to Proximal Policy Optimization that adaptively clips extreme importance ratios based on their statistical reliability rather than using a fixed range, demonstrating improved performance on MuJoCo and Atari benchmarks.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
Source：arXiv Robotics — research abstracts · arxiv.org