Beyond Ensemble Disagreement: Predictive STL Monitoring of RL Locomotion under Distribution Shift
Beyond Ensemble Disagreement: Predictive STL Monitoring of RL Locomotion under Distribution Shift
This research investigates the reliability of ensemble disagreement as a safety indicator for reinforcement learning locomotion policies facing distribution shifts. Using frozen Ant-v5 and HalfCheetah-v5 policies, the authors evaluate predictive Signal Temporal Logic (STL) monitoring with closed-loop world-model rollouts across external-force, observation-noise, and mass shifts. Results show that while predictive robustness provides useful STL-specific ranking, it offers little additional value over current-state robustness metrics, which remain within 0.03 AUROC of predictive robustness under severe noise. The study cautions against treating ensemble disagreement as a reliable proxy for safety uncertainty, noting that violations persist at rates up to 64.2% under high noise conditions and that composite targets often lack positive violations in evaluated seed-condition slices.
The study evaluates predictive Signal Temporal Logic (STL) monitoring for reinforcement learning locomotion policies under distribution shifts, finding that ensemble disagreement does not reliably track safety uncertainty and that current-state robustness metrics offer comparable performance to predictive approaches in many tested scenarios.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
robustness
0.03
View original evidence
-state robustness without prediction comes within $0.03$ AUROC of predictive robustness. Penalizing predicted robustness by disagreement does not consistently improve ranking. Ant provides complementary per-specification evidence, but its composite target has no positive violations and its any-violation target is single-class in most evaluated seed-condition slices. Overall, predictive robustness provides useful STL-
Open source S4
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
lfCheetah-v5 policies using closed-loop world-model rollouts and a three-model ensemble. Across external-force, observation-noise, and mass shifts, predicted robustness ranks HalfCheetah's future STL violations strongly, whereas disagreement does not reliably track observation prediction error or violations. Under severe noise, violations remain common ($64.2\pm24.4\%$ at $K{=}20$ across three PPO seeds), yet current
Open source S3
-state robustness without prediction comes within $0.03$ AUROC of predictive robustness. Penalizing predicted robustness by disagreement does not consistently improve ranking. Ant provides complementary per-specification evidence, but its composite target has no positive violations and its any-violation target is single-class in most evaluated seed-condition slices. Overall, predictive robustness provides useful STL-
Open source S4
Source：arXiv Robotics — research abstracts · arxiv.org