Revisiting Temporal Regularization for Smooth Control in Deep Reinforcement Learning
Revisiting Temporal Regularization for Smooth Control in Deep Reinforcement Learning
This research addresses nonsmooth action oscillations in deep reinforcement learning policies that hinder physical robot deployment. The authors challenge the assumption that temporal regularization cannot provide spatial smoothness under observation noise, proving it bounds expected action differences between states sharing a next state. They introduce CATS (Conditioning for Action using only Temporal Smoothness), which integrates a temporal penalty with linear ramp-up to learn rewarding behavior before progressively smoothing actions. Experiments demonstrate that CATS substantially reduces action oscillation without degrading task performance and with minimal computational overhead.
The paper proposes CATS, a method combining temporal regularization with linear ramp-up to achieve spatial smoothness in deep reinforcement learning policies while preserving task performance, validated through simulation and real-world experiments.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
arXiv:2610.07910v3 Announce Type: replace-cross Abstract: Deep Reinforcement Learning policies can produce nonsmooth action oscillations that hinder deployment on physical robots. Existing architectural and penalty-based approaches seek spatial smoothness by directly reducing sensitivity to changes in state inputs, but their broad constraints can degrade task performance as stronger smoothing is pursued. Temporal reg
Open source S2
turn preservation and both temporal and spatial smoothness. Experiments in both simulation and the real world show that CATS substantially reduces action oscillation without degrading task performance, with little computational overhead.
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org