Detecting Silent Faults at the Environment-Trainer Boundary in Reinforcement Learning
Detecting Silent Faults at the Environment-Trainer Boundary in Reinforcement Learning
This research introduces a method to detect silent faults occurring at the boundary between reinforcement learning trainers and external environments. By implementing a step ledger on both sides of the wire protocol, the authors demonstrate that their approach can catch and locate every injected fault across various robotics benchmarks (Pendulum, HalfCheetah, Hopper, Pusher) with minimal overhead (median 3% more training time). The study evaluates up to 23 types of faults in Gymnasium 1.3 and other multi-agent libraries, recommending specific logging practices for robust evaluation.
The study proposes a step ledger and episode count tracking method to detect silent faults at the environment-trainer boundary in reinforcement learning, validated across multiple environments including Pendulum, HalfCheetah, Hopper, and Pusher.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
faults injected into wire protocol
≤23
View original evidence
arXiv:2611.08144v1 Announce Type: new Abstract: Robot reinforcement learning increasingly runs environments outside the trainer: in another process, on another machine, in another language. Many faults at this boundary raise no error. We make the boundary a wire protocol and ask what notices a fault in it. On Pendulum, HalfCheetah, Hopper and a 7-joint arm (Pusher), we injected up to 23 faults into what crosses the w
Open source S2
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
arly all ended on other weights and, for reward faults, logged other returns. Checks of the run itself, not of its outcome, saw every fault. A step ledger kept on both sides and compared step by step caught and located every fault that crossed, on Pendulum, Hopper and Pusher on one machine and on Pendulum across two, with no fault-free run and, on Pendulum, a median 3% more training time. An episode record kept at th
Open source S4
ire and 2 into the trainer's log, from one value to every value. Unless every value was wrong, a learning-curve threshold caught at most one run in ten, and a comparison with ten fault-free seeds at most 21%. A clean evaluation of all 1,060 final policies agreed with the curves: of the 728 boundary-fault runs both curve checks passed, 723 scored no more than three standard deviations below fault-free seeds, though ne
Open source S3
arXiv:2611.08144v1 Announce Type: new Abstract: Robot reinforcement learning increasingly runs environments outside the trainer: in another process, on another machine, in another language. Many faults at this boundary raise no error. We make the boundary a wire protocol and ask what notices a fault in it. On Pendulum, HalfCheetah, Hopper and a 7-joint arm (Pusher), we injected up to 23 faults into what crosses the w
Open source S2
Source：arXiv Robotics — research abstracts · arxiv.org