Simulation-aided Reinforcement Learning with Control Variates
Overview
This paper introduces a conservative simulation-aided reinforcement learning method based on control variates. Unlike standard approaches that directly mix simulated and real data—which can degrade performance due to simulator inaccuracies—this method uses simulated data solely to reduce variance in the policy evaluation process, a fundamental building block of RL algorithms. The approach is integrated into the actor-critic framework for policy improvement and extended to offline RL scenarios. Experiments on multiple MuJoCo benchmark tasks show that this method outperforms standard simulation mixing approaches, highlighting control variates as an effective tool for improving simulation-aided reinforcement learning.
Generated from attributed reports · Updated 3 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Environment: simulation
“arXiv:2611.12643v1 Announce Type: cross Abstract: Reinforcement Learning (RL) algorithms often rely on data collected from a target environment, which can be expensive and noisy. Although simulators can generate large amounts of additional data, standard simulation mixing approaches, which directly combine simulated and real experience, can degrade performance due to simulator inaccuracies, especially when they bias”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalSimulation-aided Reinforcement Learning with Control Variates
This paper introduces a conservative simulation-aided reinforcement learning method based on control variates. Unlike standard approaches that directly mix simulated and real data—which can degrade performance due to simulator inaccuracies—this method uses simulated data solely to reduce variance in the policy evaluation process, a fundamental building block of RL algorithms. The approach is integrated into the actor-critic framework for policy improvement and extended to offline RL scenarios. Experiments on multiple MuJoCo benchmark tasks show that this method outperforms standard simulation mixing approaches, highlighting control variates as an effective tool for improving simulation-aided reinforcement learning.
Event coverage history
There is not enough continuous observation data to show a trend.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).