Q-OIL: Optimistic Exploration in Learning from Human Interventions
Q-OIL: Optimistic Exploration in Learning from Human Interventions
This research addresses the challenge of robot learning with human feedback by analyzing real-world intervention data, which reveals that humans often intervene when progress stalls and may provide suboptimal corrections. The authors propose Q-OIL, a method that uses a progress-based model of human behavior to guide exploration in reinforcement learning. By adding optimistic bonuses to transitions involving human corrections and employing decoupled critics for stability, Q-OIL achieves 10-50% higher success rates compared to baselines in both simulation and on real robots, demonstrating improved performance on pretrained vision-language-action models for bimanual manipulation tasks.
The paper introduces Q-OIL, a reinforcement learning method that incorporates optimistic bonuses guided by realistic human intervention patterns to improve robot adaptation in simulation and on real hardware.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- Environment
- SimulationOpen source S2
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
arXiv:2611.08576v1 Announce Type: new Abstract: As robots are deployed in the real world, they will inevitably make mistakes. Human interventions provide valuable feedback for improving their behavior, enabling them to adapt online to new scenarios. However, studying human-in-the-loop robot learning is difficult: real-world experiments are slow and expensive, so most algorithm development relies on simulated human mo
Open source S2
n states. As a result, Q-OIL learns more efficiently than competitive baselines, achieving 10-50% higher success in simulation and on real robots, and improving pretrained VLAs on two bimanual manipulation tasks. As we ground both evaluation and algorithms in realistic human behavior, this work provides a practical path toward scalable robot learning from interventions.
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org