Off-Policy RL with Online Inference Tuning: My Solution to the CoRL 2026 RoboEval Challenge
Off-Policy RL with Online Inference Tuning: My Solution to the CoRL 2026 RoboEval Challenge
This arXiv preprint details a solution for the CoRL 2026 RoboEval challenge, which involves eight bimanual tasks. The approach utilizes off-policy reinforcement learning on a policy that serves as its own value function, trained with AWR and RECAP algorithms. It incorporates online inference tuning via Thompson sampling to adjust approximately 20 inference parameters per rollout. The method also integrates generated demonstrations reaching full rewards and recovery episodes from policy failures. The submitted policy achieved a score of 0.990 RDS on held-out seeds and 0.987 on official seeds, compared to an out-of-the-box performance of 0.85 RDS.
The author presents a solution to the CoRL 2026 RoboEval challenge that combines off-policy reinforcement learning with online inference tuning, achieving reported scores of 0.990 RDS on held-out seeds and 0.987 on official seeds.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
RDS score
0.85
View original evidence
}$-based policy that is also its own value function. It is trained with AWR and RECAP on advantages of the exact official score, in a loop where every rollout is at the same time a Thompson-sampling trial of about 20 inference parameters. Out of the box, it took the policy to 0.85 RDS on held-out seeds in two days (and it was still very far from convergence). The competition-specific part adds data that local explora
Open source S3RDS score
0.99
View original evidence
tion cannot find: generated demonstrations that reach the full reward, and recovery episodes that start from the policy's own failures. The submitted policy reaches 0.990 RDS on held-out seeds and 0.987 on the official seeds.
Open source S4RDS score
0.987
View original evidence
tion cannot find: generated demonstrations that reach the full reward, and recovery episodes that start from the policy's own failures. The submitted policy reaches 0.990 RDS on held-out seeds and 0.987 on the official seeds.
Open source S4inference parameters
≈20
View original evidence
}$-based policy that is also its own value function. It is trained with AWR and RECAP on advantages of the exact official score, in a loop where every rollout is at the same time a Thompson-sampling trial of about 20 inference parameters. Out of the box, it took the policy to 0.85 RDS on held-out seeds in two days (and it was still very far from convergence). The competition-specific part adds data that local explora
Open source S3tasks
8
Reported trials
View original evidence
arXiv:2611.11464v1 Announce Type: new Abstract: The CoRL 2026 RoboEval challenge has eight bimanual tasks, and it ranks a policy not only on whether it succeeds but on how: every stage of the task, done safely, efficiently and with both arms in coordination, scored against human demonstrations. My solution has two parts. The generic part, carried over from my LeHome challenge solution, is off-policy RL on a $\pi_{0.5
Open source S2
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
}$-based policy that is also its own value function. It is trained with AWR and RECAP on advantages of the exact official score, in a loop where every rollout is at the same time a Thompson-sampling trial of about 20 inference parameters. Out of the box, it took the policy to 0.85 RDS on held-out seeds in two days (and it was still very far from convergence). The competition-specific part adds data that local explora
Open source S3
tion cannot find: generated demonstrations that reach the full reward, and recovery episodes that start from the policy's own failures. The submitted policy reaches 0.990 RDS on held-out seeds and 0.987 on the official seeds.
Open source S4
arXiv:2611.11464v1 Announce Type: new Abstract: The CoRL 2026 RoboEval challenge has eight bimanual tasks, and it ranks a policy not only on whether it succeeds but on how: every stage of the task, done safely, efficiently and with both arms in coordination, scored against human demonstrations. My solution has two parts. The generic part, carried over from my LeHome challenge solution, is off-policy RL on a $\pi_{0.5
Open source S2
Source：arXiv Robotics — research abstracts · arxiv.org