Asymmetric On-Policy Distillation Learns In-Context Exploration
Asymmetric On-Policy Distillation Learns In-Context Exploration
This work proposes ASTEROID, a student-teacher supervised learning framework that enables agents to learn efficient test-time exploration under partial observability. By distilling expert actions conditioned on the student's own on-policy history, the method resolves uncertainty and exploits revealed states without requiring privileged expert exploration or impractical reinforcement learning training costs. Across eleven diverse environments—including robotic manipulation, navigation, and discrete benchmarks—ASTEROID achieves approximately 2x higher returns under fixed exploration budgets and uses roughly 100x fewer interactions than RL baselines. The approach also transfers from simulation to vision-denied real-world robot manipulation, increasing task success by +40% over standard distillation.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
orcement learning. Across a diverse range of eleven different environments, including discrete exploration benchmarks, robotic manipulation, navigation, and Procgen environments, ASTEROID achieves ~2x higher returns under fixed exploration budgets, uses ~100x fewer interactions than RL baselines, and outperforms frontier models at exploration. Finally, our method transfers policies trained in simulation to vision-den
Open source S5
ied real-world robot manipulation, exhibiting efficient exploration and increasing success by +40% over standard distillation. Overall, it provides a simple yet surprisingly effective framework to learn efficient test-time exploration.
Open source S6
Asymmetric On-Policy Distillation Learns In-Context Exploration
Open source S1
arXiv:2610.30105v2 Announce Type: replace Abstract: Sequential decision-making under partial observability is difficult because agents must explore to infer hidden, task-relevant information before acting effectively. Learning such information-gathering behaviors tabula rasa via online reinforcement learning is sample-inefficient, whereas offline imitation averages expert actions across hidden states and fails to lea
Open source S2
rn exploration. In this work, we propose ASTEROID, a simple and effective student--teacher supervised learning framework that can learn test-time exploration. Although a privileged expert never needs to explore, we demonstrate that a history-conditioned student can learn to do so when trained with on-policy contexts: the student first collects on-policy context under partial observability, and then imitates the privi
Open source S3
leged expert conditioned on that context. By distilling expert actions directly conditioned on the student's on-policy history, the student learns to systematically resolve partial observability, i.e., search while uncertain and exploit once the hidden state is revealed. This enables efficient test-time exploration (similar to Bayesian posterior sampling) without incurring the impractical training-data costs of reinf
Open source S4
Source：arXiv Robotics — research abstracts · arxiv.org