Explicitly capturing demonstrator multi-modality for distillation into diffusion policies
Explicitly capturing demonstrator multi-modality for distillation into diffusion policies
This paper addresses the need for exploration in distilled diffusion policies by adapting vine sampling to behavior cloning. The authors query demonstrators multiple times at identical observation states to resolve ambiguity between unimodal memorization and true multi-modal distribution learning. Experiments on synthetic tasks and a 2D truck unloading scenario show that increasing the number of actions per observation ($k$) yields policies that better match demonstrator distributions, improving best-of-$N$ and expert iteration performance.
The authors propose adapting vine sampling to behavior cloning data collection to explicitly capture multi-modal demonstrator actions, finding that querying the demonstrator multiple times at the same state improves policy diversity and downstream performance on synthetic and truck unloading tasks.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
Explicitly capturing demonstrator multi-modality for distillation into diffusion policies
Open source S1
arXiv:2611.10949v1 Announce Type: new Abstract: The diversity of a distilled diffusion policy's outputs matters beyond how it impacts the policy's success rate. In particular, downstream applications like test-time search and policy improvement require exploration. Thus, we want to distill multi-modal demonstrators into diffusion policies which reproduce their conditional action distributions. Standard data collectio
Open source S2
n for behavior cloning applies a sliding window to demonstration episodes, which pairs unique observation histories with actions. These datasets can be fit by a policy that is either multi-modal, or unimodal with high Lipschitz constant to capture a diversity of actions seen at nearby observation samples. Prior work finds that diffusion policies trained on sparse data learn the latter, memorizing one action per obser
Open source S3
vation. We adapt vine sampling to behavior cloning data collection to resolve this ambiguity: at some states the demonstrator visits, query it $k$ times from exactly the same observation. On a synthetic task, we study how the number of actions per observation, $k$, affects the learned policy, and find that larger $k$ yields a policy that better matches the demonstrator's conditional distribution, including its multi-
Open source S4
modality and diversity. On a 2D truck unloading task, vine sampling likewise leads to a policy closer to the demonstrator and substantially improves both best-of-$N$ and expert iteration performance.
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org