$\pi$-StepNFT: Wider Space Needs Finer Steps in Online RL for Flow-based VLAs
$\pi$-StepNFT: Wider Space Needs Finer Steps in Online RL for Flow-based VLAs
This research paper presents $\pi$-StepNFT, a flow-native online reinforcement learning framework designed for vision-language-action (VLA) models. It addresses the challenge of multi-step denoising by coupling wider stochastic exploration with finer step-wise supervision without computing final-action likelihood. The method treats observed stochastic differential equation (SDE) transitions as supervision units to improve policy updates. Experimental results on the LIBERO benchmark show success rates increasing from 57.6% to 92.0% for $\pi_0$ and from 77.1% to 94.0% for $\pi_{0.5}$ starting from shared few-shot SFT initializations. Additionally, on the ManiSkill benchmark, it achieves the highest reported mean success among evaluated RL baselines across five test-only visual perturbations.
The paper introduces a flow-native online RL framework that addresses exploration-supervision mismatches in multi-step denoising for vision-language-action models, reporting specific success rate improvements on the LIBERO benchmark and top performance on ManiSkill under visual perturbations.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- code: not_reported
- weights: not_reported
- dataset: not_reported
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
ing the final-action likelihood. $\pi$-StepNFT treats observed SDE transitions as supervision units, using variance-normalized transition errors to rank mirrored positive and negative velocity branches and map rollout feedback to local velocity updates. On LIBERO, starting from shared few-shot SFT initializations, $\pi$-StepNFT raises average success from 57.6\% to 92.0\% for $\pi_0$ and from 77.1\% to 94.0\% for $\p
Open source S4
i_{0.5}$. On ManiSkill, it achieves the highest reported mean success among the evaluated RL baselines on five test-only visual perturbations. Our project homepage is available at https://wangst0181.github.io/pi-StepNFT/.
Open source S5
$\pi$-StepNFT: Wider Space Needs Finer Steps in Online RL for Flow-based VLAs
Open source S1
arXiv:2603.02083v3 Announce Type: replace Abstract: Flow-based vision-language-action (VLA) models have emerged as a powerful paradigm for embodied control, but multi-step denoising makes policy-gradient online reinforcement learning (RL) difficult because the final-action likelihood induced by the sampler is not directly tractable. We identify an exploration--supervision mismatch within the denoising trajectory, whe
Open source S2
Source：arXiv Robotics — research abstracts · arxiv.org