When Does Test-Time Search Help World Action Models? An Empirical Audit of Candidate Selection and Simulator Integrity
When Does Test-Time Search Help World Action Models? An Empirical Audit of Candidate Selection and Simulator Integrity
This research paper audits the efficacy of test-time search in world-action models using Cosmos Policy and RoboCasa. Key findings include that a learned value selector performs 1.66 percentage points worse than random selection on clean data, token pruning yields at most a 1.15x end-to-end speedup falling short of the 1.5x target, and an adaptive-allocation experiment remains inconclusive under power gating. The authors also identify and fix silent rendering corruption that increases along compute-heavy planning paths while leaving direct-policy results unaffected, releasing associated detectors, fixes, and scripts.
The study empirically audits test-time search mechanisms on world-action models, finding that learned value selectors underperform random selection by 1.66 percentage points on clean data and identifying silent rendering corruption in compute-heavy planning paths.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
per-forward FLOPs reduction
≤47
View original evidence
se-first, pre-registered methodology. On clean data, a learned value selector is 1.66 pp worse than random; a hindsight oracle gains only 1.40 pp, with an episode-level interval spanning zero; and 82% of states show identical candidate outcomes, leaving little selection headroom. Token pruning cuts per-forward FLOPs by up to 47% but yields at most a measured 1.15$\times$ end-to-end speedup on the deployed path (1.25$
Open source S3end-to-end speedup on deployed path
1.15
View original evidence
se-first, pre-registered methodology. On clean data, a learned value selector is 1.66 pp worse than random; a hindsight oracle gains only 1.40 pp, with an episode-level interval spanning zero; and 82% of states show identical candidate outcomes, leaving little selection headroom. Token pruning cuts per-forward FLOPs by up to 47% but yields at most a measured 1.15$\times$ end-to-end speedup on the deployed path (1.25$
Open source S3end-to-end speedup on decode-excluded path
1.25
View original evidence
se-first, pre-registered methodology. On clean data, a learned value selector is 1.66 pp worse than random; a hindsight oracle gains only 1.40 pp, with an episode-level interval spanning zero; and 82% of states show identical candidate outcomes, leaving little selection headroom. Token pruning cuts per-forward FLOPs by up to 47% but yields at most a measured 1.15$\times$ end-to-end speedup on the deployed path (1.25$
Open source S3value selector performance gap vs random
1.66
View original evidence
se-first, pre-registered methodology. On clean data, a learned value selector is 1.66 pp worse than random; a hindsight oracle gains only 1.40 pp, with an episode-level interval spanning zero; and 82% of states show identical candidate outcomes, leaving little selection headroom. Token pruning cuts per-forward FLOPs by up to 47% but yields at most a measured 1.15$\times$ end-to-end speedup on the deployed path (1.25$
Open source S3hindsight oracle performance gain
1.4
View original evidence
se-first, pre-registered methodology. On clean data, a learned value selector is 1.66 pp worse than random; a hindsight oracle gains only 1.40 pp, with an episode-level interval spanning zero; and 82% of states show identical candidate outcomes, leaving little selection headroom. Token pruning cuts per-forward FLOPs by up to 47% but yields at most a measured 1.15$\times$ end-to-end speedup on the deployed path (1.25$
Open source S3
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
se-first, pre-registered methodology. On clean data, a learned value selector is 1.66 pp worse than random; a hindsight oracle gains only 1.40 pp, with an episode-level interval spanning zero; and 82% of states show identical candidate outcomes, leaving little selection headroom. Token pruning cuts per-forward FLOPs by up to 47% but yields at most a measured 1.15$\times$ end-to-end speedup on the deployed path (1.25$
Open source S3
Source：arXiv Robotics — research abstracts · arxiv.org