ZeroDex: Zero-Shot Long-Horizon Dexterous Manipulation via Multi-View 3D-Grounded VLM Reasoning

ZeroDex is a zero-shot framework for long-horizon dexterous manipulation that grounds language instructions into executable 3D task plans using calibrated multi-view RGB images. It extends vision-language model (VLM) grounding from 2D to 3D by fusing multi-view data, enabling contact points, placement targets, and tool-motion anchors. The system executes complex instructions by decomposing them into atomic tasks using a small library of primitives, and outperforms zero-shot visual-prompting and constraint-based baselines. It also demonstrates robustness to intermediate failures and generalization to novel objects and tool-use tasks.