PhysMoE: Fast Spatial Grounding with Single-Token Coordinate Representation in Modality-Decoupled VLMs
Overview
This paper presents PhysMoE, a new architecture for Embodied Vision-Language Models (VLMs) designed to improve coordinate prediction in robotic grounding tasks. The authors address limitations in existing models where discrete tokenization disrupts spatial continuity and standard point regression struggles with multi-contact affordances. PhysMoE shares lower layers for multimodal context while routing upper layers to specialized experts for spatial and linguistic processing, encodes coordinates as single tokens using learned positional embeddings, and employs differentiable Signed Distance Field (SDF) supervision. The model achieves significant speedups and accuracy gains across multiple benchmarks while preserving multimodal understanding.
Generated from attributed reports · Updated 2 hours ago
Event evidence and corrections
0 attributed source owners. Ownership does not establish independent confirmation. Quantities are reported separately and are never added together.
Reported quantity · throughput speedup: 8.31 other · Basis not reported
“atial priors to natively align coordinates with visual features and ease training; and (3) differentiable Signed Distance Field (SDF) supervision on pointing, accommodating affordances and irregular geometries without costly mask decoders. Compared to the base model, PhysMoE achieves a 15.7% average gain and an 8.31x throughput speedup across 9 coordinate-based grounding benchmarks (trajectory, affordance, and locali”
Exact source · revision 1Source owner not reported
Reported quantity · number of coordinate-based grounding benchmarks: 9 other · Basis not reported
“atial priors to natively align coordinates with visual features and ease training; and (3) differentiable Signed Distance Field (SDF) supervision on pointing, accommodating affordances and irregular geometries without costly mask decoders. Compared to the base model, PhysMoE achieves a 15.7% average gain and an 8.31x throughput speedup across 9 coordinate-based grounding benchmarks (trajectory, affordance, and locali”
Exact source · revision 1Source owner not reported
Reported quantity · number of text-based embodied benchmarks: 11 other · Basis not reported
“zation), while preserving multimodal understanding and reasoning across 11 text-based embodied benchmarks.”
Exact source · revision 1Source owner not reported
Report timeline
Follow attributed reports and material updates.
- arXiv Robotics — research abstractsSignalPhysMoE: Fast Spatial Grounding with Single-Token Coordinate Representation in Modality-Decoupled VLMs
This paper presents PhysMoE, a new architecture for Embodied Vision-Language Models (VLMs) designed to improve coordinate prediction in robotic grounding tasks. The authors address limitations in existing models where discrete tokenization disrupts spatial continuity and standard point regression struggles with multi-contact affordances. PhysMoE shares lower layers for multimodal context while routing upper layers to specialized experts for spatial and linguistic processing, encodes coordinates as single tokens using learned positional embeddings, and employs differentiable Signed Distance Field (SDF) supervision. The model achieves significant speedups and accuracy gains across multiple benchmarks while preserving multimodal understanding.
Event coverage history
There is not enough continuous observation data to show a trend.
Timezone · UTC
Article dates follow your selected timezone. Briefing editions use Hong Kong time (UTC+8).