PAIR: Proprioception-Aligned Image-Ray Representations for Viewpoint-Robust Manipulation
PAIR: Proprioception-Aligned Image-Ray Representations for Viewpoint-Robust Manipulation
This research paper presents PAIR, a framework that aligns joint image-ray representations using robot proprioception to address viewpoint ambiguity in visuomotor policies. By encouraging observations with similar robot states to remain close in representation space across viewpoints, PAIR provides structured grounding for learning viewpoint-robust features. The method is adaptable for training Diffusion Policy or post-training vision-language-action models and demonstrates significant performance improvements over behavior cloning baselines without ray conditioning in both simulated and real-world manipulation benchmarks with camera variation.
The paper introduces PAIR, a contrastive learning framework that aligns image-ray representations using robot proprioception to improve viewpoint robustness in manipulation tasks, reporting performance gains over behavior cloning baselines in simulated and real-world benchmarks.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- paper: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
benchmarks with camera variation, PAIR consistently outperforms the behavior cloning baseline without ray conditioning by 12.7% and 26.7% points, respectively, demonstrating stronger generalization to unseen viewpoints. Project website: PAIRrobot.github.io.
Open source S6
PAIR: Proprioception-Aligned Image-Ray Representations for Viewpoint-Robust Manipulation
Open source S1
arXiv:2611.11798v1 Announce Type: new Abstract: While imitation learning has made substantial progress toward generalizable robot manipulation through scaling, viewpoint robustness remains a major bottleneck. Visuomotor policies rely heavily on visual inputs, yet images provide an inherently ambiguous signal under viewpoint change: similar scenes may appear markedly different across viewpoints, while distinct states
Open source S2
may share certain visual overlaps. A policy must therefore infer not only the task-relevant state, but also how it is visually projected under a particular camera configuration. This entanglement of viewpoint-dependent appearance and task-relevant state can make learning inefficient and limit generalization to unseen viewpoints. Recent work has begun to address this challenge through camera-ray conditioning and repre
Open source S3
sentation learning using geometric cues to induce invariance. However, providing camera information as additional context does not by itself ground visual representations in how the task scene is visually projected, while geometric alignment without camera context can introduce spurious correspondences by ignoring viewpoint-dependent projection. We propose PAIR, a contrastive learning framework that aligns joint imag
Open source S4
e-ray representations using robot proprioception. PAIR encourages observations with similar robot states to remain close in representation space across viewpoints, providing a structured grounding signal for learning viewpoint-robust features. PAIR is a highly adaptable framework: it can be used to train Diffusion Policy (DP) or post-train vision-language-action (VLA) models. In simulated and real-world manipulation
Open source S5
Source：arXiv Robotics — research abstracts · arxiv.org