A Straight Line Is Not a Camera Pose: Auditing Short-Clip Pose Benchmarks
A Straight Line Is Not a Camera Pose: Auditing Short-Clip Pose Benchmarks
This research abstract presents an audit of camera pose benchmarks, demonstrating that the standard Absolute Trajectory Error (ATE) metric is misleading because it favors constant-velocity predictions regardless of actual geometry. The authors show that for over 95% of model-clip pairs on the Sintel dataset, a simple straight-line prediction yields a lower ATE than complex models, yet performs worse geometrically. They introduce 'pose-anchored ATE,' a metric that incorporates orientation alignment to penalize such artifacts, offering a more robust evaluation method for SLAM and 3D reconstruction systems.
The study identifies systematic biases in the Absolute Trajectory Error (ATE) metric where constant-velocity assumptions artificially lower scores, proposing a rotation-aware 'pose-anchored ATE' to better reflect geometric accuracy.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
median SNR for Sintel
0.86
View original evidence
ity prediction onto the best-fitting line through the true path, so the line scores the path's regression residual. A residual signal-to-noise ratio (SNR) criterion predicts when moving a prediction towards its constant-velocity fit lowers ATE. It gives the outcome for 95% of 1056 model-clip pairs, 98% of 792 held-out pairs and 97% of 1034 ScanNet pairs, and it separates Sintel (median SNR 0.86) from TUM, ScanNet and
Open source S3cost per clip
0.17
View original evidence
ric mean: a lower ATE need not mean better geometry. Fitting the alignment's rotation to the predicted orientations gives pose-anchored ATE, a one-function change in ATE's units costing 0.17 ms per clip (ATE: 0.11 ms). It ranks every image-free reference below every model on Sintel, tracks the reprojection error under rotation and straightening, where ATE does not (Spearman 0.61 vs. 0.32), largely preserves the model
Open source S5ATE cost per clip
0.11
View original evidence
ric mean: a lower ATE need not mean better geometry. Fitting the alignment's rotation to the predicted orientations gives pose-anchored ATE, a one-function change in ATE's units costing 0.17 ms per clip (ATE: 0.11 ms). It ranks every image-free reference below every model on Sintel, tracks the reprojection error under rotation and straightening, where ATE does not (Spearman 0.61 vs. 0.32), largely preserves the model
Open source S5Spearman correlation for ATE under rotation
0.32
View original evidence
ric mean: a lower ATE need not mean better geometry. Fitting the alignment's rotation to the predicted orientations gives pose-anchored ATE, a one-function change in ATE's units costing 0.17 ms per clip (ATE: 0.11 ms). It ranks every image-free reference below every model on Sintel, tracks the reprojection error under rotation and straightening, where ATE does not (Spearman 0.61 vs. 0.32), largely preserves the model
Open source S5Spearman correlation for pose-anchored ATE under rotation
0.61
View original evidence
ric mean: a lower ATE need not mean better geometry. Fitting the alignment's rotation to the predicted orientations gives pose-anchored ATE, a one-function change in ATE's units costing 0.17 ms per clip (ATE: 0.11 ms). It ranks every image-free reference below every model on Sintel, tracks the reprojection error under rotation and straightening, where ATE does not (Spearman 0.61 vs. 0.32), largely preserves the model
Open source S5
- paper: not_reported
- code: Reported pending Open source S6
- weights: not_reported
- dataset: not_reported
- code: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
ity prediction onto the best-fitting line through the true path, so the line scores the path's regression residual. A residual signal-to-noise ratio (SNR) criterion predicts when moving a prediction towards its constant-velocity fit lowers ATE. It gives the outcome for 95% of 1056 model-clip pairs, 98% of 792 held-out pairs and 97% of 1034 ScanNet pairs, and it separates Sintel (median SNR 0.86) from TUM, ScanNet and
Open source S3
ric mean: a lower ATE need not mean better geometry. Fitting the alignment's rotation to the predicted orientations gives pose-anchored ATE, a one-function change in ATE's units costing 0.17 ms per clip (ATE: 0.11 ms). It ranks every image-free reference below every model on Sintel, tracks the reprojection error under rotation and straightening, where ATE does not (Spearman 0.61 vs. 0.32), largely preserves the model
Open source S5
s' order, and ties 10-frame translational relative pose error (RPE) and the discernible trajectory error (DTE) on our rotation and reference tests. Code, predictions and results will be released.
Open source S6
Source：arXiv Robotics — research abstracts · arxiv.org