EgoLabel: Zero-Shot Egocentric Auto-Labeling with Cross-Modal Consistency
EgoLabel: Zero-Shot Egocentric Auto-Labeling with Cross-Modal Consistency
This research introduces EgoLabel, a zero-shot pipeline for egocentric auto-labeling that combines off-the-shelf models to ensure cross-modal consistency in depth, camera pose, object, and human-pose estimates without requiring target-domain labeled training.
The study presents EgoLabel, a zero-shot pipeline for egocentric auto-labeling that integrates off-the-shelf models to ensure cross-modal consistency in depth, camera pose, object, and human-pose estimates without target-domain labeled training.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
- Data origin
- Human egocentric dataOpen source S2
Reported numbers
pooled median hand depth residual
84.6
View original evidence
an hand--depth residuals by $23\%$ relative to a single global scale, yielding a pooled median of $84.6\,\mathrm{mm}$ against a $20.5\,\mathrm{mm}$ ground-truth reference. Hand PA-MPJPE is $14.53\pm1.29\,\mathrm{mm}$, whereas absolute MPJPE is $119.1\pm31.3\,\mathrm{mm}$, revealing a substantial gap between aligned and metric accuracy. On three RGB-only sequences, camera ATE is $1.40\pm0.29\,\mathrm{cm}$ after $\math
Open source S4ground-truth reference hand depth residual
20.5
View original evidence
an hand--depth residuals by $23\%$ relative to a single global scale, yielding a pooled median of $84.6\,\mathrm{mm}$ against a $20.5\,\mathrm{mm}$ ground-truth reference. Hand PA-MPJPE is $14.53\pm1.29\,\mathrm{mm}$, whereas absolute MPJPE is $119.1\pm31.3\,\mathrm{mm}$, revealing a substantial gap between aligned and metric accuracy. On three RGB-only sequences, camera ATE is $1.40\pm0.29\,\mathrm{cm}$ after $\math
Open source S4hand PA-MPJPE
14.53
View original evidence
an hand--depth residuals by $23\%$ relative to a single global scale, yielding a pooled median of $84.6\,\mathrm{mm}$ against a $20.5\,\mathrm{mm}$ ground-truth reference. Hand PA-MPJPE is $14.53\pm1.29\,\mathrm{mm}$, whereas absolute MPJPE is $119.1\pm31.3\,\mathrm{mm}$, revealing a substantial gap between aligned and metric accuracy. On three RGB-only sequences, camera ATE is $1.40\pm0.29\,\mathrm{cm}$ after $\math
Open source S4absolute MPJPE
119.1
View original evidence
an hand--depth residuals by $23\%$ relative to a single global scale, yielding a pooled median of $84.6\,\mathrm{mm}$ against a $20.5\,\mathrm{mm}$ ground-truth reference. Hand PA-MPJPE is $14.53\pm1.29\,\mathrm{mm}$, whereas absolute MPJPE is $119.1\pm31.3\,\mathrm{mm}$, revealing a substantial gap between aligned and metric accuracy. On three RGB-only sequences, camera ATE is $1.40\pm0.29\,\mathrm{cm}$ after $\math
Open source S4camera ATE
1.4
View original evidence
an hand--depth residuals by $23\%$ relative to a single global scale, yielding a pooled median of $84.6\,\mathrm{mm}$ against a $20.5\,\mathrm{mm}$ ground-truth reference. Hand PA-MPJPE is $14.53\pm1.29\,\mathrm{mm}$, whereas absolute MPJPE is $119.1\pm31.3\,\mathrm{mm}$, revealing a substantial gap between aligned and metric accuracy. On three RGB-only sequences, camera ATE is $1.40\pm0.29\,\mathrm{cm}$ after $\math
Open source S4number of sequences evaluated
20
Reported trials
View original evidence
labeled training and provides explicit cross-modal registration and consistency diagnostics. The pipeline integrates MonST3R for geometry, LocateAnything with GMC-BoT-SORT and SAM~2 for objects, and V-HPOT for hands; FisheyeViT provides a separate whole-body compatibility demonstration. Our main hand evaluation uses H2O sensor depth. Across 20 sequences, per-hand depth anchoring reduces the mean of per-sequence medi
Open source S3
- paper: not_reported
- code: not_reported
- weights: not_reported
- dataset: not_reported
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
arXiv:2611.12149v1 Announce Type: cross Abstract: Egocentric auto-labeling requires depth, camera pose, object, and human-pose estimates to agree across independently developed models. Differences in units, coordinate conventions, and scale can make individually plausible predictions geometrically inconsistent. We present \textbf{EgoLabel}, a zero-shot pipeline that combines off-the-shelf models without target-domain
Open source S2
an hand--depth residuals by $23\%$ relative to a single global scale, yielding a pooled median of $84.6\,\mathrm{mm}$ against a $20.5\,\mathrm{mm}$ ground-truth reference. Hand PA-MPJPE is $14.53\pm1.29\,\mathrm{mm}$, whereas absolute MPJPE is $119.1\pm31.3\,\mathrm{mm}$, revealing a substantial gap between aligned and metric accuracy. On three RGB-only sequences, camera ATE is $1.40\pm0.29\,\mathrm{cm}$ after $\math
Open source S4
labeled training and provides explicit cross-modal registration and consistency diagnostics. The pipeline integrates MonST3R for geometry, LocateAnything with GMC-BoT-SORT and SAM~2 for objects, and V-HPOT for hands; FisheyeViT provides a separate whole-body compatibility demonstration. Our main hand evaluation uses H2O sensor depth. Across 20 sequences, per-hand depth anchoring reduces the mean of per-sequence medi
Open source S3
Implications for data suppliers
RoboSignal interpretation and collection questions, not statements of buyer demand.
- Check whether the intended buyer needs human observations, hand or object annotations, or aligned robot actions. Human video alone does not provide robot commands.
- Compare the reported units and scope before using these quantities in a budget. Recording hours, sensor-hours and trajectories are different measures.
- Validate demand and acceptance criteria with a buyer before scaling. Publication, popularity and a research result do not establish a purchase commitment.
Source：arXiv Robotics — research abstracts · arxiv.org