Fast and Accurate Hand Estimation in Pixels, Interaction and World Space
Fast and Accurate Hand Estimation in Pixels, Interaction and World Space
This research presents FAH (Fast and Accurate Hand Estimation), a model family built on SAM 3D Body's hand branch. It directly predicts 3D joints that align better with 2D predictions and fits MANO or MHR-Hand meshes to reduce error. The models optionally estimate anatomical parts and contact without object geometry, use causal refinement to reduce acceleration error, and place hands in world space via camera estimation. Experiments show improvements in image-space alignment, hand geometry, contact estimation, and world-space tracking over baselines. The default ViT-L model runs at 65 fps with mesh fitting on an RTX 4070 Ti SUPER (or 49 fps with parts and contact enabled), while the smallest ViT-S model reaches 125 fps without mesh fitting, parts or contact. Code is available at https://github.com/noahcao/FAH.
The paper introduces FAH, a family of models fine-tuned from SAM 3D Body that directly predicts 3D joints and fits MANO or MHR-Hand meshes, achieving improved image-space alignment and world-space tracking compared to baselines while running at up to 125 fps on an RTX 4070 Ti SUPER without mesh fitting.
Source: arXiv Robotics — research abstracts · Read original article ↗
Loading article text…
What the source reports
Publisher-reported claims, with original evidence. These results have not been independently verified by RoboSignal.
Reported numbers
frames per second
65
View original evidence
ing over the evaluated baselines. The default ViT-L model runs at 65 fps with mesh fitting on an RTX 4070 Ti SUPER, or 49 fps with parts and contact enabled. The smallest ViT-S model reaches 125 fps without mesh fitting, parts or contact. Code: https://github.com/noahcao/FAH.
Open source S4frames per second
49
View original evidence
ing over the evaluated baselines. The default ViT-L model runs at 65 fps with mesh fitting on an RTX 4070 Ti SUPER, or 49 fps with parts and contact enabled. The smallest ViT-S model reaches 125 fps without mesh fitting, parts or contact. Code: https://github.com/noahcao/FAH.
Open source S4frames per second
125
View original evidence
ing over the evaluated baselines. The default ViT-L model runs at 65 fps with mesh fitting on an RTX 4070 Ti SUPER, or 49 fps with parts and contact enabled. The smallest ViT-S model reaches 125 fps without mesh fitting, parts or contact. Code: https://github.com/noahcao/FAH.
Open source S4
- code: Available Artifact link Open source S4
Source excerpts and review record
Automatically extracted; no manual editorial approval recorded.
nd feature distillation. The models predict corresponding 2D and 3D joints and surface points, fit MANO or MHR-Hand, and optionally estimate anatomical parts and contact without object geometry at inference. Causal refinement reduces acceleration error, and camera estimation places the hands in world space. Experiments show improvements in image-space alignment, hand geometry, contact estimation and world-space track
Open source S3
ing over the evaluated baselines. The default ViT-L model runs at 65 fps with mesh fitting on an RTX 4070 Ti SUPER, or 49 fps with parts and contact enabled. The smallest ViT-S model reaches 125 fps without mesh fitting, parts or contact. Code: https://github.com/noahcao/FAH.
Open source S4
Source：arXiv Robotics — research abstracts · arxiv.org