スマホでリアルタイムに人体メッシュを復元するInstantHMR
InstantHMR : Real-Time Human Mesh Recovery on Your Phone
単一画像からスマホ上でリアルタイムに3D人体メッシュを復元する軽量モデル群を提案し、Galaxy S23のNPUで1人あたり5.5msという高速推論を実現した。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Mohamed Adjel, Kahina Chalabi, Vincent Bonnet
分類: cs.CV
原文アブストラクト
Real-time 3D human mesh recovery from a single image on mobile devices is a challenging problem with many applications, including robotics, biomechanics, AR/VR, and exergames. Current state-of-the-art models are either too slow to run efficiently on mobile devices or sacrifice accuracy for speed. We present InstantHMR, a family of open-source single-image regressors suited for mobile inference, based on the Momentum Human Rig (MHR) parametric model and trained exclusively on publicly available annotations from the SAM 3D Body dataset. Our approach achieves remarkable performance on a Samsung Galaxy S23 NPU, recovering a 6,890-vertex body mesh in 5.5 ms per person, and 0.98 ms on an NVIDIA RTX 4070 GPU. This speed surpasses all existing methods we evaluated. On mobile, it is 12x faster than the single mobile-compatible method that slightly outperforms ours in accuracy, while achieving competitive accuracy with 40.0 mm PA-MPJPE on 3DPW, 48.8 mm on EMDB and 41.7 mm on RICH. To ensure fair comparisons, we re-evaluate 11 state-of-the-art methods under a unified protocol aligned with the closest MHR-based competitors, including Meta's official MHR-to-SMPL conversion, and report their reproduction results. Additionally, we measure latency across eight different targets, ranging from desktop TensorRT to two phone NPUs. Code and models are available at https://github.com/Naturalpad-studio/InstantHMR.
関連論文
- 動的カメラ映像からの遮蔽に強いグローバル人体メッシュ復元人体メッシュ復元
- 人体メッシュ復元のための局所リカレントモデル学習人体メッシュ復元
- 誰もがユニーク：偏りのない人体メッシュ復元に向けて人体メッシュ復元
- 階層的運動学に基づく人体メッシュ復元人体メッシュ復元