RadarTwin: 特定環境向けミリ波レーダーシミュレーションと屋内認識のための学習
RadarTwin: Scene-Specific mmWave Radar Simulation and Learning for Mobile Indoor Perception
実データ収集前に、対象空間の3D再構成からレーダー関連の表面材質を推定し、物理ベースのレイトレーシングでFMCWレーダー測定を合成するフレームワークを提案。シミュレーションと実データの特徴が一致し、シミュレーションのみで学習した表現が実物体を偶然の2.5倍の精度で認識、少数のラベルで95.3%に向上することを示した。
著者: Emily Bejerano, Federico Tondolo, Devang Gupta, Aaron Mano Cherian, Taeyoo Kim, Ayaan Qayyum, Xiaofan Yu, Xiaofan Jiang
分類: cs.CV, cs.LG
原文アブストラクト
Millimeter-wave (mmWave) radar perception is limited by data scarcity: models trained on existing radar datasets fail to generalize to new objects, environments, and sensing trajectories. We present RadarTwin, a framework for generating deployment-specific radar training data before real data collection. Given a 3D reconstruction of a target space (phone LiDAR, robot-mounted sensing, or RGB-to-3D), RadarTwin uses a vision-language model to infer radar-relevant surface materials and a physics-based ray tracer to synthesize raw frequency-modulated continuous-wave (FMCW) radar measurements with multi-bounce propagation. To study what transfers from simulation to reality, we collect a paired real-simulated dataset spanning household objects, material classes, distances, rotations, translations, and mobile sensing trajectories. We show that simulated and real radar share the same object-discriminative shape and material features, and that modeling the environment's multipath is essential to matching real measurements. A representation trained on simulation alone recognizes real objects at 2.5 times chance with no real radar labels, and a few labeled examples raise this to 95.3% on a 12-way recognition task. RadarTwin enables training radar perception for a new space before any real radar data is collected there.