遅延対応ガイダンス融合による非同期マルチモーダル拡散ポリシー合成
Asynchronous Multimodal Diffusion Policy Composition via Latency-Aware Guidance Fusion
異なるセンシングレートや推論遅延を持つ複数のモダリティを、各モダリティ固有の速度で動作させつつ、遅延を考慮してガイダンスを融合する非同期拡散ポリシーフレームワークLAG-Fusionを提案。接触を伴う操作タスクで視覚と力覚の非同期合成を実証し、同期融合や専用ベースラインより性能が向上。
著者: Zihao He, Hongjie Fang, Shirun Tang, Cewu Lu, Haoshu Fang
分類: cs.RO, cs.AI
原文アブストラクト
Diffusion policies have shown strong potential for robotic imitation learning, and recent extensions incorporate additional modalities to improve manipulation performance. However, these modalities often differ not only in information content but also in sensing rates and inference latencies. Existing multimodal diffusion policies typically rely on synchronous fusion or manually designed multi-frequency architectures, which either slow down high-frequency feedback or limit extensibility to new modality combinations. We propose LAG-Fusion, a latency-aware guidance fusion framework for asynchronous multimodal diffusion policy composition. LAG-Fusion allows modality-specific policies to operate at their native inference rates and contribute denoising guidance whenever available. To make asynchronous composition consistent, we derive a reference-frame rebasing rule for diffusion variables under relative action representations, enabling delayed guidance to be aligned before fusion. We instantiate LAG-Fusion in contact-rich manipulation by composing a low-frequency vision policy with a high-frequency force policy. Experiments under heterogeneous modality latencies show that LAG-Fusion improves policy responsiveness and task performance over synchronous fusion and specially designed force-aware baselines.