DreamFAR: 多様な物体とのリアルタイム長期的インタラクション
DreamFAR: Real-Time and Long-Horizon Interaction with Diverse Objects
ロボット操作のためのワールドモデルDreamFARを提案。大規模なロボットテレオペレーションデータのみで学習し、接触や因果関係の理解に優れ、商用GPUで10FPS動作、長期予測で既存手法を大きく上回る。
著者: Yiming Dou, Botao He, Antonio Loquercio, Andrew Owens, Jitendra Malik, Sumedh Anand Sontakke, Ruoshi Liu
分類: cs.RO
原文アブストラクト
World models have shown great potential in serving as a simulator for robot manipulation. However, existing models are either fit to a single environment or lack ability to simulate long-horizon contact-rich events. We present DreamFAR, a world model whose dynamics model is trained from scratch on a large robot-native teleoperation dataset, with no internet-video pretraining. We present an architecture and learning recipe designed for training on robot-native data. DreamFAR is significantly smaller than the high-performance video generation models, yet shows better understanding of contact and causality, and runs 10 FPS on a commercial GPU. Experimentally, DreamFAR significantly outperforms existing models in predicting the long-horizon future. We further demonstrate various applications where DreamFAR can be used, including policy evaluation and mixed-data co-training. Project page: https://www.yimingdou.com/DreamFAR/
