剛体オブジェクトのための行動条件付き・オブジェクト中心ガウシアンスプラッティング世界モデルの学習
Learning Action-Conditional and Object-Centric Gaussian Splatting World Models for Rigid Objects
オブジェクト中心のガウシアン表現と時空間トランスフォーマーを用いて、剛体オブジェクトの将来の動きを行動履歴から予測する世界モデルを提案し、非把持操作のモデル予測制御で評価した。
著者: Jens U. Kreber, Lukas Mack, Joerg Stueckler
分類: cs.RO, cs.CV, cs.LG
原文アブストラクト
World models enable intelligent agents to predict the consequences of their actions on the environment. In this paper, we propose Multi Rigid Object Gaussian World Model (MRO-GWM), a novel model that learns action-conditional dynamics of rigid objects in 3D. By representing the scene by object-centric Gaussians, we can represent arbitrary object shapes and multi-object scenes. We develop a novel spatio-temporal transformer architecture that predicts future rigid body motion from a history of object Gaussians and future actions. Objects are represented by their Gaussians in a canonical frame, which allows for describing object motion as rigid body transformation. Our model is trained on reconstructions from multiple viewpoints, which requires the model to handle partial observations of objects due to occlusions. We analyze prediction performance of our approach on synthetic datasets composed of typical household objects with multi-object dynamics and interactions by a robot end effector. We also evaluate our model in model-predictive control for non-prehensile manipulation in simulation.