トポロジー変化に対応する4Dガウシアンスプラッティングによる一人称視点動画の再構成
Topology-Aware 4D Gaussian Splatting for Egocentric Videos
切断や開閉など形状のトポロジーが変わる一人称視点動画を再構成するため、部位ごとの剛体変換とガウシアンの動的生成・消去を組み合わせたTopo-4DGSと、その評価用ベンチマークTopoEgoを提案した論文。
著者: Leonardo Vanni, Stefano Gasperini, Federico Tombari, Chiara Plizzari
分類: cs.CV
原文アブストラクト
Reconstructing dynamic scenes and objects from monocular egocentric video is crucial for embodied AI, yet existing 4D Gaussian Splatting (4DGS) approaches struggle with everyday interactions involving topological changes, such as cutting, assembly, opening, or closing. Since 4DGS methods rely on deforming a fixed representation, events that split, merge, or reveal surfaces cause severe visual artifacts. To address this challenge, we introduce TopoEgo, the first benchmark dedicated to topological changes in egocentric video (covering fission, fusion, open, and close), alongside Topo-4DGS, a topology-aware reconstruction method. Topo4DGS jointly models part-level motion and time-varying surface support by assigning independent rigid transformations to interacting parts and dynamically activating or deactivating Gaussians as surfaces appear, disappear, or separate. Extensive experiments demonstrate that Topo-4DGS significantly outperforms state-of-the-art 4DGS approaches across topological changes, especially on post-event frames. Code and data will be made publicly available.