物理ベースのロボット操作のための遮蔽に頑健な多物体分離
Occlusion-Robust Multi-Object Decoupling for Physics-Based Robotic Interaction
遮蔽や視点不足による物体の結合を、スパースビューからの3D再構成問題として捉え、マスク不要で複数物体を分離・再構成する手法を提案。3Dガウシアンスプラッティングと拡散モデルを組み合わせ、シミュレーション可能な物体を生成する。
著者: Xin Dong, Lihan Zhang, Tianru Dai, Wenfeng Deng, Yansong Tang
分類: cs.CV
原文アブストラクト
We propose a mask-free method for lossless multi-object 3D reconstruction from sparse and occluded real-world views, enabling physically plausible robotic interaction via Material Point Method (MPM) simulation. Our key insight is that object coupling stems from occlusion and limited viewpoints, which we address by formulating multi-object decoupling as a sparse-view reconstruction problem. Using 3D Gaussian Splatting as base representation, we first obtain coarse instance partitions with a SAM2-trained segmentation field. Rather than relying on masks, we reconstruct fragmented geometries by leveraging a joint Score Distillation Sampling (SDS) process, which integrates reference-view supervision with novel-view synthesis guided by 2D and 3D diffusion priors to enforce both texture fidelity and 3D consistency. Furthermore, we incorporate geometry-aware priors such as intra-object and inter-object similarity to regularize geometric reasoning. Experimental results demonstrate that our method produces complete, simulation-ready 3D objects without requiring manual masks, enabling realistic dynamic interactions on both synthetic, robotic and real-world datasets.