ViewStruct: 視点に基づく関節・接続構造の推論
ViewStruct: View-Grounded Articulation and Connectivity Inference
RGB画像と部分セグメンテーション・深度から、物体やシーンの剛体グループ・接続・関節パラメータを予測するフレームワークを提案。合成データのみで学習し、未知カテゴリや実世界動画へ転移、ロボット動作生成に応用可能。
著者: Wenjing Bian, Ranran Huang, Andreas Geiger
分類: cs.CV
原文アブストラクト
Interacting with unfamiliar objects requires understanding how their parts connect and move. We introduce ViewStruct, a framework for jointly predicting connectivity and articulation in objects and scenes. Given an RGB image with aligned part segmentation and depth, optionally supplemented by images or video, ViewStruct predicts rigid-body groupings, attachments, and joint parameters. Its view-grounded representation links predictions to observed regions and expresses joint geometry in a gravity-aligned, view-relative frame, without requiring category-specific canonical orientations. A pretrained vision--language model is adapted to predict structure in seconds, without requiring observed articulation motion or per-instance optimisation. Experiments demonstrate improved structural prediction over evaluated baselines and transfer to unseen categories, asset sources, and real-world human-interaction videos, despite task-specific training exclusively on synthetic data. Simulated manipulation experiments further demonstrate the utility of predicted joints for generating robot motion trajectories.