すべてのカメラは大人になる:インタラクティブ動画ワールドモデルにおけるカメラ高さ回帰
Every Camera Grows Up: Camera-Height Regression in Interactive Video World Models
インタラクティブ動画ワールドモデルがカメラ移動時に初期のカメラ高さを保てず、人間の目の高さへと引き寄せられる「カメラ高さ回帰」現象を6モデルで実証し、EyeHeightベンチマークで定量化した論文。
著者: Sheng Lu, Guangpin Tao, Tong Lu, Ying Tai
分類: cs.CV
原文アブストラクト
Interactive video world models serve as simulators for embodied agents: from a starting image and an action trajectory, they generate the views the agent would see. The starting image fixes how high the camera is, for example 0.15 m for a floor robot or 2.2 m for an overhead camera, and a faithful simulator must keep this height as the camera moves. We show that six recent open-weight models fail to do so: as motion replaces the starting view with generated content, they move the camera toward human eye level, a failure we call camera-height regression. Our benchmark, EyeHeight, pairs six rendered elevations (0.15 m to 2.2 m) and four calibrated real-world rig heights with scale-free multi-view height estimation. All six models lose their initial height during a $180^\circ$ turn-and-walk rollout: initialized at 0.15 m, only 0-20% of clips end within $1.3\times$ of the starting height as inferred viewpoints converge to human heights of 1.1 m to 1.5 m, whereas ground-truth renders remain within a few percent. In Matrix-Game models, homography-tracked floor motion stays consistent with constant elevation during rotation even as newly outpainted room geometry yields a higher reconstructed viewpoint. Regression grows with rotation angle, begins while the start frame remains in context, and largely reverses in SANA-WM when a $360^\circ$ turn restores the initial view. We trace this pull to human-centric video priors in which floor-level views are nearly absent: in SANA-WM, textual height cues and dense floor-disparity maps do not prevent it, whereas classifier-free text guidance during the first half of denoising strengthens it. Combining unguided sampling with height-balanced fine-tuning recovers more retention but costs 5 MUSIQ points, a loss fixed during those same early steps. Viewpoint drift alone shifts the paths predicted by visual navigation policies.
