時間的ギャップに注意：運転における時間的理解のための動画モデル評価
Mind the Temporal Gap: Evaluating Video Models for Temporal Understanding in Driving
運転動画での行動識別タスクを通じて、動画言語モデルが時間的ダイナミクスを頑健に捉えているかを評価し、プロンプト手法の不安定さとLoRA微調整の有効性を示した。
著者: Shiva Sreeram, Erfan Aasi, Tsun-Hsuan Wang, Alaa Maalouf, Daniela Rus
分類: cs.CV, cs.RO
原文アブストラクト
Modern video-language models are increasingly treated as general-purpose video reasoners, yet it remains unclear whether they robustly capture temporal dynamics in safety-critical settings such as driving. We study this question through controlled ego-motion tasks that require identifying actions from driving videos. Across video-language models, we find that standard prompting, visual prompting, few-shot prompting, and chain-of-thought prompting remain unreliable and highly task- and model-dependent, with persistent class biases, although newer models show substantial task-specific few-shot gains. In contrast, lightweight LoRA finetuning and simple probes trained on intermediate visual features recover strong motion-discriminative performance, indicating that motion-relevant cues are often present inside the models but are not reliably used by the final video-to-language pathway. These results suggest that, in driving, current video-language models remain unreliable temporal reasoners and often behave more like frame-level, bias-driven recognizers than dependable temporal world models.