VLM評価arXiv:2610.34151
実行ログがロボット進捗を評価するVLM審判を揺るがす
Execution Logs Sway VLM Judges of Robot Progress
ロボット動画の進捗を判定する視覚言語モデルが、画面横のテキスト（実行ログ）に影響され判定を誤ることを示し、警告文でその効果が軽減されることを明らかにした。
著者: Weiyan Zhao, Sunny Yang
分類: cs.LG, cs.RO
原文アブストラクト
Vision-language models now grade robot videos, and text placed beside the frames can change their verdicts. On clips where the robot is losing progress, we add one log line saying the robot is making progress. Across Gemini, Qwen and GPT judges, this flips correct verdicts to progress 10 to 35 percentage points more often than a neutral line does. A one-line warning that the log may be wrong removes most of this effect. Selected fake "task complete" claims flip half of Qwen's correct verdicts, against a tenth for unrelated log lines. They have no detectable effect on Gemini.