エージェント視点で基盤モデルの道徳的意思決定を評価するマルチモーダルベンチマークABHBench
ABHBench: Evaluating Moral Decision-Making of Foundation Models from an Agentic Perspective
ゲーム『Detroit: Become Human』を用い、基盤モデル自身が一人称のエージェントとして道徳的ジレンマで意思決定する様子を評価するマルチモーダルベンチマークを提案し、12モデルの行動が状況依存で大きく変化することを示した。
詳しい要約
1. どんなもの？
2. 先行研究と比べてどこがすごい？
3. 技術・手法の肝は？
4. どうやって有効だと検証した？
5. 議論はある？
6. 次に読むべき論文は？
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Chaoran Li, Yukun Li, Zeyuan Zhao, Shao Zhang, Xihuai Wang, Ying Wen
分類: cs.AI, cs.CL, cs.CY
原文アブストラクト
Foundation models are increasingly deployed as agentic systems that perceive context and act autonomously, raising the need to evaluate their moral behavior as active decision-makers. Existing benchmarks, however, largely ask models to judge human actions from third-person textual vignettes, leaving untested how models behave when they themselves become the acting agent. We introduce ABHBench, a multimodal benchmark built from Detroit: Become Human, an interactive narrative game, to evaluate how foundation models make moral decisions in first-person human-android dilemmas. ABHBench places foundation models in branching narrative scenarios with visual observations, dialogue, memory, and action choices. Inspired by Asimov's Three Laws of Robotics, our framework measures Harmlessness, Obedience, and Self-preservation to capture the core tensions in human-AI relations, along with Autonomy, Stability, and Certainty to assess decision-making patterns. Across 12 foundation models, we find that moral behavior changes substantially under situated agentic context: 75% of models shift significantly between a text-only abstraction and the matched multimodal DBH scenario, and reasoning-enhanced models show no monotonic gains in consistency or human-oriented behavior. These results highlight the necessity of evaluating moral behavior in situated agentic contexts rather than relying on third-person judgment alone.