実行時に推論を切り離せる柔軟で効率的な自動運転フレームワークDRIFT
DRIFT: Detachable Reasoning for Runtime-Flexible and Efficient End-to-End Driving
軽量言語モデルと代理推論特徴を組み合わせ、実行時に言語推論をオンオフしても性能を保てる省パラメータのEnd-to-End自動運転手法を提案した。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Emad Aghajanzadeh, Bethany Anne Wong, Leonhard Kunze, Moritz Thoma, Maximilian Forstenhäusler, Nael Fasfous, Manoj Rohit Vemparala, Pierpaolo Mori, Christos Anagnostopoulos, Daniel Mueller-Gritschneder, Ulf Schlichtmann
分類: cs.CV, cs.CL, cs.LG, cs.RO
原文アブストラクト
The integration of vision-language models (VLMs) into end-to-end (E2E) autonomous driving has demonstrated strong potential by providing scene description and reasoning capabilities. However, VLMs are computationally expensive, and switching language reasoning on and off at runtime without hurting closed-loop performance has not been demonstrated, which hinders practical adoption. To address these issues, we propose DRIFT, an E2E autonomous driving framework that integrates a lightweight language model (LM) for visual question answering and action commentary. By efficiently projecting intermediate driving representations into the language embedding space, we demonstrate that a compact LM (<150M parameters) can achieve strong scene description and action commentary. Furthermore, we introduce a runtime-flexible planning module that is trained on surrogate reasoning features distilled from the LM and can be conditioned on either the LM or the surrogate reasoning features at inference. This design enables bypassing the LM entirely when required by deployment constraints, without sacrificing driving performance. With only 255M parameters, DRIFT reaches up to a Driving Score (DS) of 97.03 and a Success Rate of 89.39% (mean over three runs) on Bench2Drive by interleaving its LM and surrogate reasoning at a chosen reasoning rate. This is on par with the strongest non-language models while being far more compact than existing VLM-based models. DRIFT further achieves competitive results on the real-world NAVSIM and AlpaSim benchmarks at significantly lower computational cost than existing VLM-based approaches. Project page: https://drift-e2e.github.io/
関連論文
- 成果指向蒸留:自動運転におけるVLM推論を進化させる教師-学生フレームワーク自動運転/VLM
- Robusto-2: リマとニューヨーク市における自動運転のための人間とVLMのベンチマーキング自動運転/VLM
- 工事区間におけるVLM軌道計画の課題:緩和策と堅牢な自動運転に向けて自動運転/VLM
- CVPR2024 E2Eチャレンジ準優勝ソリューション:視覚言語モデルを用いたエンドツーエンド自動運転自動運転/VLM
- MTRDrive: 記憶とツールの協調推論によるコーナーケースに頑健な自動運転自動運転/VLM
- VLAD: 階層的計画と解釈可能な意思決定プロセスを備えたVLM拡張自動運転フレームワーク自動運転/VLM
