ORCESTRA: 複合現実におけるVLM駆動の視覚的ロボットプログラミング
ORCESTRA: VLM-driven Visual Robot programming in Mixed Reality
複合現実環境で、ユーザーがロボットのデジタルツインを配置し、軌道を教示したり、言語で指示を与えることで、視覚言語モデルが構造化された計画に変換し、ロボットをプログラミングするシステムを提案した論文。
著者: Ivan Snegirev, Elizaveta Semenyakina, Mikhail Konenkov, Artem Lykov, Miguel Altamirano Cabrera, Dzmitry Tsetserukou
分類: cs.RO, cs.CV, cs.HC
原文アブストラクト
ORCESTRA is a mixed-reality system for programming robot digital twins through no-code waypoint teaching and language-guided control. In a passthrough mixed-reality workspace, users place robot twins on real surfaces, teach trajectories, save robot-relative episodes, or issue spoken/typed commands that a vision-language model converts into structured digital-twin plans. Both interaction modes share a backend for metric grounding, embodiment-aware validation, preview, confirmation, and digital-twin execution. The system supports heterogeneous robot embodiments, including fixed-base manipulators, a mobile base, and a humanoid robot, demonstrating MR validation as a safety layer for language-guided robot programming before physical deployment.