変分模倣学習によるマニピュレータ書道のエンドツーエンド計画
End-to-end Manipulator Calligraphy Planning via Variational Imitation Learning
画像と姿勢データを組み合わせた深層模倣学習により、ペン先の回転を含む3次元軌道で日本の書道をロボットアームで自動計画する手法を提案した。
著者: Fangping Xie, Pierre Le Meur, Charith Fernando
分類: cs.RO, cs.AI
原文アブストラクト
Planning from demonstrations has shown promising results with the advances of deep neural networks. One of the most popular real-world applications is automated handwriting using a robotic manipulator. Classically it is simplified as a two-dimension problem. This representation is suitable for elementary drawings, but it is not sufficient for Japanese calligraphy or complex work of art where the orientation of a pen is part of the user expression. In this study, we focus on automated planning of Japanese calligraphy using a three-dimension representation of the trajectory as well as the rotation of the pen tip, and propose a novel deep imitation learning neural network that learns from expert demonstrations through a combination of images and pose data. The network consists of a combination of variational auto-encoder, bi-directional LSTM, and Multi-Layer Perceptron (MLP). Experiments are conducted in a progressive way, and results demonstrate that the proposed approach is successful in completion of tasks for real-world robots, overcoming the distribution shift problem in imitation learning. The source code and dataset will be public.