視覚障害学生のための包括的STEAM教育：高度なコンピュータビジョンを活用したコーディングとロボティクス教育フレームワーク
Inclusive STEAM Education: A Framework for Teaching Cod-2 ing and Robotics to Students with Visually Impairment Using 3 Advanced Computer Vision
視覚障害のある学生が迷路探索ロボットをプログラミングできるよう、CLIPによる画像のテキスト化と音声VR、SLAMによるフィードバックを組み合わせた学習支援システムを提案した。
著者: Mahmoud Hamash, Md Raqib Khan, Peter Tiernan
分類: cs.HC, cs.CV, cs.RO
原文アブストラクト
STEAM education integrates Science, Technology, Engineering, Arts, and Mathematics to foster creativity and problem-solving. However, students with visual impairments (VI) encounter significant challenges in programming and robotics, particularly in tracking robot movements and developing spatial awareness. This paper presents a framework that leverages pre-constructed robots and algorithms, such as maze-solving techniques, within an accessible learning environment. The proposed system employs Contrastive Language-Image Pre-training (CLIP) to process global camera-captured maze layouts, converting visual data into textual descriptions that generate spatial audio prompts in an Audio Virtual Reality (AVR) system. Students issue verbal commands, which are refined through CLIP, while robot-mounted stereo cameras provide real-time data processed via Simultaneous Localization and Mapping (SLAM) for continuous feedback. By integrating these technologies, the framework empowers VI students to develop coding skills and engage in complex problem-solving tasks. Beyond maze-solving applications, this approach demonstrates the broader potential of computer vision in special education, contributing to improved accessibility and learning experiences in STEAM disciplines.