QTOP:分位クリティックによるヒューマノイド動作追従のスケーリング
QTOP: Scaling Humanoid Motion Tracking with Quantile Critics
状態と参照動作のペアに条件づけた分布型アドバンテージ推定を導入し、多様な動作の学習干渉を抑えてサンプル効率を高める動作追従手法QTOPを提案。シミュレーションと実機で最大10%向上し、スケーリング則も示した。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Markus Peschl, Dana Kianfar, Daniel Dijkman
分類: cs.RO
原文アブストラクト
Learning human motion tracking policies through reinforcement learning (RL) has emerged as a core capability that underpins a majority of recent progress in humanoid whole-body control. However, achieving generalization to unseen motions at high fidelity requires jointly scaling training to thousands of GPU and dataset hours. This work hypothesizes that motion tracking is inherently compute-bound because training on a diverse set of motions introduces learning interference, similar to multi-task RL. To tackle this, we propose Quantile Tracking Optimization (QTOP), which improves sample efficiency by introducing a distributional advantage estimate conditioned on state and reference pairs. QTOP outperforms on and off-policy RL baselines both in simulation and real-world tracking scenarios on multiple datasets by up to 10\%. Furthermore, we establish a scaling law for QTOP and find that, aside from scaling predictably, dataset size has minimal impact on performance for moderately sized GPU hour regimes.
