内容適応型空間アドレッシングによる階層的時空間ワールドモデリング
Hierarchical Spatiotemporal World Modeling with Content-Adaptive Spatial Addressing
海馬と嗅内皮質の仕組みに着想を得て、局所時間RoPEと多周期空間RoPEを階層的に組み合わせた固定容量メモリのワールドモデルHiSTFormerを提案し、長期的な予測horizonへの汎化性能を向上させた。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Tiantian Li, Junfeng Lu, Yifei Wang, Zhaoyang Zhang, Lei Wang, Tianyi Qian
分類: cs.NE, cs.LG
原文アブストラクト
Long-horizon world modeling in partially observable environments requires compact memories that preserve spatial structure despite accumulated localization drift. Inspired by hippocampal episodic and spatial memory and the multi-periodic coding of entorhinal grid cells, we propose HiSTFormer, a hierarchical spatiotemporal world model, which adopts a layer-wise organization of local temporal RoPE and multi-periodic spatial RoPE, together with content-conditioned address offsets, to capture recent temporal context and long-range spatial structure. A fixed-capacity memory keeps KV storage and per-step attention cost bounded as the sequence grows. On MemoryMaze and HM3D, HiSTFormer outperforms strong world-model baselines within the training sequence length and generalizes substantially better to prediction horizons far beyond those seen during training, while using a compact model. The hierarchical spatiotemporal design remains effective across deeper Transformer stacks, with consistent gains as depth increases. These results show that hierarchical spatiotemporal organization and content-adaptive spatial addressing provide effective inductive biases for efficient long-horizon world modeling.
