TStore: テンソル中心の圧縮によるAIモデルハブの再考
TStore: Rethinking AI Model Hub with Tensor-Centric Compression
モデルハブのストレージ削減のため、テンソル単位の重複排除と圧縮を行うシステムTStoreを提案した。
著者: Tingfeng Lan, Zirui Wang, Yunjia Zheng, Zhaoyuan Su, Juncheng Yang, Yue Cheng
分類: cs.DC, cs.AI, cs.LG
原文アブストラクト
Modern AI models are growing rapidly in size and redundancy, leading to significant storage and distribution challenges in model hubs. We present TStore, a tensor-centric system for reducing storage overhead through fine-grained deduplication and compression. TStore leverages tensor-level fingerprinting and clustering to identify redundancy across models without requiring annotations. Our design enables efficient storage reduction while preserving model usability and performance. Experiments on real-world model repositories demonstrate substantial storage savings with minimal overhead.