事前分布内でのドラッグ:分布制約下でのテキスト条件付きポイントベース画像編集
Drag within Prior Distribution: Text-Conditioned Point-Based Image Editing within Distribution Constraints
拡散モデルを用いたポイントベース画像編集において、CLIPモデルによる意味整合性の評価と事前分布保持損失により、編集結果を元のデータ分布内に保ちつつ、大規模移動時のアーティファクトを低減する手法を提案した。
著者: Haoyang Hu, Masataka Seo, Yen-Wei Chen
分類: cs.CV
原文アブストラクト
Diffusion-based point editing methods have gained significant traction in image editing tasks due to their ability to manipulate image semantics and fine details by applying localized perturbations on the manifold of noise latent. However, these approaches face several limitations. Traditional point-based editing relies on pairs of handle and target points to define motion trajectories, which can introduce ambiguity or unnecessary alterations. Furthermore, when the distance between the handle and target points is large, the accumulated perturbations often cause the noise latent deviation from inversion score trajectory, resulting in unnatural artifacts. To address these issues in global editing tasks, we introduce a CLIP-based model to evaluate and guide intermediate editing steps, ensuring that the generated results remain both semantically aligned. Additionally, we propose a prior-preservation loss that constrains the optimized latent code to stay within the sampling space of the diffusion prior, improving consistency with the original data distribution, to ensure the model generates images along a familiar score trajectory. For fine-grained tasks, we present a directionally-weighted point tracking mechanism that steers the editing process toward the target direction within similar feature regions. This improves both the tracking accuracy and generation quality, while also reducing the editing time.