AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion

Yitong Jiang, Zhaoyang Zhang, Tianfan Xue, Jinwei Gu
The Chinese Univerity of Hong Kong
Paper Supplementary Code arXiv

AutoDIR can automatically deal with multiple image restoration tasks with text prompt instructions via an iterative process and also provides users the option to edit according to their preferences.

Automatic Image Restoration on Multiple Unknown Degradation via Iterative Process

MY ALT TEXT

Iterative image editing on unseen Under-Display Camera dataset TOLED.

Automatic Image Restoration on Different Image Restoration Tasks

Method

>
MY ALT TEXT

We employ a blind image quality assessment model (BIQA), which automatically identifies the predominant degradation in the image. The corresponding text embedding is then extracted and supplied to the image editing module. Additionally, users have the option to provide manual instructions, such as "Reduce the noise," by passing the instruction text embedding. Next, our diffusion-model-based all-in-one image editing model (AIE) leverages either text embedding generated by BIQA or text embedding of user prompts, depending on the user's preference, to generate the edited image. This editing process aims to address the identified degradation and improve the overall image quality. Finally, we combine the edited image with the original image and feed them into our Structural-Correction module (SCM).

BibTeX

@article{jiang2023autodir,
  title={AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion},
  author={Jiang, Yitong and Zhang, Zhaoyang and Xue, Tianfan and Gu, Jinwei},
  journal={arXiv preprint arXiv:2310.10123},
  year={2023}
}