CCIR: high fidelity face super-resolution with controllable conditions in diffusion models

Diffusion probabilistic models have demonstrated great potential in producing realistic-looking super-resolution (SR) images. However, the realism doesn’t necessarily guarantee that the SR images are faithful to the ground truth high- resolution images. This paper develops a novel training-free framework namely Iterative Refinement with Controllable Condition (CCIR), for face SR based on controllable prior conditions in diffusion model. The goal is to generate SR images that are both realistic and faithful to the ground truth by controlling the prior conditions. Our framework consists of a pre-trained SR network, Local Implicit Image Function (LIIF), and a pre-trained diffusion model. The LIIF enhances the conditions provided by low-resolution images, while the diffusion model recovers fine details in the SR images. Notably, for the diffusion model, we propose a non-uniform low-pass filtering sampling strategy that dynamically adds controllable conditions to latent features during sampling process. This strategy provides a flexible balance between fidelity and realism in SR images, enabling the restoration of highly similar SR images from the same low-resolution input with different noise samples. Extensive experiments conducted on the benchmark of facial SR task demonstrate CCIR outperforms the state-of-the-art SISR methods, in qualitative and quantitative assessments, particularly in the case of magnifying very-low-resolution images or high-magnification factors.