Double discriminative face super-resolution network with facial landmark heatmaps

At present, most of face super-resolution (SR) networks cannot balance the visual quality and the pixel accuracy. The networks with high objective index values often reconstruct too smooth images, while the networks which can restore texture information often introduce too much high-frequency noise and artifacts. Besides, some face super-resolution networks do not consider the mutual promotion between the extracting face prior knowledge part and the super-resolution reconstruction part. To solve these problems, we propose the double discriminative face super-resolution network (DDFSRNet). We propose a collaborative generator and two discriminators. Specifically, the collaborative generator, including the face super-resolution module (FSRM) and the face alignment module (FAM), can strengthen the reconstruction of facial key components, under the restriction of the perceptual similarity loss, the facial heatmap loss and double generative adversarial loss. We design the feature fusion unit (FFU) in FSRM, which integrates the facial heatmap features and SR features. FFU can use the facial landmarks to correct the face edge shape. Moreover, the double discriminators, including the facial SR discriminator (FSRD) and the facial landmark heatmap discriminator (FLHD), are used to judge whether face SR images and face heatmaps are from real data or generated data, respectively. Experiments show that the perceptual effect of our method is superior to other advanced methods on 4x reconstruction and fit the face high-resolution (HR) images as much as possible.