PAFusion: A general image fusion network with adversarial representation learning

A perceptually adversarial fusion network is proposed to achieve concise and adaptive unsupervised training across various image fusion tasks. The proposed method features the adaptation of pre-trained representations through a novel discriminator, which evaluates the fused image from the generator by distinguishing its representation from a learned representation. A spatial context attention mechanism is proposed to obtain the learned representation by adapting and fusing the representations of the source image. The attention models scene salience, with its adapted representation being discerned by the discriminator, which in turn facilitates the acquisition of the learned representation. Moreover, a feature salience loss function is proposed, which compares the fused image representation with the learned representation through recursive downsampling to refine the generated content from multiple scales. The proposed method is evaluated on multi-focus, multi-exposure, medical, and infrared-visible image fusion tasks through subjective and objective comparisons with state-of-the-art fusion methods, as well as ablation studies. The notable vision effect and favorable numerical metrics of the proposed method achieved in these evaluations demonstrate the effectiveness of adversarial representation learning in enhancing fusion training in the absence of explicit ground truths. The code is released at https://github.com/6xw/PAFusion.