An Infrared and Visible Image Fusion Network based on Two-stream Feature Decomposition

Infrared and visible image fusion is an important task in areas such as video surveillance and vehicle navigation. It aims to integrate complementary features of the source images to generate a fused image containing salient targets and rich texture details. However, most existing fusion algorithms ignore the fact that infrared and visible modalities have different representations of features at different frequencies. To address the challenges of cross-modal feature decomposition and fusion, this paper proposes a two-stream feature decomposition network for infrared and visible image fusion, called TFDFusion. Firstly, a scene encoder with Transformer as the architecture extracts low-frequency global scene features, while an attribute encoder extracts high-frequency local attribute features using CNN. Secondly, self-attention and cross-attention fusion modules are used to facilitate feature decomposition and adaptive fusion of complementary features, respectively. In addition, we construct the fusion loss using a contrast mask based on variance computation, which guides the fusion network to retain the high-contrast regions in the source image. Extensive experiments show that our TFDFusion achieves satisfactory results in infrared and visible image fusion tasks, outperforming state-of-the-art methods in terms of visual quality.