Target-aware fusion of infrared and visible images

Image fusion technology involves the use of complementary information from multiple sensors to generate a composite image that can highlight the details in the region of interest. Some recent methods have tackled the problem of characterization of different source image features that are lacking in conventional methods. However, during fusion processing, these methods may lose some information of interest such as smoke. This paper proposes a method called target-aware decomposition and parallel gradient fusion (TAD-PGF) that fuses infrared and visible images to maintain the high brightness characteristics of infrared targets while transferring the appearance of both source images to the fused image, where 'appearance' means the details pertaining to the environment and 'infrared target' often means hot objects such as human body. The target layer is extracted from the infrared image and used as a guide to extract appearance-related information from the visible image. Given that the background of the infrared image contains useful background information, a parallel gradient fusion scheme is proposed to fuse the relevant features with appearance-related information in the visible image. The final blended image is obtained by adding a target layer and a fused appearance layer directly to the fused image. Numerous experiments using publicly available databases were conducted to provide qualitative and quantitative comparisons between the state-of-the-art methods and the proposed TAD-PGF. The results reveal that the TAD-PGF can attain good visual effect in various scenarios and maintain useful information from source images to enhance the details of interest.