Classification of Multisource Remote Sensing Images Using Multimodal Equilateral Absorption Network

Fusing multisource remote sensing data is an important approach to improve pixel-wise classification performance. Generally, the richer the information input into the model, the more diverse the knowledge it can learn, thereby improving classification performance. However, existing fusion methods are usually only applicable to two modal inputs and find it difficult to balance the consistency and diversity of multisource features. In this paper, we propose a novel classification network named multimodal equilateral absorption network (MEANet) which can fuse multiple kinds of remote sensing images. Specifically, three modal features are firstly extracted by a three-branch CNN. Then, the cross-modal interacting module (CIM) is utilized to realize feature fusion on the multimodal features. Thirdly, the improved triplet loss is designed to make a tradeoff between feature diversity and consistency, thus making the network acquire multisource information more efficiently. Finally, pixel-wise summation and a fully connected (FC) layer are utilized to obtain the final classification results. Experiments on two datasets show that the proposed MEANet has a competitive classification performance compared to several state-of-the-art methods.