Collaborative Classification of Hyperspectral and LiDAR Data Based on Gram Matrices Constrained Fusion Net

Multi-sensor information collaborative utilization has attracted considerable attention in remote sensing area. While earth observation benefits from information diversity, multi-sensor collaborative classification technique is still confronted with varied challenges, including inconsistent data volume, different data structures, and uncorrelated physical properties. In this paper, a Gram matrices constrained fusion net (GMCF-Net) is designed for controlling multi-source heterogeneous information and improving classification performance. Exploiting the capability of the Gram matrix in capturing image textures, a multi-source structure control module is constructed to simultaneously address issues involved with different data volume and data structures, which matches the Gram matrices of multi-domains in a double-interweaving pattern. Finally, classification results are obtained based on the discriminative fusion from GMCF-Net. Extensive experiments built from two benchmark remote sensing data sets are reported, and the results demonstrate that the proposed framework yields state-of-the-art performance on hyperspectral and LiDAR data collaborative classification.