A MODEL COMPRESSION METHOD BASED ON MULTI-TEACHER DISTILLATION FOR SAR SCENE CLASSIFICATION

In recent years, with the advancement of deep learning, convolutional neural networks tailored for processing image content have gained increasing attention within the remote sensing imagery domain. However, deep convolutional neural networks that exhibit exceptional performance often encompass a substantial number of parameters, thereby necessitating heightened computational resources and time for network training and prediction. Given the constraints posed by resource limitations in the realm of remote sensing, deploying such high-performance networks onto these devices becomes a challenging endeavor. To address this set of challenges, this paper introduces a methodology grounded in target differential probability for multi-teacher knowledge distillation. This approach effectively reduces the model's complexity while preserving the network models' commendable performance. By conducting a statistical analysis of the model's output probability distribution and computing the differential probabilities between teacher and student networks, we regulate the output proportions of teachers within the multi-teacher model. As a result, a high-performing student network is cultivated. In aggregate, experimental validation on two distinct datasets substantiates the efficacy of this approach in facilitating efficient knowledge transfer, consequently yielding heightened performance outcomes.