Multimodal Knowledge Distillation for Arbitrary-Oriented Object Detection in Aerial Images

Recently, many arbitrary-oriented object detection (AOOD) methods have been proposed and applied to remote sensing and other fields. For aerial platforms, lightweight structure and multimodal adaptations of convolutional neural network (CNN) models are urgently needed. Due to the limited model size, the performance of existing lightweight AOOD methods is low, especially in multimodal tasks. In this paper, a multimodal knowledge distillation (MKD) method is proposed for AOOD in aerial images. In MKD, a multimodal dynamic label assignment strategy is designed to select the optimal positive samples dynamically to adapt to different modalities and environments. Different multimodal localization and feature distillation modules are designed to make multimodal knowledge to be complementary and effectively learned by the lightweight model. Experiments on the public dataset demonstrated the effectiveness and advancement of MKD.