TY - GEN
T1 - Multimodal Knowledge Distillation for Arbitrary-Oriented Object Detection in Aerial Images
AU - Huang, Zhanchao
AU - Li, Wei
AU - Tao, Ran
N1 - Publisher Copyright:
© 2023 IEEE.
PY - 2023
Y1 - 2023
N2 - Recently, many arbitrary-oriented object detection (AOOD) methods have been proposed and applied to remote sensing and other fields. For aerial platforms, lightweight structure and multimodal adaptations of convolutional neural network (CNN) models are urgently needed. Due to the limited model size, the performance of existing lightweight AOOD methods is low, especially in multimodal tasks. In this paper, a multimodal knowledge distillation (MKD) method is proposed for AOOD in aerial images. In MKD, a multimodal dynamic label assignment strategy is designed to select the optimal positive samples dynamically to adapt to different modalities and environments. Different multimodal localization and feature distillation modules are designed to make multimodal knowledge to be complementary and effectively learned by the lightweight model. Experiments on the public dataset demonstrated the effectiveness and advancement of MKD.
AB - Recently, many arbitrary-oriented object detection (AOOD) methods have been proposed and applied to remote sensing and other fields. For aerial platforms, lightweight structure and multimodal adaptations of convolutional neural network (CNN) models are urgently needed. Due to the limited model size, the performance of existing lightweight AOOD methods is low, especially in multimodal tasks. In this paper, a multimodal knowledge distillation (MKD) method is proposed for AOOD in aerial images. In MKD, a multimodal dynamic label assignment strategy is designed to select the optimal positive samples dynamically to adapt to different modalities and environments. Different multimodal localization and feature distillation modules are designed to make multimodal knowledge to be complementary and effectively learned by the lightweight model. Experiments on the public dataset demonstrated the effectiveness and advancement of MKD.
KW - Aerial images
KW - arbitrary-oriented object detection
KW - knowledge distillation
KW - multimodal
UR - http://www.scopus.com/inward/record.url?scp=86000380736&partnerID=8YFLogxK
U2 - 10.1109/ICASSP49357.2023.10097119
DO - 10.1109/ICASSP49357.2023.10097119
M3 - Conference contribution
AN - SCOPUS:86000380736
T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
BT - ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing, Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023
Y2 - 4 June 2023 through 10 June 2023
ER -