An Efficient Marine Species Classifier via Knowledge Distillation and Hybrid Attention

Marine species recognition is a crucial task in ocean exploration. Despite the impressive performance of deep-learning-based classification methods, these approaches often suffer from high complexity in terms of model parameters, storage size, and FLOPs. This complexity leads to slow inference and high consumption of computational resources, which presents a significant challenge for ocean engineering. To address this challenge, we propose a lightweight framework for marine species recognition based on knowledge distillation and the attention mechanism. Our method introduces a relatively complex teacher classifier with the backbone of ResNet-18, and an attention module to enhance its performance. We design the student classifier using only a few convolutional layers, thereby significantly reducing the number of parameters. Extensive experiments demonstrate that the student network achieves competitive performance to other representative lightweight models such as ShuffleNetv2 and MobileNetv3-small, while its model parameters are only 1.624% of the MobileNetv3-small.