Improved Stroke Lesion Segmentation via Cross-Model Knowledge Distillation

Stroke lesion segmentation is a crucial step in stroke disease analysis as it provides essential anatomical information for the diagnosis and prognosis. Deep learning approaches, including the convolutional neural network (CNN) and Transformer, have improved the accuracy of automated stroke lesion segmentation. Since CNN-based models generally learn to extract local features with convolutional filters, whereas Transformer-based models excel in modeling long-range dependencies with self-attention mechanisms, combining the strengths of the two types of models may bring additional benefits. Existing studies have proposed various hybrid network structures combining convolution and self-attention; however, empirical evidence shows that these hybrid structures do not necessarily outperform purely CNN-based or Transformer-based methods. Thus, this work further explores the integration of the strengths of CNNs and Transformers. Instead of constructing a hybrid network architecture, we propose a cross-model interaction method based on knowledge distillation, which can effectively allow CNN-based and Transformer-based models to learn the strength of each other. The guidance is achieved with an adaptive recall-enhancing loss for knowledge distillation, which suppresses negative knowledge transfer and encourages the model to reduce false negative predictions that are common in stroke lesion segmentation. We validated the proposed method on two public stroke lesion segmentation datasets, where it improves the segmentation performance compared with the best competing segmentation model by 2.0% and 1.4% for the ISLES and ATLAS dataset, respectively.