CLIP and image integrative prompt for anterior mediastinal lesion segmentation in CT image

The automatic segmentation of anterior mediastinal lesions in enhanced CT imaging is of significant importance in clinical diagnostics. Anterior mediastinal lesions are characterized by various types and blurred boundary, which increases the difficulty of anterior mediastinal lesions segmentation. This study leverages the robust zero-shot classification capability and semantic expression of CLIP to formulate CLIP-prompt that express the semantic correlation between images and text, so that CLIP-prompt guided cross-attention has been proposed. By integrating the CLIP-prompt into the image features through cross-attention, the network can focus more intently on the lesion areas. Additionally, to better capture the unknown categorical features of the images, this paper introduces a learnable image prompt that works in conjunction with an attention module integrated with textual information, thereby enhancing the constraints on the segmentation targets. Finally, to address the blurred boundary of the anterior mediastinal lesion, this study proposes a boundary-enhanced loss. By augmenting the weights of difficult-to-segment edge points, the network is enabled to focus on these challenging boundary areas, consequently improving the segmentation accuracy of these points. Compared to existing state-of-the-art methods, our approach has achieved an overall Dice coefficient of 89.43% and has achieved good performance in terms of ASSD metric for segmentation edges.