MTCNet: A Multi-task Collaborative Network for Accurate Identification of Different Types of Colorectal Lesions

Automatic classification and segmentation of lesions in colonoscopy images is an important research direction in computer-aided diagnosis and plays a crucial role in clinical applications. However, most existing methods fail to sufficiently explore the relationship between lesion types and shapes, and the boundaries of lesions are often unclear. To address these problems, this paper proposes a multi-task collaborative network—MTCNet. MTCNet optimizes both classification task and segmentation task, and enhances the boundary awareness in the segmentation branch. To better exploit the relationship between boundary shapes and lesion categories, We propose a deep cooperation strategy based on Multi-Scale Transformer module. Through this module, the segmentation and classification tasks can be jointly optimized, enabling effective interaction between the two tasks. Meanwhile, to further enhance the model’s ability to recognize lesion boundaries, an Edge-Guided Attention (EGA) module based on the Laplacian algorithm is proposed. In addition, we construct a multi-center dataset PADset. Experimental results demonstrate that the proposed method outperforms commonly used approaches in both segmentation and classification tasks on the PADset dataset and the public SUN dataset.