UC-YOLOX: Enhancing urothelial carcinoma detection with an improved YOLOX architecture leveraging attention mechanisms

Bladder cancer, particularly urothelial carcinoma (UC), remains a major global health concern. Urine cytology offers a non-invasive and cost-effective approach for UC screening. Although recent advances in artificial intelligence (AI) have shown promise in improving diagnostic accuracy, most existing AI-based tools are limited to classifying UC and normal urothelial cells. These methods often require manual selection of regions of interest or apply machine learning algorithms to all cells indiscriminately, resulting in increased computational complexity, suboptimal user experience, and the potential overlooking of abnormal cells. To address these limitations, we propose UC-YOLOX, an enhanced YOLOX-based algorithm for the automated detection of abnormal urothelial cells in whole slide images (WSIs) from urine cytology. UC-YOLOX integrates a Vision Transformer (ViT) block into its backbone to capture long-range dependencies and enhance feature fusion, enabling more accurate differentiation between suspicious UC cells, normal cells, and background regions. Additionally, we incorporate the Convolutional Block Attention Module (CBAM) and introduce skip connections to highlight morphological features critical for distinguishing normal from abnormal cells, thereby mitigating the loss of cell characteristics caused by downsampling. Furthermore, we replace the traditional convolutional layers in the detection head with a Fully Connected Head (FC-Head), which improves spatial sensitivity and enhances classification accuracy for both low-grade and high-grade UC. We evaluated UC-YOLOX on a dataset of 751 WSIs collected from a local pathology center. Experimental results show that UC-YOLOX achieves a mean Average Precision (mAP) of 92.10%, significantly outperforming existing methods. Moreover, the model was validated on publicly available datasets, where it also achieved superior performance.