CSPDAE: Colorization via SuperPixel Downsampler Denoising AutoEncoder towards enhanced color structural clarity

Based on the various structural textures, sizes, and shapes of the gray image, the SuperPixel (SP)-based methods convert the rigid pixels into adaptable image patches that share common features. As a result, the SP-based colorization technique not only enhances the color appearance but also preserves the topological integrity of the images. Despite the effectiveness of neural network-based colorization methods, their integration with SP segmentation has traditionally been complex and cumbersome. To address this issue, we propose the Colorization SP Downsampler Denoising AutoEncoder (CSPDAE), where the SP downsampler integrates SP segmentation directly into the network, eliminating the need for any prior input. The SP downsampler addresses the challenges of computational complexity and small region clustering, which are the main obstacles preventing Transformer architectures from being applied to pixel-level segmentation, through the use of SP cross-attention and aggregated positional embedding (APE). Furthermore, we have incorporated a Color Weight (CW) loss, based on the CIEDE2000 color difference formula, to ensure balanced pixel sampling and to improve the precision of detailed color representation. The experimental results confirm the effectiveness of our method, demonstrating its capacity to produce colors with greater structural accuracy and visually appealing details.