Medical image retrieval based on unclean image bags

Traditional content-based image retrieval (CBIR) scheme with assumption of independent individual images in large-scale collections suffers from poor retrieval performance. In medical applications, images usually exist in the form of image bags and each bag includes multiple relevant images of the same perceptual meaning. In this paper, based on these natural image bags, we explore a new scheme to improve the performance of medical image retrieval. It is feasible and efficient to search the bag-based medical image collection by providing a query bag. However, there is a critical problem of noisy images which may present in image bags and severely affect the retrieval performance. A new three-stage solution is proposed to perform the retrieval and handle the noisy images. In stage 1, in order to alleviate the influence of noisy images, we associate each image in the image bags with a relevance degree. In stage 2, a novel similarity aggregation method is proposed to incorporate image relevance and feature importance into the similarity computation process. In stage 3, we obtain the final image relevance in an adaptive way which can consider both image bag similarity and individual image similarity. The experiments demonstrate that the proposed approach can improve the image retrieval performance significantly.