MSTDKD: a framework of using multiple self-supervised methods for semisupervised learning

Image classification is a basic task in the field of computer vision, and general image classification task training requires a large amount of labeled data to achieve good generalization performance. However, in practical applications, the cost of obtaining labeled data is expensive. In contrast, unlabeled images are easy to obtain, so semi-supervised image classification is more meaningful for research. This paper pro- poses a framework for semi-supervised classification utilizing multiple self-supervised methods. Our approach is divided into three steps, firstly, pre-train multiple models on unlabeled data using different self-supervised methods. Then use the labeled data to fine-tune these models except the model pre-training by Contrastive learning to obtaining multiple self-supervised teacher models. Finally, the multi-teacher knowledge distillation framework is used to transfer the knowledge of multiple self-supervised teacher models to the model pre-training by Contrastive learning to help it achieve further performance. We conducted experiments on cifar10 and miniimagenet60. Our method achieves further results than using only a single self-supervised method, and also achieves superior performance compared to other semi-supervised methods.