Gradient Recalibration for Improved Visibility of Tail Classes in Supervised Contrastive Learning

Contrastive learning and supervised contrastive learning (SCL) have proven their effectiveness in graphs. However, they suffer from representation collapse when meet imbalance. To address these, we first proposed a quantitative model, similar to the Thomson problem when all classes are of equal size. It maps classes on the hypersphere where different classes repel each other. Based on this, we theoretically showed that when applied to imbalanced node classification, tail classes will be pushed together due to the dominating repellent forces from head classes. Therefore, we recalibrate the gradient of SCL loss to enforce all classes to maintain a uniform distribution in feature space, improving the visibility of tail classes. Extensive experiments on graph datasets indicates that the proposed method can significantly enhance the uniformity of class representation, thus achieving better performance for imbalanced node classification.