Class Incremental Learning with Important and Diverse Memory

Class incremental learning (CIL) has been attracting increasing attention in computer vision and machine learning communities, where a well-known issue is catastrophic forgetting. To mitigate this issue, a popular approach is to utilize the replay-based strategy, which stores a small portion of past data and replays it when learning new tasks. However, selecting valuable samples from previous classes for replaying remains an open problem in class incremental learning. In this paper, we propose a novel sample selection strategy aimed at maintaining effective samples from old classes to address the catastrophic forgetting issue. Specifically, we employ the influence function to evaluate the impact of each sample on model performance, and then select important samples for replay. However, given the potential redundancy among selected samples when only considering importance, we also develop a diversity strategy to select not only important but also diverse samples from old classes. We conduct extensive empirical validations on the CIFAR10 and CIFAR100 datasets and the results demonstrate that our proposed method outperforms the baselines, effectively alleviating the catastrophic forgetting issue in class incremental learning.