Adaptive Store-and-Recall Memory for Domain Continual Learning

Domain continual learning requires overcoming catastrophic forgetting while training in sequential domains. A common solution is to store and replay samples from previous domains to preserve knowledge, but such replay-based strategies raise privacy and storage concerns and are often impractical in real-world settings. In this paper, we propose a rehearsal-free framework, which leverages a store-and-recall strategy to generate a compatible optimization subspace, ensuring effective transfer for preventing catastrophic forgetting. To establish a foundation for effectively leveraging historical knowledge, we introduce a Frequency-Aware Filtering, which extracts key representations in the frequency domain and compacts them into internalmemory to support rehearsal-free retention. To further exploit historical information effectively, we design a Gain-Adaptive Gating mechanism integrated with a Knowledge Fusion module, which selectively activates historical knowledge based on gain signals to avoid negative transfer caused by distribution shifts. Extensive experiments show that our method outperforms state-of-the-art continual learning methods across standard benchmarks under the domain-incremental setting.