Caricature-visual face recognition based on jigsaw solving and modal decoupling

In recent years, face recognition technology has made significant progress in the field of real visual images, yet face recognition involving caricature-visual images remains a challenge due to the exaggerated and unrealistic features of caricature faces. To tackle this issue, this paper introduces the Caricature-visual Face Recognition Model Based on Jigsaw Solving and Modal Decoupling (CVF-JSM). The CVF-JSM consists of two modules: feature extraction and decoupling. The feature extraction module incorporates a graph attention network at the intermediate stage of the backbone network, which constructs and solves jigsaw puzzles to enable the network to extract shape features. The feature decoupling module features a three-branch structure that divides the features into modal and identity features. The real and caricature face recognition branches separate identity features for recognition through parameter sharing and orthogonality constraints. The feature common subspace alignment branch maps the anchor image, as well as the positive and negative sample images, into a common subspace to isolate identity features. Subsequently, by aligning the features, it further refines the effective identity features. The experimental results conducted on multiple datasets demonstrate that the CVF-JSM model outperforms existing technologies in the realm of caricature-visual face recognition.