Causal HyperPrompter: A Framework for Unbiased Hyperspectral Camouflaged Object Tracking

Hyperspectral camouflaged object tracking remains a significant challenge due to the high similarity between objects and replicas in texture and color. Despite recent progress, the bias present in the tracker and the embedding token hinders the model training. Specifically, most methods rely on false-color three-channel images to fine-tune RGB-based trackers. However, it introduces a confounding effect within the RGB domain, potentially leading to harmful biases that misguide the model toward spurious correlations while neglecting the critical spectral discrimination inherent in hyperspectral images. Furthermore, current token-type embedding methods overlook the key correlations between templates and searches, ultimately confusing correlation and impairing tracking performance. To address these challenges, this paper proposes a new unbiased tracking framework named Causal HyperPrompter. It first introduces a structural causal model to disentangle and control exclusive causal factors during tracking, and incorporates a counterfactual intervention strategy to eliminate confounding variables and mitigate the bias inherited from RGB-based models. In addition, we present a novel token-type embedding module that integrates local spectral angle modeling to enhance the semantic link between template and search tokens, thereby improving the model's sensitivity to object localization. Lastly, to overcome the difficulty of manually initializing the bounding box and addressing data scarcity, we introduce a large-scale hyperspectral camouflaged object detection and tracking dataset, BihoT-130k, consisting of 130750 annotated frames across various camouflage scenes. Extensive experiments on multiple large-scale datasets illustrate the effectiveness of our proposed methods.