TY - JOUR
T1 - Heterogeneous Graph-Based Multimodal Brain Network Learning
AU - Shi, Gen
AU - Zhu, Yifan
AU - Liu, Wenjin
AU - Yao, Quanming
AU - Li, Xuesong
N1 - Publisher Copyright:
© 1989-2012 IEEE.
PY - 2025
Y1 - 2025
N2 - Graph neural networks (GNNs) provide powerful insights into brain neuroimaging technology from the view of graphical networks. However, most existing GNN-based models treat the brain connectome, derived from neuroimaging, as a homogeneous graph characterized by uniform node and edge types. In fact, emerging studies have reported and emphasized the significance of heterogeneity among human brain activities, especially between the two cerebral hemispheres. Thus, homogeneous-structured brain network-based graph methods are insufficient for modeling complicated cerebral activity states. To overcome this problem, we introduce a novel heterogeneous graph neural network (HeBrainGNN) for multimodal brain neuroimaging fusion learning. HeBrainGNN first conceptualizes the brain network as a heterogeneous graph with multiple types of nodes (representing the left and right hemispheres) and edges (categorizing intra- and interhemispheric interactions). We further develop a self-supervised pretraining strategy for this heterogeneous network to address the potential overfitting problem caused by the conflict between a large parameter size and a small medical data sample size. Empirical results show the superiority of the proposed model over other existing methods in brain-related disease prediction tasks. Ablation experiments show that our heterogeneous graph-based model attaches more importance to hemispheric connections that may be neglected due to their low strength by previous homogeneous graph models. Additional experiments reveal that our pretraining strategy not only addresses the challenge of limited labeled data but also significantly enhances accuracy, affirming the potential of our approach in advancing neuroimaging analysis.
AB - Graph neural networks (GNNs) provide powerful insights into brain neuroimaging technology from the view of graphical networks. However, most existing GNN-based models treat the brain connectome, derived from neuroimaging, as a homogeneous graph characterized by uniform node and edge types. In fact, emerging studies have reported and emphasized the significance of heterogeneity among human brain activities, especially between the two cerebral hemispheres. Thus, homogeneous-structured brain network-based graph methods are insufficient for modeling complicated cerebral activity states. To overcome this problem, we introduce a novel heterogeneous graph neural network (HeBrainGNN) for multimodal brain neuroimaging fusion learning. HeBrainGNN first conceptualizes the brain network as a heterogeneous graph with multiple types of nodes (representing the left and right hemispheres) and edges (categorizing intra- and interhemispheric interactions). We further develop a self-supervised pretraining strategy for this heterogeneous network to address the potential overfitting problem caused by the conflict between a large parameter size and a small medical data sample size. Empirical results show the superiority of the proposed model over other existing methods in brain-related disease prediction tasks. Ablation experiments show that our heterogeneous graph-based model attaches more importance to hemispheric connections that may be neglected due to their low strength by previous homogeneous graph models. Additional experiments reveal that our pretraining strategy not only addresses the challenge of limited labeled data but also significantly enhances accuracy, affirming the potential of our approach in advancing neuroimaging analysis.
KW - brain network
KW - heterogeneous graph neural network
KW - multimodal neuroimaging fusion
KW - pretraining strategy
UR - http://www.scopus.com/inward/record.url?scp=105005306969&partnerID=8YFLogxK
U2 - 10.1109/TKDE.2025.3569648
DO - 10.1109/TKDE.2025.3569648
M3 - Article
AN - SCOPUS:105005306969
SN - 1041-4347
JO - IEEE Transactions on Knowledge and Data Engineering
JF - IEEE Transactions on Knowledge and Data Engineering
ER -