A Novel Graph Wavelet Model for Brain Multi-scale Activational-Connectional Feature Fusion

For the field of cognitive neuroscience and medical image analysis, feature fusion of multimodality from fMRI data is a significant yet challenging problem, and it usually requires brain data from different imaging methods which often leads to the result deviation caused by registration problems, and cannot make full use of data information. In addition, most of them emphasize on single scale spatial of brain, which omits lots of potentially available information, while human brain is multiscale in character. To solve these problems and obtain more useful information from single modality image, we introduced the method of graph signal wavelet transform. It could bond latent graph structure and signal constituted by the value in each vertex on graph, which possess the advantage of fusing activation (signal on graph) and connection information (graph structure) of brain. Besides, the property of multi-scale in wavelet transform could contribute to extracting multi-scale information of brain. Inspired with that, in this paper, we proposed a novel Graph Signal Wavelet Multi-Scale (GSWM) feature construction framework, for fusing multi-scale information extracted from both functional activation and underlying functional connection of brain, to retain more comprehensive information only using fMRI data. The results showed that the multi-scale features also catch the tendency of changes among different scales information, which reflects the cognitive process. In addition, with the multiple task fMRI data from the Human Connectome Project (HCP), the prediction capability of the GSWM features showed its overwhelming advantage in feature fusion and further brain states decoding.