TY - JOUR

T1 - Multimodal magnetic resonance image and electroencephalogram constrained fusion algorithm using deep learning

AU - Li, Xiaofeng

AU - Huang, Heyan

N1 - Publisher Copyright:
© 2021, The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.

PY - 2021

Y1 - 2021

N2 - Traditional fusion algorithm is prone to skipping the multimodal attribute of the magnetic resonance image (MRI), lack of consideration of the spatial information of electroencephalogram (EEG) signal, resulting in the ineffective fusion of magnetic resonance image and EEG signal. Thus, constraint fusion algorithm of the multimodal magnetic resonance image and EEG signal using deep learning is proposed. First, the magnetic resonance image and EEG signal are simultaneously acquired for preprocessing. Secondly, the three modal magnetic resonance image features of functional Magnetic Resonance Imaging(fMRI), structural Magnetic Resonance Imaging(sMRI) and Diffusion Tensor Imaging are extracted, respectively, to analyze different modal images by multi-index features fusion. Then, different neural network channels use different coding methods, focusing on the construction of local channel convolutional neural network(CNN) model based on Cross Stage Partial Network transformation, coding network and recognition network design. Finally, the inverse problem of EEG signal is solved by the twomey standardized algorithm, completing the fusion algorithm design. The results show that the CNN model of local channel renders good training effect, the signal preprocessing of the proposed algorithm has obvious effect, the extraction accuracy of magnetic resonance image features of different modals is high, the MI value of image fusion is as high as 0.92, and the image fusion precision is high, and the time consumption is much lower than other algorithms.

AB - Traditional fusion algorithm is prone to skipping the multimodal attribute of the magnetic resonance image (MRI), lack of consideration of the spatial information of electroencephalogram (EEG) signal, resulting in the ineffective fusion of magnetic resonance image and EEG signal. Thus, constraint fusion algorithm of the multimodal magnetic resonance image and EEG signal using deep learning is proposed. First, the magnetic resonance image and EEG signal are simultaneously acquired for preprocessing. Secondly, the three modal magnetic resonance image features of functional Magnetic Resonance Imaging(fMRI), structural Magnetic Resonance Imaging(sMRI) and Diffusion Tensor Imaging are extracted, respectively, to analyze different modal images by multi-index features fusion. Then, different neural network channels use different coding methods, focusing on the construction of local channel convolutional neural network(CNN) model based on Cross Stage Partial Network transformation, coding network and recognition network design. Finally, the inverse problem of EEG signal is solved by the twomey standardized algorithm, completing the fusion algorithm design. The results show that the CNN model of local channel renders good training effect, the signal preprocessing of the proposed algorithm has obvious effect, the extraction accuracy of magnetic resonance image features of different modals is high, the MI value of image fusion is as high as 0.92, and the image fusion precision is high, and the time consumption is much lower than other algorithms.

KW - CNN of local channel

KW - Deep learning

KW - Fusion

KW - Inverse problem solving

KW - Magnetic resonance image

KW - Multimodal

KW - ROI region

UR - http://www.scopus.com/inward/record.url?scp=85120312173&partnerID=8YFLogxK

U2 - 10.1007/s00500-021-06574-8

DO - 10.1007/s00500-021-06574-8

M3 - Article

AN - SCOPUS:85120312173

SN - 1432-7643

JO - Soft Computing

JF - Soft Computing

ER -