A Convolutional Neural Network Interpretable Framework for Human Ventral Visual Pathway Representation

Mufan Xue, Xinyu Wu, Jinlong Li, Xuesong Li, Guoyuan Yang*

*此作品的通讯作者

科研成果: 期刊稿件会议文章同行评审

摘要

Recently, convolutional neural networks (CNNs) have become the best quantitative encoding models for capturing neural activity and hierarchical structure in the ventral visual pathway. However, the weak interpretability of these black-box models hinders their ability to reveal visual representational encoding mechanisms. Here, we propose a convolutional neural network interpretable framework (CNN-IF) aimed at providing a transparent interpretable encoding model for the ventral visual pathway. First, we adapt the feature-weighted receptive field framework to train two high-performing ventral visual pathway encoding models using large-scale functional Magnetic Resonance Imaging (fMRI) in both goal-driven and data-driven approaches. We find that network layer-wise predictions align with the functional hierarchy of the ventral visual pathway. Then, we correspond feature units to voxel units in the brain and successfully quantify the alignment between voxel responses and visual concepts. Finally, we conduct Network Dissection along the ventral visual pathway including the fusiform face area (FFA), and discover variations related to the visual concept of ‘person’. Our results demonstrate the CNN-IF provides a new perspective for understanding encoding mechanisms in the human ventral visual pathway, and the combination of ante-hoc interpretable structure and post-hoc interpretable approaches can achieve fine-grained voxel-wise correspondence between model and brain. The source code is available at: https://github.com/BITYangLab/CNN-IF.

源语言英语
页(从-至)6413-6421
页数9
期刊Proceedings of the AAAI Conference on Artificial Intelligence
38
6
DOI
出版状态已出版 - 25 3月 2024
活动38th AAAI Conference on Artificial Intelligence, AAAI 2024 - Vancouver, 加拿大
期限: 20 2月 202427 2月 2024

指纹

探究 'A Convolutional Neural Network Interpretable Framework for Human Ventral Visual Pathway Representation' 的科研主题。它们共同构成独一无二的指纹。

引用此

Xue, M., Wu, X., Li, J., Li, X., & Yang, G. (2024). A Convolutional Neural Network Interpretable Framework for Human Ventral Visual Pathway Representation. Proceedings of the AAAI Conference on Artificial Intelligence, 38(6), 6413-6421. https://doi.org/10.1609/aaai.v38i6.28461