BrainX: A Universal Brain Decoding Framework with Feature Disentanglement and Neuro-Geometric Representation Learning

Decoding visual stimuli from human brain activity is a fundamental challenge in cognitive neuroscience and neuroimaging. While recent advances in deep learning have significantly improved the performance of fMRI-to-image decoding, most existing methods overlook the issue of inter-subject variability in fMRI data, which leads to poor generalization across subjects. Current approaches often rely on partially shared model architectures that offer limited generalization and still require subject-specific components, restricting their applicability to unseen subjects. To address this limitation, we propose BrainX, a universal brain decoding framework that constructs a unified fMRI encoder and image generator to achieve subject-agnostic modeling. Specifically, we introduce a feature disentanglement mechanism that extracts subject-shared features from the fMRI embeddings, which are then fed into the image generator to reconstruct visual stimuli. This design eliminates the need for subject-specific models and significantly enhances cross-subject generalization. Additionally, we develop a neuro-geometric fMRI representation learning method that projects 3D cortical structures onto a 2D surface space, effectively mitigating the inaccuracies caused by imprecise geodesic distance estimation in 3D Euclidean space. Extensive experiments on the Natural Scenes Dataset (NSD) demonstrate that BrainX consistently outperforms existing state-of-the-art methods across three decoding settings: within-subject, cross-subject with finetuning, and cross-subject without finetuning.