Comparison of feature selection methods based on discrimination and reliability for fMRI decoding analysis

Background: Feature selection is a crucial step in the machine learning methods that are currently used to assist with decoding brain states from fMRI data. This step can be based on either feature discrimination or feature reliability, but there is no clear evidence indicating which method is more suitable for fMRI data. Methods: We used ANOVA and Kendall's concordance coefficient as proxies for the two kinds of feature selection criteria. The performances of both methods were compared using different subject and feature numbers. The study included 987 subjects from the Human Connectome Project (HCP). Results: Classification performance suggested that features based on discrimination were more capable of distinguishing between various brain states for any number of subjects or extracted features. In addition, reliability-based features were always more stable than other features, and these properties (discernment and stability) of features, to some degree, related to the number of subjects and features. Furthermore, when the number of extracted features increased, the feature distributions also gradually extended from occipital lobe to more association regions of the brain. Conclusion: The results from this study provide empirical guides for feature selection for the prediction of individual brain states.