面向功能性用户体验质量评估的脑网络构建方法

Objective The rapid development of multimedia technology enables emerging video services, such as bullet chatting video and virtual idol. Technical parameters from network and application layers affect user’s quality of experience(QoE). In addition, the QoE changes when various adjustable functional parameters of these video services are modified, which we call QoE influenced by functional parameters(functional QoE, fQoE). Changes in functional parameters influence human cognition and emotion, and thus, fQoE is almost entirely decided by human subjective perceptions. Inferring directly from parameter design, which is difficult, makes fQoE modeling challenging. The success of the video services depends entirely on user ratings, and understanding fQoE and the reason behind its generation is crucial for service providers. Studies using questionnaire and interview methods have been conducted to understand users’perceptions of functional parameters. However, subjects may be influenced by external criteria and social desirability, which potentially result in bias of the collected results. The above methods cannot perform the quantitative assessment of fQoE nor provide scientific evidence with interpretability. Electroencephalography(EEG)signals contain a wealth of information about brain activity, and EEG features can reveal brain network patterns during complex brain activity. Studies have used EEG as a powerful tool to assess the QoE influenced by technical parameters(tQoE)and uncovered relevant EEG single-electrode features, which revealed the correlation between tQoE and basic human perceptual functions and demonstrated the strong potential of EEG for fQoE assessment. However, fQoE may involve higher-order human cognitive functions that require interactions between multiple brain regions, such as social communication and emotion, whose complex relationships are difficult to be represented by single-electrode features. To address the limitations of the above studies, this paper presents an fQoE assessment model based on the EEG technology and investigates the neural mechanisms behind fQoE. Method First, an EEG dataset for fQoE assessment was constructed. To reduce the influence of subjects’personal preferences on the experimental results, we ensured that the stimulus materials contained five types of videos(auto-tune, technology, dance, film, and music). Different levels of fQoE were induced by changing the functional parameters, and the EEG data of subjects were collected simultaneously. Second, on top of single-electrode features, we additionally extracted multielectrode features(i. e., functional connectivity features)and fused both types in the form of graph. The EEG electrodes were represented as nodes on the graph, the single-electrode features as node features, and the functional connectivity as edges of the graph. The weights of edges represent the strength of functional connectivity, and were used in the comprehensive characterization of the user’s brain state when using video service. Finally, self-attention graph pooling mechanism was introduced to construct a brain-network construction model to identify fQoE levels. The graph pooling layer can enlarge the field of view to the whole graph structure during the training process, retain key nodes, and compose new graphs to render the model with the capability to capture key brain networks. We further explored the neurophysiological principles behind it and provided theoretical support for the improvement of emerging video services. Result With the bullet chatting video, a new type of video service, as an example, this paper explored the fQoE affected by the functional parameter of bullet chatting coverage and the neurophysiological principles behind it. The finding verified the scientific validity and feasibility of the method. Experiments revealed that the assessment method proposed in this paper achieved satisfactory results in the fQoE evaluation of multiple video types, with the best recognition accuracy of 86%(auto-tune), 80%(technology), 80% (dance), 82%(film), and 84%(music). Compared with existing machine learning and deep learning models, our method achieves the best recognition accuracy. The results on fQoE-related brain network analysis reveal that the number of brain connections in the frontal, parietal, and temporal lobes decreased, which indicates a attained fQoE for viewing bullet chat videos, i. e., a better viewing experience. This result also implies that functional parameters further lead to changes in the fQoE by affecting the human brain state. Specifically, the brain connection between the frontal and temporal lobes is related to speech information processing, the parietal lobe brain connection to visual information processing, the strength of frontal lobe brain connection to cognitive load levels, and its asymmetry to emotions and motivation. Conclusion In this study, we initially presented an EEG-based fQoE assessment model to evaluate the fQoE levels using a brain-network construction model based on a self-attention graph pooling mechanism and analyzed the neurophysiological rationale behind it. The assessment method introduced in this paper serves as a quantitative tool and theoretical basis from neurophysiology for the accurate assessment of fQoE and optimization of functional parameters of video services.