Mixed reality-based brain computer interface system using an adaptive bandpass filter: Application to remote control of mobile manipulator

Brain-computer interface (BCI) systems based on mixed reality (MR) have promising applications in assisting people with disabilities to control manipulators. Using MR glasses instead of a computer screen to display visual stimulator can effectively avoid frequent switching of attention between the visual stimulator and the manipulator. When the manipulator moves out of the sight of the subject, the subject may not be able to control it accurately. Our system uses Microsoft Hololens2 as the display device to synchronize the command matrix with a live view of the mobile manipulator's position, thus tracking the position in real-time. Another problem in previous studies is that they have good accuracy in trained subjects, however, the accuracy drops dramatically when faced with untrained subjects, suggesting poor generalization capabilities. In our study, an adaptive filtering method combined with convolutional neural networks (CNN) is proposed, which has few learning parameters and fast convergence, and can improve the generalization ability of the system in the face of untrained subjects. When faced with untrained subjects, the average accuracy of our method was 93.04%, and the average ITR was 20.96 bits/min. All subjects can successfully complete the grasping task without colliding with obstacles. The results show that the BCI system developed in this study has strong practicability and high research significance.