A method for synchronized use of EEG and eye tracking in fully immersive VR

In recent years, there has been increasing interest in using steady-state visual evoked potential (SSVEP) in brain-computer interface (BCI) systems. However, several aspects of current SSVEP-based BCI systems need improvement, specifically in relation to speed, user variation and ease of use. With these improvements in mind, this paper presents an online multi-channel SSVEP-based BCI system using a canonical correlation analysis (CCA) method for extraction of frequency information associated with the SSVEP. The key parameters, channel location, window length and the number of harmonics, are investigated using offline data, and the result used to guide the design of the online system. An SSVEP-based BCI system with six targets, which use nine channel locations in the occipital and parietal lobes, a window length of 2 s and the first harmonic, is used for online testing on 12 subjects. The results show that the proposed BCI system has a high performance, achieving an average accuracy of 95.3% and an information transfer rate of 58 +/- 9.6 bit min(-1). The positive characteristics of the proposed system are that channel selection and parameter optimization are not required, the possible use of harmonic frequencies, low user variation and easy setup.

A thorough understanding of the EEG signal and its measurement is necessary to produce high quality data and to draw accurate conclusions from those data. However, publications that discuss relevant topics are written for divergent audiences with specific levels of expertise: explanations are either at an abstract level that leaves readers with a fuzzy understanding of the electrophysiology involved, or are at a technical level that requires mastery of the relevant physics to understand. A clear, comprehensive review of the origin and measurement of EEG that bridges these high and low levels of explanation fills a critical gap in the literature and is necessary for promoting better research practices and peer review. The present paper addresses the neurophysiological source of EEG, propagation of the EEG signal, technical aspects of EEG measurement, and implications for interpretation of EEG data.

To increase the ability of brain-machine interfaces (BMIs) to control advanced prostheses such as the modular prosthetic limb (MPL), we are developing a novel system: the Hybrid Augmented Reality Multimodal Operation Neural Integration Environment (HARMONIE). This system utilizes hybrid input, supervisory control, and intelligent robotics to allow users to identify an object (via eye tracking and computer vision) and initiate (via brain-control) a semi-autonomous reach-grasp-and-drop of the object by the MPL. Sequential iterations of HARMONIE were tested in two pilot subjects implanted with electrocorticographic (ECoG) and depth electrodes within motor areas. The subjects performed the complex task in 71.4% (20/28) and 67.7% (21/31) of trials after minimal training. Balanced accuracy for detecting movements was 91.1% and 92.9%, significantly greater than chance accuracies (p < 0.05). After BMI-based initiation, the MPL completed the entire task 100% (one object) and 70% (three objects) of the time. The MPL took approximately 12.2 s for task completion after system improvements implemented for the second subject. Our hybrid-BMI design prevented all but one baseline false positive from initiating the system. The novel approach demonstrated in this proof-of-principle study, using hybrid input, supervisory control, and intelligent robotics, addresses limitations of current BMIs.