Hi, in this paper we present we’re creating a new enabling UI technology
to advance fNIRS BCI for future researchers, and that we see fNIRS BCI as a promising form
of future human-computer interaction. a brain the computer interface is a communication system
that allows the real-time interaction between the human brain and external devices. Traditionally
human-computer interaction relies on keyboard, mouse and a touchpad. in contrast,
a BCI system aims to achieve the interaction implicitly via the user's brain activity.
functional near-infrared spectroscopy (fNIRS) measures the changes in oxy and deoxy hemoglobin
in the brain. It provides a convenient real-time input for BCI applications. fNIRS
shows increasing promise to enable effective brain-computer interfaces for a wide range of
users. In this paper, our goal is to classify the subject's mental workload using fNIRS data
Image on the left is our fNIRS machine and on the right is the headband currently we are using it is
quick, easy to put on and no EEG grease compared with EEG, fNIRS has a higher spatial resolution.
in this paper, we release a new multi-subject fNIRS data set along with other information,
like demographic and contextual information,
cognitive task performance records, and a post experiment interviews. We propose
a 3-phase machine learning based pipeline to improve classification of cognitive workload.
we open-source our software to provide an easily accessible system for other researchers
Ultimately, our dataset, code, and methodology can allow future researchers to explore new methods
and use cases for BCI workload classification and improve generalization to new subjects
we use the N back task to trigger a different level of short-term memory
mental workload from the subject. N back is a well-established task
in experimental psychology, and which we believe closely correlative with many
UI tasks. In the n back task, we presented the subject with a series of stimuli and asked the
subject to compare the current stimulus to the stimulus shown on n steps previously.
An example of the data is showing on the right top.
we further processed the subject's data into multi windows of two seconds each, and our model needs
to classify which n back task this subject is currently performing giving the windows data
in terms of modeling, the problem is challenging from several aspects:
first, we have only a small amount of participants. Secondly for each subject
we have limited data. Furthermore there's a substantial cross-subject
variation, which is reported in past literature and also observed from our past experiments
To build a better-personalized model for each subject,
we propose a data augmentation plus domain adaptation pipeline.
Our pipeline includes 3 phases. Say we want to build a model for subject i,
then in phase 1, we will use all other subjects trained data to create synthetic data by itself.
We train a model on this synthetic data. This one is moderated by the need of data augmentation.
we will then use the learned weight to initialize the phase 2 model.
in phase 2, we train a model with the real observed data from all other subjects. Phase 2
is inspired by the fact that the human brain shares some similarities.
We then use the learned weights from phase 2 to initialize the weights for phase 3. in phase 3
we fine-tune on the subject's trained data and evaluated the performance of subjects' test data.
Phase 3 is inspired by the fact that despite
sharing some similarities the human brain also has considerable differences.
Over three months in early 2021, 27 healthy individuals have participated in the study.
We deemed 15 of these studies eligible for this work
based on their behavior during the experiment session. Here are the results for binary
classification of the embed tasks given two-second segments of fNIRS measurements.
We report average accuracy across 15 subjects. A system that guesses at random would have a 50
accuracy. Our 3-phase approach has an average accuracy of 71 percent outperforming the compared
baseline. We further found that each of the phases contribute to the model performance
reducing individual calibration efforts has practical implications
for real-time BCI systems. Therefore, we further investigate how well is our approach at reducing
individual calibration efforts. This figure shows the performance versus the percentage of data used
for the target subjects for binary classification. We found that our approach using only 50
of the subject's training data consistently outperform traditional approaches like logistic
regression and random forest training with full size of the subject's training data.
To conclude, we provide a new fNIRS dataset collected using the rigorous procedure from
subjects performing a standard impact task and show how it can be used to improve performance
for future systems. We developed a new machine learning approach to process and utilize the
fNIRS data. We show from our experiment that our proposed 3 phase machine learning pipeline
significantly improved n back classification performance over several established baselines.
Moreover, even with a reduced amount of per-user training data, our approach still outperforms
baseline models trained with all available target subjects training data, this shows the potential
of reducing individual calibration efforts when deploying BCI applications in the future.
We hope that our new dataset, machine learning method and tools will remove barriers that have
prevented a wider range of researchers from using fNIRS-based BCI and facilitate the
development of a new generation of powerful and easy-to-use brain-computer interfaces.
Currently, there is no large benchmark of news datasets.
As an ongoing effort, we are creating a large scale fNIRS dataset,
with recommended protocols on how to use the dataset
for future researchers who work in the intersection of machine learning and BCI.
We thanks all our friends and colleagues for their help and collaboration.