Brainsourcing is a novel technique that unifies the fields of crowdsourcing and brain-computer interfacing.
It allows people to collaborate on recognition tasks using only their brain signals evoked in response to a given information stimulus.
Brainsourcing builds upon the concept of crowdsourcing,
but instead of manual labor, it relies on contributions of individual people
by implicitly monitoring their brain activity while they encounter information-intensive tasks.
Crowdsourcing, more generally, is a paradigm in which ideas, data products, and intellectual work
are produced by a large group of individuals, unbounded by time or space.
In crowdsourcing, tasks are divided among individuals to achieve a collective result.
Crowdsourcing is a broad field, and tasks can be roughly divided into two categories: macrowork and microwork.
Macrowork often requires special skills and quite a bit of time to complete.
Examples of macrowork include open source software development and editing Wikipedia articles.
Microwork does not require specialized training or skills, and most tasks can be completed in a short amount of time.
Featurizing images and transcribing scanned text, as with google’s reCAPTCHA service, are examples of microwork.
For our experiment, we focus on microwork in the form of recognition tasks,
where users look for images matching a certain description.
Brain-computer interfacing is an interaction paradigm
driven by real-time neural activity collected from a user.
This brain activity can be recorded using a variety of methods.
One of the most popular, and the one used in this is experiment,
is electroencephalography, also known as EEG.
EEG is a technique to record electrical activity of the brain,
typically done by measuring changes in voltage at the surface of the scalp.
This electrical activity is related to neural processes occuring in the brain.
EEG data has a high temporal resolution, however its spatial resolution is quite poor.
So while EEG data can be used with a high degree of accuracy
to detect when a neural event has occurred
it cannot be as easily used to determine where it occurred.
Event-Related Potentials, commonly referred to as ERPs,
are a key feature of EEG data.
ERPs are time-locked voltage fluctuations
produced by the brain in response to some event,
such as viewing a face or moving a muscle.
ERPs consist of positive and negative components characterized by a variety of amplitudes and latencies.
To automatically interpret ERPs,
machine learning models can be trained to distinguish between ERPs produced by different events.
In fact, brain-computer interfaces that use EEG
often follow an ERP-based approach to facilitate user interaction
such as with BCI spelling programs.
Statistical techniques are then used to distinguish between the types of responses.
In this experiment,
we make use of the P300,
an ERP component associated with attention and memory.
Let’s say we have a test subject who will let us record some EEG.
We ask the subject to “look for faces of smiling people”
and show them faces of people smiling (target class) and not smiling (non-target class).
Now the ERPs produced by “smiling” faces
are distinguishable from the ERPs produced by faces that were not smiling.
This is indicated by the larger positive peak visible at around 300ms post-stimulus (the P300).
Stimuli of the target class, that is, with the feature of interest
will produce ERPs that contain a P300 component.
Stimuli not containing a feature of interest,
part of the class “non-target”,
will typically produce ERPs without a noticeable P300.
This effect is reliable enough where we can train a classifier
such that, given an ERP, we can determine the class of stimuli that produced it with a reasonable level of accuracy.
Brainsourcing takes this classification approach a step further.
The classification accuracy of a single brain-based model,
while better than random,
is still not high enough to compete with more conventional approaches.
Rather than simply relying upon the brain responses
of a single individual to determine classes of stimuli,
it’s possible to combine predictions derived from many individuals’ brains.
Using this approach, Brainsourcing achieves results that are significantly better than any individual model.
We will now provide a brief overview of how the brainsourcing pipeline is constructed.
Full details can be found within the paper.
First, we need a brainsourcing worker.
Okay.
The worker is placed in front of a computer screen and fitted with an EEG cap and electrodes.
They are then asked to complete a recognition task
where they look for faces that match a certain description,
such as having dark hair or a smile.
The worker does not need to provide any explicit feedback -
they simply sit back and observe
as faces are presented to them on screen while their scalp voltages are recorded.
In practice, it looks something like this:
After collecting enough training data from the worker,
a classifier model is trained using the ERPs produced by each stimulus.
When a worker is then presented with a new stimulus,
the ERP it produces is fed into the classifier model
which uses features present in the ERP to determine the class of the stimulus.
But remember -
we’re conducting Brainsourcing here.
So we need to repeat this procedure for multiple individuals
By training models for other brainsourcing workers
and combining many predictions for the same stimulus,
we can classify images with higher accuracy than any individual model.
To ensure the brainsourcing model performs better than individual models,
the results are analyzed in three ways.
First, by showing that the brain responses are reliable.
Second, by measuring the performance of correctly classified tasks.
Third, by investigating the changes in what the classifier learns when a crowd is used instead of individuals.
To show the brain responses are reliable,
we can plot the ERPs produced by the two classes of stimuli across all tasks and visually compare them.
As we can see,
there is a distinct difference between ERPs produced by target and non-target images,
and this difference is present for all tasks.
To measure the perfomance of correctly classified tasks,
we compare classification results
as data from more participants are used in the brainsourcing estimations.
In this case, we take a representative sample
of blond faces and darkhaired faces for task “blond”,
and visualize whether these stimuli were correctly classified by the brainsourcing model using their associated ERPs.
as the data from a greater number of participants is used in the brainsourcing model,
performance improves and the number of misclassified stimuli appears to decrease.
Analyzing this more empirically across a larger sample size using F1 score,
we find the same phenomenon holds true.
Further analysis of the brainsourced estimations
reveals a clear bimodal distribution
as data from more participants are added to the models.
This reflects the nature of the stimuli data,
and further suggests that the brainsourcing model is indeed learning something meaningful.
In this research,
we showed that human brain responses
can be effectively utilized in a crowdsourcing setting.
Combining predictions from just two participants already yields an F1 score of 0.77 across all tasks.
Combining predictions from twelve participants achieves an F1 score of 0.94 across all tasks.
This performance is high enough that it could be used in real-world situations.
Due to the sensitive nature of brain-based data,
it’s important to recognize potential ethical and privacy issues
that may arise from brainsourcing.
Widespread adoption of brain-computer interfaces is not yet a reality,
however, with continued improvements to existing technologies,
it could be in the near future.
When brain-computer interfaces are as common as the mouse, keyboard, or touchscreen,
anyone could become a brainsourcing worker.
This may lead to unethical labor or revenue models,
like replacing conventional advertisements with brainsourced microtasks.
EEG data can serve as a biometric identifier -
brainsourcing data that is improperly handled could reveal the identity of a worker.
Meanwhile,
governments, corporations, and criminal organizations
might use these systems in ways that violate an individual’s right to privacy and autonomy.
New forms of malware could also emerge,
where brain-computer interfaces are hacked to conduct subliminal probing for private information.
With widespread adoption of ethical design practices
and privacy-preserving techniques
within the brain-computer interfacing community,
we may be able to reduce the risks associated with this technology.