Hello, my name is Arnaud Delorme. When I started EEG research, I struggled with the best way
to conduct EEG experiments, and it took me years to do it right. I learned by making
mistakes and often wasted months of work. Based on my experience, these are 16 advices
for junior researchers for designing and conducting EEG experiments.
The first advice is about reviewing the literature. This one is obvious. You may have an idea
and just want to perform a quick experiment to test that idea. Unfortunately, when the
time comes to draft your report or article, you will have to review the literature to
replace your experiment in its context. You then realize that someone else has already
done the same experiment or done a similar one. Or you might discover that, in light
of the literature, you should have done your experiment differently. So it's always better
to review the literature before you start. In my team, most PhD students begin their
thesis by writing a review article on their dissertation topic to understand the context
in which they will work and perform experiments.
The second advice is about planning. You think planning is a waste of time. You simply want
to program the experiment and start collecting data, and see what happens in the brain. You
think you will figure out how to analyze the data later. More often than not, it doesn't
go well. After collecting data on a few participants, you realize there is a bug in your stimulus
presentation program or that you've made a conceptual mistake in the randomization procedure.
Or you might realize you have made a mistake only when the time comes to analyze the data
and publish the article. You should plan as much as possible.
The third advice is to make sure you understand that collecting EEG data is different than
running a psychophysics experiment and adding EEG. Maybe you are an expert in experimental
psychology, where participants perform psychophysics tasks or play games on computer screens. You
may think that you can simply add EEG to your psychophysics experiment, and figure out what
happens in the brain. In 95% of the cases, that's not going to work. The first reason
is controlling movements. If participants move, the EEG signal will be ruined by muscle
artifacts which are an order of magnitude larger than brain signals. And of course,
there are exceptions: there are researchers who specialize in collecting EEG data during
movement. However, this is not the place to start. Even if the participant is not moving
their body or their head, moving their eyes can be enough to render the EEG signal unusable.
It is why, in EEG experiments, we ask people to fixate the center of the screen and place
a fixation cross for them to focus on. Here are some extreme cases of trying to limit
movements. You could use a chin rest. I have even seen a paper where the experimenter give
participants small doses of curare to limit muscle contractions. I would not advice to
do that, but it illustrates the extreme methods people sometimes use to minimize muscle artifacts.
Also, when collecting EEG data, because the EEG signal is noisy, you need multiple repetitions
of the same condition. When collecting brain responses to experimental stimuli, the rule
of thumb is to present each stimulus type at least 30 times. And, in general, it's better
to present them at least 100 times.
The fourth advice is not to be too ambitious. You might want to do a complex experiment
and answer a variety of hypotheses. The experiment lasts several hours, contains multiple tasks
and a dozen of a questionnaire to collect psychometric data that you are planning to
correlate with EEG activity. You decide to collect EEG, heart data, and eye movements.
Not only is it challenging to set up this type of complex experiment, but then you might
get lost in the amount of data and the type of analysis you can do. I have been there.
Eventually, you will likely focus on testing one hypothesis and ignore 50% of the data,
so it might actually be better to perform simpler experiments. Simpler is often better.
However, it is also possible to have complex data and answer simple questions. For example,
here, you can have a participant in a natural environment with eye-tracking and a forward-facing
camera, so we know what object the participant is looking at. So the data is very rich and
complex. But we can still ask precise questions. For example, we can look at the brain response
to faces versus non faces. So complex set up is OK, as long as you have limited and
realistic goals.
The fifth advice is to make sure you properly randomize you data. Failing to balance participants
in different groups and randomize trials properly is probably one of the most common mistakes
in EEG research. For example, participants could learn to anticipate which type of trial
is coming if you do not randomize your trials properly. Or if you present the same type
of trials in blocks as you would do for fMRI, this could bias your results. This article
showed that most machine learning approach on block design EEG data is worthless, because
stimuli of the same type are shown in sequence, and not in randomized order.
The sixth advice is to remember to do power calculations. In general, when performing
experiments, you want to make inferences on a target population, such as human adults
for example. The number of participants in your experiment depends on several factors
such as your threshold for significance, how many variables in your design, the number
of hypotheses you are testing, and the amplitude of the effect you expect. Based on this information,
you can use tools to calculate the minimum number of participants you will need. I included
one such online tool in the description. In general, you will end up with 16 to 32 participants.
But it's always good to do your due diligence, and you may use this information in the method
of your paper to justify the number of participants. For example, a common mistake is to have too
many factors in your design and not enough participants. For example, when presenting
visual stimuli, you might want to test how the shape, color, depth, transparency, texture,
background, brightness, contract, etc... influence cognition and the interactions between all
these factors. This is asking a lot questions at once and requires more participants in
your experiment.
The seventh advice is to ask for ethical approval. One reason to obtain ethical approval is that
most journals require it for publication. Obtaining ethical approval means submitting
a document to your research institute or a private entity specializing in these matters.
In this document, you describe the experiment, how you will obtain informed consent from
participants and preserve their privacy and rights. It also involves anonymizing the data.
I personally advice to only store identifiable data on the consent form that the subject
sign, and assign a subject code at this stage. Then these consent forms can be stored physically
or digitally in a secure location, but not with the data of course.
The eight advice is to pre-register your experiment. Pre-registering an experiment is different
than asking for ethical approval, although the documents are similar. Preregistration
means that you're going to describe the method you're using, how you're going to analyze
the data, and report your results, even before you have started collecting data.
The main reason to preregister is to improve scientific rigor and avoid reporting false
positive results. When you are not preregistering your experiment, you may end up performing
different analyses than the ones you had originally planned. You can still do that when you preregister,
but at least it is made explicit. The problem with exploratory analyses is that, if you
do many of them, one might turn out to be significant just by chance. Unfortunately
this is a common practice, and the main reason is that journals tend to favor positive and
exceptional results. A large number of researchers analyze their data using different angles,
and then write the paper that makes the most sense based on their results. The system rewards
this behavior. To resist the temptation, you can pre-register your experiment. The other
advantage of pre-registering your experiment is that it will force you to think deeply
on how you will analyze the data. Often, this will lead you to do minor adjustment to your
experimental design, change stimulus presentation latencies, the randomization procedure, or
make sure you have all demographic information from your participants. Some journals, for
example, have a new model where you publish a first paper with your method first, then
collect data, and publish the results in a second paper. PLOS One just started this model.
The ninth advice is to program the experiment yourself or at least check the code, even
though you might not be a programmer. There are a lot of details that go into designing
an experiment. You want to make sure the inter-stimulus interval is correct, that the randomization
is properly implemented, that there is no unwanted buffer delays. For programming experiments,
I personally advice using the psychophysics toolbox which is free and open source. It
has a large user base and you might be able to find people to help you by asking questions
on forums.
The tenth advice is not to trust your equipment and your software too much. EEG and psychophysics
are niche markets. There might be a couple of hundreds of researchers using the same
EEG hardware as you are. Given that there are bugs in commercial software packages used
by millions of people, you can imagine what it might be for software used by hundreds
of people. I recommend using open-source software packages because many other researchers have
already looked at the code and checked that it is doing what it is supposed to do. And,
of course, you should check yourself. In particular, for EEG data acquisition, I recommend testing
the synchronization of the EEG equipment with stimuli presentation and event markers. In
event-related EEG, ideally, we need one millisecond precision for all these events, although,
in practice, we rarely get more than a few millisecond precision. Problems most often
occur in two places. One is the synchronization of the event markers with the EEG acquisition.
There might be a delay or a jitter you are not controlling for. The second common problem
is synchronizing the stimulus presentation with the event markers because of various
audio or video buffers or simply because of how you have written your experimental script.
For video, for example, most screens refresh at 60 Hz, which means one frame every 17 millisecond,
which is quite slow by EEG standard. It is better to use screens with higher refresh
rate and also check that the image is shown all at once on the screen with might not be
the case for LCD screens. Testing equipment synchronization means using an oscilloscope,
microphones, and photodiodes to compare actual stimulus latencies versus event markers latencies.
Every EEG and psychophysics researcher goes through this painful process at least once.
I put some links in the description for more details.
The eleventh advice is to scan your EEG electrode positions and when possible collect anatomical
MRIs for EEG source localization. Of course, first, you want to select an EEG cap of the
right size for the participant and position your EEG cap properly. Then, scanning electrodes
is simpler than it sounds. You need an iPad Air equipped with a special 3D camera. Scanning
takes about 30 second. Then, all the software to extract electrode position from the 3-D
model is available open source. When you have the electrode position, you can perform better
alignment between the caps of different individuals. Acquiring MRIs is also a great way to up your
game in EEG analysis. Acquiring anatomical MRIs takes about 15 minutes per participants,
so it is a minor added time, especially if you have an MRI close-by. Of course, if your
participants already have MRIs acquired for a previous experiment, you can reuse them.
Once you have the MRI and scanned electrode position, there are pipelines to perform source
localization. This can greatly improve the quality of your EEG source analysis and it
is definitely something to consider even if you are not planning to do it as a primary
analysis.
The 12th advice is to acquire pilot data. Even if you've planned your experiment exceptionally
well, there are problems you will only encounter when you start collecting data. It is impossible
to think of everything. So you want to record at least one pilot participant, which is a
participant that you're not going to process. You can not only figure out what goes wrong,
but you can also get some feedback from the participants. This is a recent pilot flight
from SpaceX that went wrong. But this was part of eventually making it right.
The 13th advice is to check data quality and properly optimize the EEG signal. As we commonly
say, garbage in, garbage out. Optimizing the EEG signal means of course checking the electrode
impedance. If you cannot check electrode impedance, check by eye that the signal from each electrode
is acceptable. You should give special attention to the reference electrode. Try tweaking the
bad electrodes until the signal is acceptable. Getting rid of artifacts or bad electrodes
will be more than twice the work when you process the data. You should also check for
equipment in the room that can generate electrical noise, such as air conditioning, large backup
batteries, light source, or electrical outlets. EMF meters might help you find electromagnetic
fields that can contaminate the signal. It is not necessary to have an electromagnetic
shielded room, but you must still try to minimize external noises sources. Everybody has their
own trick. What I personally like to do, is to turn off the electricity, use a battery
powered EEG system, and run my experiment from a battery powered laptop with the subject
having no direct contact with the recording or presentation equipment. Other people use
other tricks. Recording in colder temperatures also help as shown in this graph here. Temperature
above 26 degree centigrade and high air humidity degrades the signal especially when electrode
impedances are high. I put the link to the paper in the description.
The 14th advice is to take notes when collecting data. You are now collecting data, but some
things can go wrong even when you have planned everything. So it's good to have a notebook
where you can write whatever happens. For example, you might have swapped EEG electrodes
by mistake, or the participant was scratching his head. Make sure you write the participant
ID on your note and, once you're done collecting data, take a picture with your phone and save
the picture along with the data so it's easy to find.
The 15th advice is about organizing data once you have collected it. How do you save the
data? Instead of using a random file naming convention you might have come up with, I
strongly advice using BIDS, the brain imaging data structure, where files and folders are
named based on participant ID, session, block number, task name etc... BIDS makes sure you
are using the same prefix for all the files of a given participant. The other advantage
is that, if your data is formatted in BIDS, you can directly apply pre-processing pipelines.
If your dafa is properly anonymized using BIDS when you acquire it, you can make a copy
on the cloud so it can easily be backed up, and shared with others. I made a video on
BIDS and EEGLAB automated processing pipelines, which I put in the description.
The final advice is not to change the experiment before you finish collecting data. That's
kind of obvious, but we have all done it at least once. For example, you realize that
some participants have a hard time seeing a briefly flashed visual stimulus, and you
want to increase the stimulus size as shown here. It's just a minor change, and you think
it's not going to have a dramatic impact on the EEG, but you might be wrong. I would strongly
advice against doing that, even if you think you are planning to consider this change as
a factor during the data analysis. And of course, you also don't want to stop the experiment
early because you have already started to analyze the data and realize that you have
enough participants to reach a significant effect. This is called optional stopping,
and this is dishonest science. Because you perform multiple statistical tests, one after
each participant, one of these tests could turn out significant just by chance.
So these were fifteen advice for collecting EEG data. I have also put a link in a description
about a special issue on best practices in EEG and MEG research. Good brain hacking.