For years, scientists have been developing systems
to allow people who can’t speak to communicate with their minds.
These ‘brain computer interfaces’ attempt to translate patterns of
brain activity into words and sentences.
Back in 2021 this was the state of the art,
a set-up that allowed this participant to talk at a rate of 15 words per minute
with a vocabulary of 50 words.
Two years later, a pair of teams have made rapid progress.
The new studies show off systems that enable their two participants to speak at over 60 words
per minute, with comparatively fewer errors and larger vocabularies than ever before.
This research could be the groundwork needed to take brain computer interfaces like these out
of the lab and start developing practical and potentially life-changing technology.
This is Pat. She has ALS, or amyotrophic lateral sclerosis which impairs her speech and
facial movements and she’s been working with a team of neuroscientists based at
Stanford University to help develop their brain computer interface.
When Pat tries to speak, the neurons in her speech motor cortex activate.
This is the part of the brain that controls the muscles we use to speak.
A series of microelectrode arrays inserted into Pat’s brain
can read the activity of over 100 individual neurons.
These read-outs then go into a machine learning algorithm.
During training sessions, Pat tries to read out specific phrases...
and the algorithm learns to match the sounds she’s reading with the patterns of neuronal activity.
This data is then fed into a language model with a vocabulary of over 100,000 words
which figures out what words those sounds most likely represent.
"How do you like my music?"
After training, Pat could communicate at around 62 words per minute,
much faster than previous set-ups, and with a much larger vocabulary and fewer mistakes.
"Hello how are you"
The researchers say that using more electrodes, reading activity from more individual neurons,
could make their system more accurate - and provide even more information about
how different parts of the brain actually work.
Meanwhile a second team from the University of California San Francisco
are tackling this challenge with a slightly different approach.
Their system reads brain activity from the surface of the brain and, again, can use a combination of a
machine learning algorithm and a language model to decode that brain activity successfully into text
with a vocabulary of around a thousand words or so and at a rate of up to 78 words per minute.
But as well as producing text,
this system can produce the sounds of speech - spoken by this moving avatar…
"Do not be afraid to ask me questions"
This avatar isn’t just reading out the decoded text - instead another
algorithm has been trained to translate brain activity directly into sounds.
"Great to see you a den"
The idea is that this speech isn’t limited to certain words and could potentially allow
someone to vary the speed and tone of their speech in a more natural way.
"As soon as possible"
The team also experimented with making the avatar face respond to motor signals to
produce certain expressions.
Combining all these elements together could one day give people who
can’t speak the ability to communicate with much more nuance than is possible with text alone.
"What do you think of my artificial voice?"
Despite these advances, the two systems still have about a 25% error rate and
‘speak’ at around half the average speed for english speech. As well as that,
each of these systems has so far only been tested with one person, and we still don’t know
exactly what the safest and most accurate way of getting the most relevant brain signals might be.
Still, this leap forward in the speed, accuracy and vocabulary could be enough to show that a
mass produced product is feasible. It could be enough to start developing safe and practical
hardwear that’s long-lasting and easy for people to use in their every day lives.
The next question is what the people who need it actually want from these kinds of
systems and how best to turn scientific progress on brain computer interfaces like these
into usable technology that can improve the lives of people around the world.
*laughter*
"That's great, thank you Pat"