The human senses are incredible.
Eyes that see over a million different colours,
ears that can detect a staggering range of frequencies and amplitudes,
skin sensitive to minute changes in pressure and temperature.
But none of these holds a candle to the
most powerful part of our sensory system:
the brain.
Locked in the darkness of the skull, the brain doesn't sense anything directly;
instead, it performs the most impressive feat of data analysis imaginable,
receiving electrical impulses from all over the body,
decoding their various meanings,
and combining them to create the rich, multi-modal experience we call reality.
This isn't always how things work out, though:
sometimes a connection between sense-organ and brain gets damaged,
or never fires up at all, and a sense is weakened or lost.
As a result, we have a long history
of finding ways to swap out one sense for another.
But in the last few decades,
technological advances have supercharged this relatively simple idea,
opening up new possibilities in the field of "sensory substitution".
Take the BrainPort.
This device, for the blind or partially sighted,
connects a camera to a small plastic patch,
which is placed on the user’s tongue.
The patch contains hundreds of electrodes,
which ‘draw’ what the camera sees using electrical pulses:
strong for bright spots, weaker for dark.
Initially this feels like bubbles in a fizzy drink,
but gradually the pulses begin to register as images,
allowing the user to see through their tongue.
Now... how is that even possible?
The key is neuroplasticity, the brain’s ability to rewire itself.
Ordinarily, signals travel from sense organs along dedicated nerves
to specific parts of the brain;
so if an organ or nerve is damaged, the signals can’t get through,
and the sense is lost.
But if a similar pattern of signals can reach it from another entrance,
the brain will work out where they need to go,
and get them there by a different route.
In principle, it doesn’t matter what kind of signals they are,
or how they enter:
the BrainPort can just as easily be connected to an accelerometer
– the device that tells your phone which way up it’s being held –
to help people with inner-ear damage regain their balance.
In the words of the device's inventor, Paul Bach-y-Rita:
“Just give the brain the information and it will figure it out.”
This is more than just a fun little lab experiment:
BrainPort’s potential
has been demonstrated by blind mountaineer Erik Weihenmayer,
who's used it on climbing expeditions.
It isn't perfect: BrainPort vision is grainy and grey-scale – and obviously,
while you’re using it, you can’t talk or eat.
But it is proof that one sense can be patched through another.
Other devices approach this concept in different ways,
and with more discreet wearability –
for instance, vibrating wristbands:
NeoSensory’s Buzz can clarify speech
or ambient sounds for the deaf and hard of hearing,
or train the brain to overcome tinnitus.
Wearwork’s Wayband nudges the
blind and visually impaired back onto their desired path,
while The vOICe uses the camera and speakers inside
smart glasses to translate images into beeps and squeaks,
making it possible to “see through sound.”
What these have in common is that they all compensate
for lost or diminished senses,
but this technology could go a lot further than that.
Our human senses – great though they are –
only represent a tiny fraction of the world around us.
Many animals have senses attuned to things we can scarcely imagine:
ultrasounds; UV light; black-body radiation;
even the earth’s magnetic field!
We can't sense any of those, but we have built technology that can,
and using those devices as inputs, we could enhance our
biologically evolved senses with technologically developed superpowers.
And why stop there?
Once we recognise that this is fundamentally a matter of
translating digital data into a form our brains can process,
there’s no reason to limit that input data at all.
David Eagleman, professor of neuroscience and founder of NeoSensory,
has proposed that vibrotactile vests
could be used to sense huge amounts of information – enabling
astronauts to monitor space stations;
pilots to simplify the array of dials and gauges in their cockpits;
surgeons to get direct feedback on their patients’ vital signs;
meteorologists to forecast the weather by feeling it in their bones.
The only limit is our imagination.
In the Information Age, millions of terabytes of data are produced every day –
sensory substitution could represent
a transformative way of using and understanding it.
But before we get carried away,
let’s remember that there’s still lots to be done:
to raise the performance of devices, and to lower their price-tags.
A vibrotactile wristband might cost the same as a high-end smart watch,
but that still marks it out as a luxury item – and any technology
with this much scope to improve people's lives shouldn’t be a luxury.
Sensory substitution has the capacity to extend the horizons of human perception,
but its greatest benefit should be to help the greatest number of people
Overcome obstacles to socialising, working, and getting around
in a world that all too often doesn’t take account of their needs.