it would be no surprise if I told you that you see with your eyes and hear with your ears but
what if those two senses were switched what if we saw with our ears and heard with our eyes
now that may be silly and mirrors would need redesigning so would headphones and driving
would become probably impossible but it's not that far off from reality consider for example
a really crowded party people everywhere talking loud noises coming from the kitchen a dog barking
and you're trying to have a conversation with someone who's maybe four feet away from you
now while it might be difficult for you to hear them you're unconsciously doing something
with your eyes that is allowing you to understand them this is an example of hearing with your eyes
before we delve into what we can hear and how we can see what we hear we first need to understand
how we hear and see in the first place and that goes back to our brain the brain is the centre for
control of our body our thoughts our feelings and some people even believe our consciousness
the brain isn't only for thinking and moving though it's also important for interacting with
our environment you interact with your environment that is where you're sitting or standing right now
using our senses our sight hearing feeling smelling and tasting and while I’m not saying that
you're tasting the floor you're standing on right now we do use quite a few of our senses at any
given time just like you can tell toast is burning with a combination of hearing the fire alarm and
smelling the burning toast the brain loves using multiple inputs of senses to get a real feel
and understanding of the environment that it's in and the most common example of this is seeing and
hearing if you have a conversation with someone your brain is taking in both the sound of the
words and the shape of the mouth of the person that's talking and when we combine that sound
or audio and sight that is visual we like to call it audio visual integration and that basically
just means that we combine the information of seeing and hearing this is how we get back to
hearing with our eyes but the best way to see what I’m talking about is of course to experience it as
you are right now watching and listening to me but there's a fun little trick that we can do
to show you just how manipulatable our hearing really is I will now play for you two videos based
on an experiment by McGurk and McDonald called the McGurk effect one video after another will be me
pronouncing a word in a certain way my mouth shape will change between the two videos but the audio
should stay the same and you'll find that when staring at my mouth and looking at the
shape that my mouth makes you can manipulate what you hear so we'll play the video now
when watching that video you may have found that depending on the video you were hearing different
words even though I told you the audio was staying the same and this is a bit of a trick but
it's about as close as we can get to hearing with our eyes so what's happening in your brain at that
moment and why does it matter are we faced with auditory illusions like that every day
well more than you think your ears take in the sound and turn that sound into signals in the form
of electrical activity kind of like a computer and this is directed from the ear through the
auditory nerves into the auditory centres of the brain the temporal lobes among other regions but
how is it that we can change what we hear based on what we see well we first need to think about
how we see it all the visual information that is the video that you're watching right now it goes
in through the eyes through the optic nerve into the optic centre of the brain the occipital lobe
at the back of my head and yours too the reason I say region and the reason I’m saying among other
regions is because that the brain likes to put specific senses into specific areas of the brain
well your motor control might be primarily at the top of your head hearing on the sides and
vision at the back there still needs to be some interconnection between those groups just like
the temporal lobe unfortunately aren't too close the information that's coming in needs to be
combined so that we can understand how one might influence the other this is going back to what we
said earlier about audio visual integration using this example we can now better understand how
those two videos may have appeared different but were actually the same audio the audit
auditory and visual information simultaneously give you more information than just one or the
other would on its own that's why although we were playing the same auditory information
when the visual changed it changed what you've heard this work is in the realm of my work at the
university of Birmingham for my master's project in brain imaging and cognitive neuroscience
under the supervision of dr Hyojin Park we're looking to use magnetoencephalography or mig
for short to understand how the brain integrates visual and auditory stimuli
magnetoencephalography is a type of brain imaging imaging technique that uses magnet waves to
understand the activity that occurs in the brain as the result of electrical current you may recall
as I mentioned that the brain turns the visual and auditory signals into sort of electrical currents
or electrical signals much like a computer does and these currents result in waves particularly
magnetic waves magneto encephalography uses extremely fine-tuned sensors to catch the
magnetic waves that the brains are producing and turn it into a visualization map of how strong the
waves are and in which direction they're moving using meg we can understand where in the brain
integration occurs to what extent and how much the brain is taking of each auditory and visual
source thank you very much for watching and I hope today you learned something new about the brain
even if it was something as mundane as speaking and listening I hoped it helped you to understand
and recognize the complexity behind these everyday actions that sometimes you may take for granted