hi my name is Gabriel Byczynski I’m a master's student in brain imaging and cognitive
neuroscience at the university of Birmingham and I’m here to tell you about a superpower that you
definitely didn't know you had when you're in class and the teacher is trying to get your
attention it can sometimes be hard to hear them oftentimes you'll think I need to look at them to
see what they're saying because just hearing with all the noise around you is a little too difficult
this is part of that superpower I’m talking about the ability for you to hear with your eyes so how
do we hear and see in the first place and why does it matter well we hear and see with our eyes and
ears as you probably know when you look with your eyes you're taking in all sorts of information
around you the colours of things the shapes of things where things are and what's moving and with
your ears you do sort of the same thing you can tell where things are based on how loud or soft
they are and you can tell how big things are by how loud they might be these are important things
that we've grown up with and through past history and evolution have been important in understanding
our surroundings and survival of course dinosaurs after all were quite big and so they must have had
massive hearing so what do I mean by superpower and what do I mean by hearing with our eyes well
as I mentioned pretend you're in a really busy class making lots of noise around you everybody's
talking and singing and probably dancing I mean everybody loves school right but it's hard to
hear and especially when the teacher's trying to get your attention she may do something like
a very common one the reason this works of course is because it's louder than everything around them
but it doesn't hurt to watch either to look at her hands and to see when they're hitting and
in what rhythm pattern you may not realize that you're also doing this when she speaks
when you stare at her mouth for example you can see that she's mouthing words and with the little
bits of sound that you're getting through you're able to form the full sentence she may be telling
you to pull out your books or pull out a blue pencil and you're able to tell blue from red
based on the ba and ra sound that she makes with her mouth even if you can't hear me for example
you may have been able to understand that I said blue red and yellow without any sound
just like you're also able to see blue red and yellow without seeing my mouth
but when we combine those two senses as you're doing right now
watching what I’m saying and listening to what I’m saying it gives you a better understanding
of the information I’m trying to convey so how can we look at this and how did we even discover this
well there was a very famous experiment called the McGurk effect McGurk is somebody's name and with
the McGurk effect they looked at when somebody says something how the shape of their mouth
affects what you hear this was done by saying the same sound over and over again it depends on which
experiment you're looking at but sometimes it was va or da or ga in this case we're going to use the
and with the sound the they changed the shape of the mouth
but kept the same sound playing let's have a look at it shall we and see if it works on you
vava
that was pretty cool right you were looking at my mouth and based on the
sounds you were able to hear different things but as I said the sound wasn't changing I was
still there in the background going va however it was the shape of my mouth that was changing this
is what we call audio visual integration which is just a really fancy term for audio meaning hearing
and visual meaning seeing integration meaning that the two things are combined and information
are taken from each so why is this important well we've already talked about it this is why we are
able to hear when people are talking in class even if you can't really make out what they're saying
think of a time when you were listening to some music and somebody started talking to
you at the same time you may find that if you're not looking at that person it's very
difficult to hear what they're saying this is the same principle we take the auditory or
sound information and we can combine it with what we're seeing to give us a better picture
at the university of Birmingham where I do my research we're looking at exactly this phenomenon
but the more specifics of it is that we want to know where in the brain that it's happening
it could be happening here or here or way at the back but there are lots of different brain
regions parts of the brain that could be involved in this combination of information a little pop
quiz for you what was the name of the researcher that I said that studied audio visual integration
last name McGurk for the McGurk effect
and a little question for you if you're able to guess where is auditory and visual information
that is hearing and seeing often found in the brain I’ll give you a few seconds to guess
visual information that is things going in through your eyes what you see goes right to the back
of your head right at the back whereas auditory information goes through your ears to the temporal
lobes that is just above your temples on the sides of your head these two regions are quite far apart
but there are other regions in the brain that help to combine those two pieces of information
and that's what my research hopes to do figure out where in the brain is happening
and how exactly the brain knows what it's seeing and what it's hearing are the same