So let’s talk about looking. When we gaze out at the wonders of the world, whether it's
a beach or a temple or even some beautiful gems, we can learn a whole lot about everything
around us. But if we're more interested in what's going on inside people’s heads, watching
what their eyes are doing can tell us a lot about their mental universe. I’m Moti Lieberman,
and this is the Ling Space.
For most people, a huge part of how we interact with the world is through seeing it. And one
of the biggest ways most of us deal with language isn't through the more automatic mouth to
ear route, but through our eyes, when we read. So of course, linguists and other researchers
have a diamond-sharp interest in what we can learn from how people read sentences, and
how language influences the way that we look at the world around us. So tracking eye movements
is a really great way to learn about language processing.
But the first challenge to satisfying our starry-eyed language hunger is figuring out
just how to track those eyes. I mean, we're talking about really tiny movements here – a
shift of less than half a millimeter can totally change what you’re focusing on. Sure, you
can take a camera and point it at the pupil of the eye, and set it to keep track of everything
that you're looking at on the screen. But when such small movements matter so much, we need to
be totally sure that it's the eye itself doing the moving, and not, like, the rest
of your head. So we need to find some way to keep our heads totally still, so that the
camera knows what it’s looking at.
Historically, eye tracking studies achieved this through some brute-force methods,
like using a bite bar, which is exactly what it sounds like — in this setup, participants
would bite onto a metal bar.
If someone's holding on with their teeth to a metal rod that's bolted down to the furniture,
they can't really move the rest of their head. So whatever eye movements a researcher would
record could be pretty reliably assumed to be actual eye movements. But this method got
kind of drooly, and so other setups have become more popular. These days, with better technology
available, most eye-tracking studies are carried out by just mounting the camera directly on
the head itself, and then pointing it in at the participant's pupils. That's much more pleasant.
So that's how we can be sure that what we're measuring really corresponds to a participant's
eye movements. And what kinds of movements do we see? Well, your reading experience may
feel fluid, with your eyes gracefully gliding along over the letters in a dance fusing those
forms to the meanings in your head. But your eyes are actually jumping all over the place,
from one spot in what you're reading to another, in what are known as saccades. You just don't
register these saccades, because your mind kind of ignores everything it sees in between
the things you’re focusing on.
You can try this for yourself! Try looking all the way over to the left, just using your
eyes, and then move your gaze all the way over to the right without moving your head.
You don't really experience that interim sweep across what you're looking at, right? And
that's about as big of a movement as your eyes can make, from one side to the other. For
the smaller movements you make when you're just reading words off a screen, you're
not going to notice it at all. Your eyes fixate on one word or a few words, and then bounce
over to the next thing. But in your head, everything just seems smooth.
So notice I just said that we can take in a number of words at a time. That’s true,
but it’s dependent on how long they are. On average, for an English reader, we can
get about 7-8 characters into our field of vision. That can be enough for one longer
word, like “Amethyst”, or it can fit multiple shorter words, like “Greg's son.” And
once you've taken that information in, you jump ahead to the next point.
But that's all talking at the word level. Linguists want to know how we build larger
sentences and larger meanings, too. And to do that, we can just present sentences and
have people read over them, tracking how they move through the sentence over time. And it
turns out that’s not all smooth, either. Sometimes people linger on particular words,
or jump back to an earlier point in the sentence to check something, and when that happens,
we assume that something in the sentence was harder to process, compared to
a baseline, easy sentence.
Like, let's look at how we process pronouns. Remember, pronouns by themselves don't tell
you who they're talking about - “She can see the future” doesn't tell you anything
about who “she” is. So when we go through a sentence, we like to assign pronouns to
people who have already come up, rather than having to conjure some new, unknown entity
into being. Take something like “Pearl said yesterday that she would help out with sword
practice.” You have the pronoun “she” there, but you also have a totally natural
noun to associate it with, “Pearl”. And so that's going to be pretty easy to read.
On the other hand, consider a sentence like “Jasper still thought that she could escape
from the ocean.” Here, you're reading along nicely, just like the first one, until you
get to “she”. And then you're like, hmmm. I don't remember there being a female name back there
to associate that with. So you slow down. Maybe you jump back to check if
the subject of the sentence was male or female. And then maybe you go back down to the pronoun,
and try to reanalyze that “she” as someone else, or accept it as going with Jasper. Each
of those hesitations or backtracks, we can pick up using our eye tracker, and use it
to tell us about what happens when sentences don't tick along according to our preferred plan.
We can also use eye tracking to see the ways in which our linguistic processing and our
knowledge of the world interact. Like, here are two sentences that are exactly the same,
except for one word: “The employee served the donuts was very hungry”, and “The
musician served the donuts was very hungry.” Okay. These both seem like confusing garden
path sentences, right? In both cases, we're missing a little “that was” after the
subject to make the whole sentence easier to understand.
But we might consider that employees usually do the donut serving, not musicians. So we
might predict that we'd have a bigger slow-down effect for the sentence with the employee
getting served the donuts, rather than the musician, because that one forces us to go
against our knowledge of how the world works.
But nope! That's not what happened. Instead, both sentences show similar patterns of processing
difficulty in the eye tracking data. That suggests that for both sentences, we get confused
in the same way, no matter what we might think of the real-world plausibility of musicians
selling donuts. But when people were asked to paraphrase the sentence, the more plausible
the match, the harder it was to do: employees getting served donuts is harder to come up
with than musicians getting them.
This isn't the only way that we can use eye tracking, though. We can have people look
at pictures while they listen to language, too. If we measure where people look, we can
work out how they process a sentence. So picture this: you're shown a screen with four things
on it: a frog, a napkin, a frog that's on a napkin, and a box. You're just looking at
the middle of the screen, waiting for something to happen. Then you hear, “Put the frog
on the napkin into the box.” So, what happens to you here?
Well, if you lucked out and looked at the frog on the napkin to begin with, this is
easy to process. You look at the frog, and then you see that the frog is already on a napkin,
so you don't need to do anything. And then you move the frog to the box. Easy! But if
you looked at the napkin-less frog first… ah. Well. Then you probably interpreted “Put
the frog on the napkin” as the whole sentence, and moved your eyes from that frog to the
empty napkin. And then you hear the rest of the sentence, and realize your mistake. Whoops!
Now you have to get your eyes to the right frog and then over to the box. So this can
show us, too, how we try to process incoming language before we have all the bits.
And that's if you're an adult – if you're a kid, maybe recovering from picking the wrong
frog isn't so easy, as we'll talk about back on our website. For kids who can't read well
yet, eye-tracking is another tool to let us see how they understand language.
But no matter what age you are, eye tracking lets us work our way back into people's minds,
and see how we approach reading sentences, as well as how we use what we hear to focus
on different parts of the world around us. As an experimental technique, it's definitely
worth a look.
So we've reached the end of the Ling Space for this week. If you followed my eye movements,
you learned that we don't move our eyes smoothly when we read, but instead stick and then jump
from point to point; that measuring where we get stuck when we read can tell us how
we process language; and that the way that people look around a set of pictures can also show
us how they interpret what they hear.
The Ling Space is produced by me, Moti Lieberman. It’s directed by Adèle-Elise Prévost,
and it’s written by both of us. Our editor is Georges Coulombe, our production assistant
is Stephan Hurtubise, our music is by Shane Turner, and our graphics team is atelierMUSE.
We’re down in the comments below, or you can bring the discussion back over to our
website, where we'll have some extra material on this topic. Also, try dropping by our store,
where we have mugs, t-shirts, IPA charts, and more! Check us out on Tumblr, Twitter
and Facebook, and if you want to keep expanding your own personal Ling Space, please subscribe.
And we’ll see you next Wednesday. Selamat tingal!