Scientists who studied behavior-- folks
like Niko Tinbergen and Konrad Loren--
spent their careers carefully observing animal behavior
in the natural environment.
And intuited that behavior's divided into parts.
But it's hard for humans to communicate
the process they used understand things like behavior.
We wanted to find ways to really objectify human intuition.
The ultimate goal is to understand
how the brain creates behavior.
To my knowledge, nobody has ever tried
to model the full 3D pose of the mouse as it evolves over time.
Because how your pose changes over time, that is behavior.
We developed a method that allows us,
through machine vision and machine learning techniques,
to identify the structure of behavior itself, as it unfolds.
So in other words, is there a grammar to that body language?
We've recorded the mouse's posture
just using a Microsoft Kinect.
And then we had this beautiful depth string
that we were able to use to build
a 3D model of a mouse's body.
We can now pretend like that data stream is just
spoken language.
So we can feed it to algorithms that we ordinarily
use to discover the parts of speech
in some foreign language.
And what we do is we try to discover
what the syllables are-- or the repeated motifs of action
are-- inside of that data string.
I develop the mathematical models
that define what the syllables are
and tell the computer how to find them in the data.
We sort of take all those video frames
and put them on top of each other
so that we can watch all the videos at once.
And it looks like they're doing very different things and all
of a sudden they'll snap together.
And they'll perform some specific kind of motion
that's very stereotyped.
Every way that we looked at the data, once we finally had,
it was clear there was an enormous amount of structure
that people just hadn't described before.
These brief three-dimensional motifs of action,
we saw that there was a kind of-- almost rhythm to the way
that mice behave.
About every 300 milliseconds, they
seemed to be switching to doing something else.
The average lab mouse's syllables
enact all the behavior that we see--
all kinds of little pieces that are
too fast for a human observer watching the mouse to say,
ah that is a repeated motif of behavior.
So, for example, a mouse might sniff or dart
or rear up into the air or bunch up into a ball.
My favorite syllable is the hunting dog syllable,
where the mouse is on four legs and stretches out, and then
raises its nose.
I just think that's really cute.
My favorite syllable is the Run Forrest, Run syllable,
where the animal darts across the arena as fast as he can.
If you give the mouse, say, a fox order, what we found
is that the mouse doesn't actually use any new syllables
to avoid the fox order.
The mouse just reorganizes existing syllables.
And through simply changing the way syllables are expressed
over time, the animal can generate
completely new behaviors.
It's like in spoken language, there's
only a small set of sounds we can make.
But we can combine them together in an incredible amount of ways
to produce spoken language.
We then did an experiment about the influence of genes
on behavior
In a genetic mutant, which has a funny duck-walking gait, what
people hadn't described is that the heterozygous mutant-- so,
a mouse with only one incorrect copy of the gene--
was also different from wild type.
And discovering an abnormality in a heterozygous mouse
is really important because a lot of psychological illness
that has a genetic underpinning in people
shows up as a heterozygous mutation.
And the main reason we undertook this project
was to better understand how the environment, genes, or activity
in neural circuits influences different patterns of action.
We're trying to bring the tools that can measure behavior up
to the same resolution of the tools that
can measure the brain activity that creates that behavior.
And so this is an unprecedented sensitivity into behavior
that hadn't existed before.
[INSPIRATIONAL XYLOPHONE MUSIC]