Translator: lisa thompson Reviewer: Peter van de Ven
Do we have any beekeepers in the audience?
Hey!
That's great.
So, any beekeeper would be able to tell you
that bees are truly amazing creatures.
A honeybee, for example, will set out from its hive,
do a meandering search for food,
and could fly up to 10 kilometers looking for the food.
When it finds the food, it literally will make a straight line -
a beeline - back home.
So clearly, these bees are extremely competent
at actually perceiving the world and navigating within it,
and they're all doing this with a brain that's about the size of a sesame seed
and weighs about a milligram.
Clearly these guys are using something really clever
and neat and unexpected,
that we'd like to learn about and profit from
or just enjoy as pure, basic scientists.
That's one of the goals of our lab:
to find out what it is that actually makes these creatures tick and tick so well.
The other goal of our lab is,
If we do discover that these creatures do something really neat and cool,
can we copy some of these ideas
or inspire ourselves to develop, for example,
vision systems for guiding aircraft that are based on biological inspiration
of the kind we learn from these tiny, incredible flying creatures?
That's the second mission of our lab.
Now, through the years of studying bees, one of the major insights we've gained
is that flying insects rely rather heavily
on the image motion that's induced in their eyes as they fly in their mind,
and by analyzing that pattern of image motion,
they're able to work out the 3D structure of the world through which they're flying.
Now, the idea is very simple.
Let's say you're flying along a straight line
or driving along a highway, for example.
You notice that the trees lining the highway, which are quite close to you,
whiz by your visual field very rapidly.
And this high speed of image motion
tells you that these objects are actually very close to you.
All right?
On the other hand, things that are much farther away,
for example, a distant hill or the clouds in the sky or the sun,
the images of these objects move very slowly or not at all
as you're moving along in a straight line.
And that slowness of image motion
tells you that these objects are very far away.
So, insects seem to analyze the pattern of image motion that they experience
to work out how far away objects are
and avoid collisions with them and things like that.
So now if I could go to the next slide -
So, if you've looked at a bee flying through a hole in your window,
(Laughter)
you might find that it flies very precisely
through the middle of the hole.
In other words, it balances the distances to the left and right frames
rather precisely as it's flying through.
Now, the way we think it's doing this
is actually measuring the speed of motion of this rim in this eye
and that rim in the other eye,
and positioning itself so that these two speeds are equal.
If they're equal, it means it's flying down the middle;
if it's higher on one side, it means it's flying too close to that side.
So it moves away from that side to compensate.
Simple way of flying through a narrow passage
without colliding into the edges.
Now, do bees really use this strategy that I mentioned?
You can't actually ask the bee that question
because it cannot give you the answer,
so you have to devise an experiment that tells you the answer.
So here's the experiment.
So we trained bees to come and feed at the sugar water feeder
placed over there.
They fly through a tunnel.
The inside of this tunnel is lined with a visual pattern,
for example, black and white stripes, in this case.
And, by the way, no bees were killed or harmed in these experiments.
(Laughter)
There's no coercion, either.
The poor bee is not forced to come to our apparatus.
The only reason they come to us is because the food we offer
is so much better than anything else they can find in the neighborhood.
So they're truly voluntary participants in these experiments.
All we do is we have these bees flying down the tunnel,
and we film them from above using a video camera.
And we find, sure enough,
the bees are flying fairly precisely down the middle of the tunnel,
as you see over there.
However, if you take one of these walls, now,
and move it in the same direction as the incoming bee -
these walls are mounted on conveyor belts,
so you can move these walls -
you find the bee flies a lot closer to the moving wall.
Evidently, what's happening here
is, because the bee and the wall are moving in the same direction,
the image speed as experienced by the left eye is quite low,
so the bee thinks that wall is much further away,
so it moves much closer to that wall to compensate.
Yes, makes sense?
On the other hand,
if I move this wall in the opposite direction to the bee's flight,
then the bee and the wall are going in opposite directions.
That left eye experiences a huge image velocity,
so the bee says, "There's something dangerously close here.
I'd better move away from this wall to compensate."
That's what we're seeing here.
So, this simple experiment tells us that bees really are using
the strategy of balancing the image motion -
or optic flow, as it is called -
to navigate safely down corridors.
And it turns out that since we published this work,
a number of robotics labs have built ground-based robots
that navigate down corridors using the same principle.
It turns out to be competentially simple and cheap to do it this way,
the way nature does it.
Now, how does an insect make a smooth landing?
Landing is a challenging thing.
This is why they pay aircraft pilots so much.
It's a tricky business.
You wonder if there's some way to make an appropriately smooth landing.
So if you train the bee to come and feed at a drop of sugar water
over here,
and film it as it's coming in to land, using high-speed stereo cameras,
and then reconstruct these flight trajectories and analyze them,
you find as they come closer and closer to the ground,
they're flying slower and slower, so when they're close to the ground,
their flight speed is low so they don't hurt themselves.
Make a nice, smooth landing.
The way they're doing it, it turns out,
they're keeping the image velocity to the ground constant as they come in.
If you approach the surface and all the time reduce to a speed
so that the image velocity to the ground is basically the same,
then you'll automatically fly slower and slower
as you get closer and closer to the ground.
That's how you make a beautiful, smooth landing.
The beauty about landing in this way is that you don't need to know
how far away you are from the surface at any time.
You don't need to know how rapidly you're approaching the surface.
All you have to do is keep the image velocity to the ground constant,
and everything else works as though it's by magic.
Just a beautiful biological autopilot that we're now putting into our aircraft.
Now, when the bee has found a good food source,
it comes back home and does this dance, as some of you may know,
to tell its other nestmates where the good food source is
so that they can also go and gather nectar from it.
And the dance takes the form of a figure of eight like this.
It consists of alternating left- and right-hand loops,
and every time the bee has finished one of these loops,
it waggles its abdomen from side to side.
It's called the waggle phase.
And that's a very stereotype sort of dance, part of the dance,
and it turns out that the duration of this waggle phase
is proportional to the distance to the food source.
The longer the duration of the waggle, the further away the food source.
Here's an example of a waggle dance.
That's the marked bee that's doing the waggle.
Every time she's finished her loop, she's doing a waggle.
And I would say, looking at the waggle,
that food source is probably about a kilometer away, give or take.
(Laughter)
Now, how does the bee work out how far it's gone?
Because it has to come back
and tell the other bees how far away the food source is.
So here's an experiment
that discovers or tries to understand how that happens.
We train bees to fly from a hive
through a patterned tunnel like this to a feeder.
They come back, and you observe their dance.
And they do a nice, strong waggle dance.
This tunnel, by the way, is lined with vertical black and white stripes.
This tunnel -
same bee, again, is being trained now to fly through a blank tunnel
which carries no visual patterns at all -
goes to the feeder.
When you look at these bees when they come back and dance,
they're showing no waggle at all.
So what we think is going on
is that bees are measuring how far they've flown
by measuring how much image motion they experience in the world
as they fly from the hive to the food source.
And in this case, because they see a pattern,
they're experiencing a lot of image motion,
so they think they've gone a long way.
On the other hand, here, where they're flying in a blank tunnel,
there's no motion, so their odometer is not even ticking.
So they don't think they're traveling a distance.
As it turns out, although this might seem unreliable,
it actually turns out to be a very reliable and robust way
to figure out how far you've flown.
Let's say you use some other measure, for example, time of flight.
If you measure time of flight,
that'll depend very much on how fast or slow you fly to the food source.
You need to keep track of that as well.
Complicated.
It's also not reliable to measure energy consumed to get there
because you could be flying against a headwind or flying in a tailwind;
energy consumption might be different.
That's also not a reliable way.
So measuring odometry using visual motion
seems to be a reliable and robust way to do it,
and that's how we're doing it for our aircraft, and I'll show you later.
So our goals for autonomous navigation
are, first of all, to be completely independent and self-contained,
like an insect or a bird.
We don't want to use GPS.
This is becoming more and more relevant
because drones are now going to be used more and more in indoor environments,
where you don't have access to GPS.
The other issue is, for example, planetary exploration.
If you're sending robots to explore Mars,
there's not going to be GPS satellites on Mars for a long time,
so you're going to have to really rely on your own senses,
rather like an insect or a bird, right?
That's what we're trying to put into our aircraft,
make them completely independent.
Another challenge for us - because insects do it this way -
is to rely purely on vision-based guidance.
We don't want to use radar
or any other complicated, bulky, heavy, expensive equipment.
We want to see what we can do with just pure vision,
the way the bees and the birds do it.
OK. So here's a view of a bee eye.
Bees have compound eyes, as you know: a right and a left compound eye.
Each one of these eyes has a visual field that covers roughly one hemisphere,
so the two eyes together endow the bee with a panoramic view of the environment,
an all-around view of the environment.
And we've, sort of, incorporated that in our business system
by having two cameras placed back-to-back.
Each one of these has a wide-angle lens,
so it provides a complete 360-degree coverage of the environment.
Another cool thing that insects do,
they're able to scan the profile of the horizon all around them
and use that to work out what their instantaneous orientation is
as they're flying.
The idea is very simple.
Let's say I'm flying along in a straight line,
and I find that the horizon on one side has gone down
and on the other side has gone up, it means I've rolled to the left, yeah?
And vice versa.
On the other hand, if the horizon in the front has gone down,
it means I've pitched up; if it has gone up, it means I've pitched down.
Very simple, right?
By monitoring the panoramic horizon [in a] peripheral arc around me,
I can monitor exactly what my flight attitude is
and control it if I want to.
So here's an example of the vision system we've built to do this.
So it's monitoring the orientation of the aircraft.
That was a tight turn.
That was a loop.
Tilted the craft quite successfully.
OK. This video illustrates the basic elements of visual guidance
that we put into this aircraft.
So we're taking off now, autonomously.
These are the two fisheye images
that are combined together to form a panoramic image.
View of the horizon profile
to work out the aircraft's orientation
and stabilize it.
The clouds in the sky and the sun are used as the compass
to work out the heading direction and control the heading direction.
We're also keeping track of how far we are traveling
by measuring the motion of the ground.
That tells us how high we are, how fast we're traveling,
and also what our progress on the ground is.
You'll see, pretty soon, these imaginary, virtual yellow circles being dropped.
They're not actually physical circles being dropped.
They're virtual circles to keep track of our progress.
Each one of those signifies 25 meters of forward progress on the ground.
Now we're coming in to land, and we're doing a bee-inspired landing
by measuring the image motion of the ground
and regulating it to make a really nice, smooth landing.
So all of this is done without any GPS or anything like that.
So it turns out these landings are really smooth,
even smoother than our best young student pilots'
because the system is very quickly responsive,
so it reacts to wind gusts and so on in a very, very, very useful way.
So, this is my final slide.
It's another challenge; it's a fully automated flight.
And the challenge here is to take off from the airstrip,
fly a certain distance, achieve a certain altitude,
and then turn back and come back and land from where we took off
without using any GPS information,
using just your visual sensors, right, just vision.
And the turning around is also done -
just to make life a little more interesting and challenging for us,
we don't simply turn around;
we do an aerobatic maneuver called an Immelmann turn,
which consists of a half loop followed by a half roll.
If you do that and do it quickly, you're flying in the opposite direction.
So let's see how that worked.
So we're taking off.
Those are the imaginary circles being dropped.
The aircraft is not seeing any discs overlaid;
it's just us putting these discs on the video
to keep track of our progress.
So we're gaining altitude.
We're going further away from the airstrip,
and pretty soon, you will see this turn, this Immelmann turn.
That's the half loop, followed by the half roll.
And now we're heading back.
As you can see, we're retracing our steps, we're following those white dots.
Although, we don't see the white dots; that's just a route of the way we've been.
And we're coming in to land.
You can see people scurrying out of the way.
(Laughter)
Yeah, so, mission accomplished.
So,
when I was trying to think of a good way to conclude this presentation,
I was boarding a flight to go somewhere,
and I happened to be moving along the airway,
and I found this very interesting ad, which basically says it all.
(Laughter)
And look.
It even indulges me with a great system,
very similar to what we have on our aircraft.
Thank you very much.
(Applause)