Hello everybody and thanks for your attendance!
My name is Marc Lieser,
I'm with the Computer Vision & Mixed Reality Group auf RheinMain University of Applied Sciences
in Wiesbaden, Germany and today I'd like to talk about our paper
Tactile Human-Quadrotor Interaction: MetroDrone.
I will give an introduction to the topic, talk about Human-Quadrotor Interaction in general
before I get to our main contribution, that is tactile interaction without intermediary devices.
I will show you a demo video before i conclude this talk,
propose future work and we meet in the Q&A.
So everyone knows today's smart assistants like the Alexas and Siris and Cortanas and Google Assistants.
But they have lesser known robotic siblings like the Jibo or the Anki Vector.
They are ground based smart assistants in the form of robots.
When we look into space, their robot assistants float.
Who would have thought? But how cool is that?
And how cool would it be if in future scenarios we had both our hands free
while the cooking recipe would hover just in the right spot?
Speaking of hovering: You probably know that there are robots that can do that.
Those are named Unmanned Aerial Vehicles and gained a lot of popularity
especially through recent years.
Drone delivery is just around the corner with recent month the FAA
approving the first commercial drone delivery in the US.
Other scenarios are Search and Rescue scenarios where small quadrotors fit through tiny spaces
in order to locate disaster victims.
But they also made their way into everyday life.
All these scenarios have in common that they require some sort of human interaction.
So traditionally they were piloted by remote controls and still are today but since they
are gaining autonomy this calls for more natural ways of interaction.
So at further distances, gesture interaction has been proposed or at closer distances facial
expression, voice or hand gestures and at immediate proximity quadrotors can be
controlled with touch interaction or physical buttons.
All these interaction scenarios require some intermediary device but even small buttons
introduce problems and add to the weight of the quadrotor that is already critical when
you intend to keep the size of the quadrotor small.
So we tried to think of a way to interact with a quadrotor without intermediary devices.
And so we had a look at the components that are already there.
So let's have a look at the quadrotor components.
Quadrotors usually consist of a frame, four motors, four rotors, a receiver,
a battery and a flight controller.
The flight controller is the core or the brain of the quadrotor and uses the control input
of the user, sensor readings to calculate the signal that is then sent to the motors.
The main sensor of a quadrotor is the inertial measurement unit, that is small electronic
device that usually consists of a gyroscope, an accelerometer and sometimes
a magnetometer and a barometer.
So there is already a sensor that measures forces. And touch interaction is force acting
on the vehicle.
So to detect forces that do not happen during regular flight conditions, we'll have short
look at quadrotor trajectories.
They are usually represented using fifth-order polynomials.
Their first and second order temporal derivatives are continuous and thus result in smooth trajectories.
A linear system can be solved for boundary conditions like position, velocity, acceleration
and time and multiple polynomials can be combined into full trajectories.
So since the trajectory generators and controllers usually minimize jerk or higher derivatives
of position for smooth flight, there are no jerky movements during regular flight.
So we just implemented a straightforward thresholding algorithm
in order to detect taps to the quadrotor's frame.
We do this by low-pass filtering the acceleration signal in order to reduce high-frequency noise
and to smooth the signal, then we determined the jerk vector simply by numerical differentiation
and when the norm of the jerk vector exceeds a given threshold, a tap was detected.
So the next step was to distinguish between multiple virtual buttons to distinguish the
tap direction and we did this by introducing unit vectors in the
And here is demonstration video of that.
quadrotor's coordinate system.
And to determine which button was operated we maximized the cosine similarity so we look
for the button that is closest to the jerk vector and this lies below an angular threshold
then the button was operated.
So the next step was to think of a scenario to demonstrate the tap interaction and we
thought of maybe in future home scenarios when quadrotors offer services around our
homes it would be cool if you're enjoying some music
and the quadrotor would move to the beat instead of just hovering mechanically
or next time step or next quarter note the trajectory was executed.
to give it some sort of character.
And so we thought of a musical experience and the idea was that the user taps the frame
of the quadrotor repeatedly and depending on the button the quadrotor would respond
with different dance trajectories matching the beat.
[Music]
I hope you enjoyed the video.
Here is a short overview of the four trajectories we used.
When we operated the blue button, we executed a side-to-side trajectory.
The boundary conditions were predefined, but we solved the linear system for the time depending
on the tap frequency of the user and the first trajectory was a simple side-to-side motion.
You can see the four vertical lines are the four taps of the user and by the fifth tap
For the red button we introduced an up-and-down trajectory.
For the orange button an hourglass-shaped trajectory and
or more reserved movements depending on the tap intensity.
for the green button a zigzag back-to-front motion.
We introduced a Human-Quadrotor Interaction scenario that refrains from intermediary devices
by detecting taps in the accelerometer data.
This is a minimalist approach by exploiting what is already on-board.
By this we kept the size of the quadrotor to a minimum and the quadrotor responding
to music makes it appear a little less mechanical and intimidating and a thus a little bit more
emotional and by this we hope that we could lower the inhibition threshold that exists
when it comes to interaction between humans and mechanically-looking machines.
And to improve our work we thought about introducing a spherical shaped frame so that the tap directions
of the virtual buttons are not restricted to the horizontal plane of the quadrotor.
To improve the MetroDrone scenario, we thought about utilizing the tap intensity which we
already determined but didn't use for this scenario so the quadrotor could fly wider
Furthermore it would be thinkable to query appropriate databases for the musical genre
and to execute dance trajectories depending on that genre.
So thank you very much, we'll see us in the Q&A.