Hi, my name is Yudai Tanaka and I’m gonna talk about our latest work that I did in collaboration with my colleague Jun Nishida and my advisor Pedro Lopes at the University of Chicago.
Here, the user, myself is experiencing a fire safety training. They are looking for the fire extinguisher under the table.
It seems they are looking at a wrong direction ... but somehow they magically find the fire extinguisher.
Existing interactive systems solve this situation by presenting some form of instruction.
such as showing arrows or markers, or showing picture-in-picture windows, or allocating spatial sounds to targets, or even providing vibrotactile feedback.
All those approaches are great, and they are actually used in a lot of user interfaces today. But looking at them closer, they all follow the same principle.
They present feedback to other sensory modalities but not to the head motion itself. It is actually the user's task to translate these feedback into their head motions.
For instance, I feel the vibration here on my neck, and I move my head in this way by myself.
Instead, I would like to propose a different concept, in which interfaces directly manipulate the user’s head orientation and turn it into an output device.
So, to reveal our trick, we don’t use any visuals or sound, but instead, we electrically stimulate the neck muscles and actuate the user’s head to look at the fire extinguisher,
which we call electrical head actuation.
Let’s continue watching this example.
Now the user put out the fire then the training says find a new fire.
Our device actuates the head upward to look at the fire on the shelf.
Now, the training says evacuate the building because the fire is on the ceiling.
However, the user is unsure about the evacuation route.
Here, our device dynamically actuates the user’s head along the trajectory of the evacuation route.
So, as we’ve seen so far, in our approach, we don’t need to instruct the user to turn their head, but instead, it just happens.
Well, our work was inspired by a line of research that applies haptic feedback to the user’s head. Let me show you a few great examples.
First, gyroVR uses a spinning weight to create inertia as long as the user is moving their head.
HeadBlaster takes this concept further and uses air jet to apply weak resistive forces.
Unfortunately, both of them are not capable of robustly actuating user’s head involuntarily.
Instead, HangerOver applies mechanical pressures around the user’s head, which causes a reflex that turns the head left or right.
However, since it’s a type of stretch reflex, it’s hard to control this effect.
So far, the only way to realize robust and controllable head actuation is this custom-made medical exoskeletons, which unfortunately require bulky motors to be attached all around the head.
In contrast, our approach does not require any hardware above the neck line.
I’m actually wearing the complete system, which can reliably actuate the user's head orientation in all four cardinal directions using electrical muscle stimulation.
Here, my colleague Alex nods down involuntarily via stimulation from these two electrodes.
This pair turns the head to the left, similarly, to the right, and this pair for nodding up.
If you are interested in customizing our approach for your application, our paper shows more different electrode placements and the resulting head motions.
We implemented a PID control of the stimulations for cardinal directions, which runs on a wearable set up with a 4-channel stimulator carried in a backpack.
Also, to support a variety of applications across Mixed Reality, VR and real life, our system is compatible with three different tracking systems, Hololens, VR Headset and AirPods.
As we already saw with the fire safety training, our direct head actuation enables applications not possible before.
For instance, we can transmit the head motions from the user on the right to the user on the left.
Here they synchronously nod to the music without looking at each other.
In the same way, we can also communicate how fast they should play the instrument.
It’s even possible to design a hands-free / eyes-free interaction base on the user’s neck.
Here, to notify the current sound level, the sound slider directly actuates the user’s neck angle. and the user can update the sound level by voluntarily moving their neck.
Our device can be also used for rendering force feedback. Here the user playing the VR boxing, feels the impact of the punches threw by the opponent.
We evaluated electrical head actuation in two studies.
In the first study, we validated the control accuracy. we actuated the participants' head orientation to acquire different targets.
We found that our system has an accuracy of around 8 degrees for static targets; and around 9 degrees for moving targets.
In our second study, we explored participants' experiences in our four applications. Let me show you some highlights.
Now, to conclude,
We introduced an alternative concept to traditional modalities in controlling our head movements,
where interactive systems directly manipulate our head orientation by electrically stimulating the neck muscles.
We hope our work will inspire researchers to further delve into this avenue.
There are several ways to expand on this work, such as how the head actuation interacts with gaze redirection,
or the long-term effect of our technique on the user’s neck posture.
Preserve the user’s agency in this modality is also an exciting topic.
To accelerate researchers in exploring these questions and more, we have uploaded all the source codes to the link shown on the screen.
So, I’d like to thank my collaborators again. First, my colleague Jun, and my advisor Pedro.
Thank you for your listening. Please check our paper for more details.