Hello.
My name is
Katherine Kuchenbecker from the Max Planck Institute for Intelligent Systems.
And I'm happy to tell you about our research on the S-BAN.
The first author of this TOCHI journal paper is Ad Spiers
a former Senior Research Scientist
with my team and now a lecturer at Imperial College London.
Our second author is Eric Young, a former Ph.D.
student with me.
We were motivated by a situation most of us have faced
when we visit a new city or a new neighborhood and want to reach
an important destination, but we don't know how to get there.
So what we do is we pull out our smart device.
We choose our favorite navigation app, ask for some directions,
and then we get visual turn-by-turn directions, a map, some audio instructions
that help guide us to that final destination.
Now we know that these cues are not accessible to many users.
Individuals with visual impairment and or auditory impairment
cannot benefit from these technologies.
And furthermore, even when we can perceive these cues,
we're operating in a very dynamic and sometimes dangerous environment
with taxis racing to reach their destination
and looking down at our phone or listening to audio cues
may prevent us from perceiving hazards like this
pile of scooters parked in front of the Hauptbahnhof,
the main train station, or a drop off into a canal of water,
bicyclists rushing by and buses.
There are many things
in our ambient environment that we should be paying attention to,
and some of them are also positive, like beautiful sights
or interesting opportunities that could enrich our journey.
So if visual cues and audio cues
don't solve this problem satisfactorily, what could?
I’m a haptics researcher,
and I think the sense of touch could do a lot to help guide users.
Now, the traditional approach would be vibrations;
that cell phone can vibrate and get your attention.
But vibrations don't convey directions, at least not very elegantly.
You can try to have some
complicated pattern that takes a while to learn, and furthermore,
just getting lots of vibration alerts on your phone is super annoying.
So we wanted a different solution, and the solution we
propose is shape change.
We created the S-BAN: we call it Shape-Based Assistance for Navigation.
It's a handheld device.
You hold it similar to a flashlight
that can change its own pose, its own configuration.
It can bend the end effector over to the left and over to the right,
and it can also retract the end effector and push it out forward.
So we have a continuous two-degree-of-freedom
workspace that you can feel in your hand.
Now the screen, this display, you see, this was not used in this work.
It’s just used in this video to help you understand the commands
that the S-BAN is trying to convey to the user.
And a very important feature of the design that we came to as we did
iterative prototyping was we found we needed to add tactile notches.
These are little divots, depressions, on the side of the end effector and the base
where they meet.
To give the user a reference to know are we extended
from our base pose or retracted?
And that was very important for perceiving the second degree of freedom.
When you hold the S-BAN in your hand, those tactile notches are right
there under the sensitive skin of your fingertips, so they're easy to perceive.
So now you know how the device looks from the outside and how it can move.
And I want to show you,
if we take it apart, you'll find we have an inertial measurement unit
that can be used to sense the device’s orientation.
And if we take the backplate off, you can see the actuators.
Inside we have two linear actuators that can extend and retract independently
and together that dictates the position and orientation of the end effector.
These actuators are strong and quiet and it's very safe and effective.
It can move very quickly
also in your hand to new poses and you can perceive that quite easily.
So when you hold this device in your hand with the hardware, a designer
then also has the option of how to convey the command.
Actually, there's actually more than one way
to command a particular
angle change and extension, and to explore those options.
We wrote out the full forward kinematics and inverse kinematics for the device.
This is a lot of math that you can see in the appendix of our paper.
That's where will the end effector be given the extension and retraction
of the two actuators, and we use that to explore
two different mappings between our target pose,
which is our left-right angle and our forward-backward extension.
And the first of those is what we call
the mid-point control, and the second is called leading-notch.
They differ in how they convey a turn.
So you can see the mid-point; in mid-point control,
the end effector rotates around the mid-point between the two tactile
notches on the base
and in the leading notch we rotate the end effector around
the notch on the side towards which we want the user to turn.
They convey extension in the same way, simply shifting out.
But this still creates a quite interesting difference in how to try to tactilely
communicate through shape change to user a two-dimensional command.
To evaluate the effectiveness of these two options
for motion mappings, we conducted a user study.
Ten individuals came into our lab
and were randomly assigned to feel either the mid-point or leading-notch controller,
and each of them did a task, repeatedly, about
perceiving the absolute pose of the device.
And then they also did a relative motion task, which was more difficult,
and you can see the paper for more details about that.
But what I will be able to show you is the average error
with which users perceived commands across this two-dimensional workspace.
And you can see that the mid-point controller
was actually more effective and had lower errors overall.
We also gathered
qualitative feedback and ratings, which were quite positive for the device.
And I share here a few comments.
Users said they trust the device; it feels like it knows precisely
where it wants to guide me.
And also another participant said with more training,
I can think of using such a device in an urban setting,
and that's very encouraging for our envisioned application.
So we moved on to test;
the final part of this work is testing the S-BAN in a navigation
study, and we chose to do that in a virtual environment.
Each individual put on an Oculus Quest and held two Oculus
controllers in their hands, one of which was attached to the S-BAN.
Their task was to navigate a large, grassy field virtual environment,
and specifically to follow a series of waypoints which were randomized
at different distances and orientations along the way.
Our control
condition was natural vision, inspired loosely by a guide dog
wherein an animated dog character would at the start in each waypoint
would run out and show you the direction in which you should move and how far.
And then we also compared, of course, to our haptic device, the S-BAN,
which would haptically through shape change convey the same information.
We have also a smartphone proxy, a visual device, which is an arrow
giving the same information.
And we tested the combination of those last two.
Here's what a particular trial might look like.
The user can navigate around and is feeling through their hand
and then teleporting to go towards the waypoint and waiting
for the next command to be sent
or to be indicated by the feedback modality that's active.
Our participants were able to do this navigation task very well,
regardless of the feedback condition that they were in.
They were fastest when they were getting natural vision, the virtual dog.
We have a lot of experience with this kind of cue.
And the haptic tool was slower, especially for some people.
We do think that people would go faster as they learn and gain more experience;
this was their first time interpreting shape-change cues,
and we also think that these differences would probably be less extreme
in an actual physical environment where you have to walk and can't just teleport.
But my most favorite part of the most interesting
finding from this study, I think, is the data
about where people were looking in the virtual environment.
And as expected, with the natural vision, people had to look down a lot
to see where the dog was, and the same is true for the visual tool.
But when we were guiding them with the S-BAN they kept their gaze
elevated, looking around and could therefore much better appreciate
the ambient environment and perceive any hazards that were there.
Now that's the design of the S-BAN.
We've shared the entire design open source on our project website,
which you can reach by clicking on this QR code.
We share all the files for printing these parts.
You can therefore modify them and improve the design,
change it to suit your needs or compare it to other modalities.
We also think this form factor is well-suited to combine
with other navigation aids, like a white cane or a cell phone.
Overall, I hope I have now convinced you
that shape change in the form of something like an S-BAN
is a very promising new approach to using the user’s sense of touch
to guide them to unknown destinations in hazardous and beautiful environments.
And I invite you to look at our paper, look at our project website
or contact Ad or me for more information.
Thank you so much for your interest in our work.