- Okay, good afternoon.
I am Theo Amanatidis and I'm a PhD student
at the University of Cambridge,
and today I will try to do my best and present
this thesis paper, instead of the main author,
which is Charlie Hewett and the paper is on
Assessing the Public Perception of Self-Driving Cars,
the autonomous vehicle acceptance model.
First of all, I'd like to start by thanking
the previous presentation because it was very interesting.
We kind of have some similar topics
and I'll try to point it out as we go along
in this presentation.
So, as indeed mentioned in the previous presentation,
vehicles with limited levels of autonomy
are already coming up on the market.
They already are on the market
and deployed in different places in the world,
in different environments.
However, while there's clearly people who are buying
these vehicles, there's little evidence to suggest
that users in general, kind of broadly,
desire the wide-spread use of autonomous vehicles.
I mean, what I'm trying to say is,
we don't know if this is just a specific subset of users
that are choosing to buy these vehicles,
or if this is a broader trend in the industry.
More specific than that, most existing studies do not,
of public perception of autonomous vehicles,
do not use established user acceptance models, okay?
And therefore, it might be difficult to make comparisons
or interpret the results between these studies.
For example, as this chart shows,
as the level of autonomy increases basically,
and the level of interaction
between the user and the vehicle reduces,
it is unclear exactly what part
of that interaction is reducing.
Is this, as the previous presentation meant,
is it the hands that are taking off the wheel?
Is it that you're not paying attention
to auditory interactions within the vehicle and the user
or is it eyes and brain that are off?
So what we're trying to do, is basically,
what this paper tries to do,
is establish a model of,
based on user or the previous user acceptance models,
that allows us evaluation and comparison
between different studies.
Okay?
So our proposed approach is to unify current efforts
in measuring acceptance of autonomous vehicles,
to combine the different elements
between current technology acceptance models,
car acceptance models, and introduce our own element
of autonomous vehicles, and levels of autonomy.
Furthermore, we believe that
the concrete industry-set levels of autonomy
might not be intelligible to the average user,
or might not be extremely indicative
of what the vehicle can and cannot do.
So, we want to enable participants
to better visualize these hypothetical technologies
and how they are implemented in their vehicle.
So as part of that effort, this paper presents the AVAM,
which is basically the initials of our
autonomous vehicle acceptance model,
as presented in the beginning.
So the AVAM is a 26 item questionnaire,
which is basically an adaptation of two previous models,
specifically the Universal Technology
of Usage and Acceptance of Technology,
and the Car Technology Acceptance Model.
And I'm gonna explain later, specifically what parts
we have adapted and what parts we've added in.
It uses six autonomy scenarios,
so these represent as zero to five level of autonomy,
but they're not labeled level one,
or level three, or level five.
Instead, it's an explanation that's given to the user
of each level of autonomy.
Finally, this model was first piloted with 54 people,
and then developed with 187 participants,
all taken through Amazon Mechanical Turk.
Just a little bit about their characteristics,
the average age is 22 to 65 years old,
with a mean of 34 and a standard deviation of 9.2.
We have 111 male and 76 female,
sorry, not quite 50-50, but getting there,
and all of them have been taken from the US
and I will describe a little bit at the end
what this means in terms of the sample.
And finally, and this is kind of reporting to us,
the full dataset is available upon request,
so everybody who wants to use it
or include it in their research.
Okay, so the questionnaire was established
based on, as I said, the two previous models.
It consists of some background questions
to understand about what our users are,
what the usage of the vehicle is,
do you have any previous driving experience?
Actually, 181 of the 187 people
did have a driving license in world driving.
So, the individual items,
we're taking to correspondents as much as possible to UTAUT.
That means that out of the eight factors,
if all of the eight factors of your total were included,
and then the ninth item was the last factor of CTAM,
which is about perceived safety.
And again, I'll explain that in detail.
We kept also the seven-point Likert scale, as per UTAUT,
and the five-point sLikert scale was introduced
for the additional questions.
So the additional questions, what are they are?
Well, as given a little bit in the previous talk,
perhaps it is uncertain for users what elements,
what level of attention
do they need to have when they're driving?
Do they need to use their hands?
Do they need to use their feet?
Do they need to use their eyes?
So basically we added three additional questions
at the end of the questionnaire to evaluate
if users knew what part of their body, in a way,
they needed to make sure that they're using
during the driving at this individual levels.
We use the term eyes instead of brain or mind
because we thought that this might
imply more complex engagement.
So it's just feet, hands and eyes.
And also, I'd like to mention that the participants
had to answer the questionnaire six times,
one per each driving scenario.
This was done in a row.
Now there's obviously trade-offs
and I think that was mentioned
in one of the questions beforehand.
At what stage do you answer the questions,
and if you answered them in an order or not.
We chose to do this in order,
which might introduce some sort of bias in that
but the trade-off, the positive if you will,
is that it will basically allow users
to kind of visualize in increments,
in hypothetical increments,
the distance between where we are in the present
in the technology, and where we will be in the future.
So this is basically what I was saying beforehand.
Of all these scenarios, there's one that
proved to be particularly difficult,
and that is level four.
The reason why level four is particularly difficult
is because the definition of level four in SAE standards
is very broad and can be divided, if you will,
in different ways.
For consistency in this study, we used
a definition that follows a specific type of roadway,
so basically an extension of level three.
And by the way, you know the text at the bottom,
this is just kind of illustrative.
This is not the text we had actually in the paper.
You can find that on the paper.
In terms of the results and what we're aiming for,
we are aiming for high internal consistency
and external validity obviously.
As you can see on the right-hand side,
we had scores of 0.8 on all of the,
so this is the Cronbach's alpha, I should point out,
on all the different factors
that are constituents of the AVAM,
and above 0.9 on some of them.
In terms of external validity,
we wanted to compare this study with Rodel et al 2014.
However there are limitations because obviously
the study at the time used a different kind of scale,
didn't use SAE,
there were some differences between the questions.
So we tried to match that as good as possible.
What ended up happening is that
we had to exclude level three
because there was not, so the Rodel et al did not have
the equivalent of what we now call SAE level three in 2014.
Again, for external validity, we couldn't quite fit it all
in this very short presentation,
so you can find that in the paper if needed.
So having established this model
we went on to kind of evaluate some questions that,
what might we use this model four?
Specifically, the first tried of these two you can see.
How does the user acceptance vary
with the level of autonomy of the vehicle?
And which methods of control do users expect to have
for vehicles with different levels of autonomy?
So this is what I was saying about hands, feet and eyes.
So what you can see here,
and hopefully this is not too small,
is the result of the nine factors
that were described earlier for AVAM.
So I'm not gonna go through all of them
because otherwise that'll take forever,
but what we can see in these,
so by the way I should point out that
higher is better for everything except anxiety,
which obviously lower is better.
So what we can see is that generally
a trend towards reduction of, say,
performance expectancy or effort expectancy,
as we increase the level of autonomy.
We believe that this may be because of
lack of trust in users,
but also because of the increase in having to
make a hypothetical guess about
what this technology will be in the future.
So people cannot quite perceive how this technology will be
and how basically it will affect them when they're driving.
So this is particularly visible
in things like perceived safety,
while the vast majority of studies, for instance,
recommend autonomy as a means to reduce accidents,
it turns out that users mostly disagree,
and as the level of autonomy increases
perceived safety reduces.
We attribute that to uncertainty about
where the technology is and how they perceive it,
how the users perceive it.
In terms of the second research question
this is also somewhat different.
In this case, what we see is that the level of interaction
between level zero and four
remains relatively constant for hands, feet and eyes
with perhaps a little reduction in terms of hands
as we go along.
But then there's a massive, or a sharp, drop
between level four and level five.
We believe that this is because users are
understanding the difference between level five
and all the other levels.
However it seems like the difference between
level one and level four remains uncertain
in the eyes of a lot of the users.
So this is basically what we wanted to highlight,
one of the things I wanted to highlight, with AVAM.
So I have to go a bit faster to conclude here.
So partial autonomy is perceived to require
uniformly higher driver engagement than full autonomy.
So this is clear from the last few graphs.
The public is not yet fully convinced about autonomous cars,
and they have some uncertainty, lack of trust,
and they're anxious about some of these technologies.
Also, perceived user engagement does not escalate clearly.
Some engagement may be cumbersome,
regardless of how often it is required.
So the trait we talked about,
so we heard about handovers and taker of this previously,
this is an example of that.
However, there's a lot more work to be done.
Analyses by age, gender, driving experience
are yet to be concluded and hopefully
we will do that in the near future.
In addition to that, in this study
we only used US participants.
I think it would be interesting to see the difference
between users from different countries
and see if there's a cultural difference involved.
So in conclusion, this was a standardized evaluation
of autonomous vehicles.
Sorry, AVAM provides a standardized evaluation
of autonomous vehicles across studies,
and a meaningful assessment of
changes in perception over time
because you can re-do the study
over a few years to see if there's a difference.
Similarly with different autonomous vehicle technology,
so different levels of automation.
It provides high internal consistency
and external validity,
and the results of using that to research questions
was that participants generally are positive about
autonomous vehicles but with preference
to lower levels of autonomy.
The utility of autonomous vehicles
seems to be more appreciated in level five.
And we hope that,
and this is kind of extension to the rest of the room,
that findings should inform efforts
in simplifying autonomous vehicle human-machine interfaces
across different autonomy levels.
Yeah, that's it.
I'm Theo Amanatidis, thank you very much.
(applause)
Any questions?
- Actually I will try pass the mic around the room.
So do you have questions?
Raise your hand, anyone?
Oh yes, here we go.
Say your name and your affiliation and question.
- [Audience Member] Hi, thanks for the talk.
My name is Karl, from LMU Munich,
and I would like to ask about level four
and the presentation.
Before we've seen that there is no takeover request,
so if there's no takeover request,
can you elaborate the difference to
level five and level three?
Thanks.
- Thanks for the question.
So I think this depends on your definition of level four.
In some definitions of level four,
there's no takeover request.
So for instance if this is in transit vehicles we've
only used within low speeds, say, outside of normal roads
or if it's used within kind of a certain very,
not necessarily very but somewhat, constricted way.
So for instance, it could anything weather,
it can be roads, it can be speeds.
In this case we only looked at roads,
and an example of that would be, for instance,
that this vehicle is fully autonomous
in all city environments, OK?
But it's not autonomous in highway environments.
According to the SAE levels, that is level four.
There's no level three.
There's no level five either.
But you can apply the same thing, for instance,
autonomous vehicle that's fully autonomous
in any situation except, say, snow,
and then has to manually driven in the snow.
Or can be fully autonomous up to 80 kilometers per hour
but then has to be manually driven
after 80 kilometers per hour.
So there will be some sort of takeover
between autonomous and manual.
So that's half the answer.
The other half of the answer is that, clearly,
we are discussing this here
and we like to think that we are experts
in autonomous vehicles, right?
People who are not experts in autonomous vehicles
do not know the difference,
do no know when,
if they buy an autonomous level four autonomous vehicle,
do not know its capabilities
and when they can use it or not.
And I think that's perhaps what we're trying to
kind of explain in the second half of the research,
the second question that we had.
- [Audience Member] Hi, my question is
if you already have everything on US,
I think it would be interesting to see
where the people are in the US
who have answered the question,
because I think for instance in Silicon Valley
there's driving around a lot of autonomous vehicles
so it might be the scales for anxiousness or anxiety
would be potentially different,
that would be something that you could look into.
Whether the exposure to those types of cars
will actually change anything to that.
- Yeah, thank you for that.
You're totally right.
I don't think we have done detailed analysis
of which city people live,
but we definitely looked at sort of the environment
that they lived in,
if there was urban environment,
if it was countryside, if it was suburbia,
so this kind of sort of level.
And we had a good representation of people
in urban environments and suburban environments,
a bit lesser in rural areas,
but they were still represented.
So it's not as if all these people are in Silicon Valley
but it's also not that these people
all live in Kansas or something,
it's kind of a combination.
I think you're right, there's potential to look at a
even more detailed level, certainly.
- So we are right on time.
Thanks our speaker again.