- Will virtual reality, augmented reality
and mixed reality replace screens as we know it?
And should UX designers be considering
this future technology in their process?
At the August virtual conference, we held a Q&A session,
where Brandon from Melbourne asked about the future
of AR, VR, MR and UX.
Jakob Nielsen shared his thoughts.
- Well, I don't think that they will replace screens.
I mean, two-dimensional monitors the screens
and they can be bigger and bigger,
are very powerful ways of interacting
with a lot of information.
I mean, a lot of information is actually very well suited
for two-dimensional display.
I mean, text is one, but a lot of other things as well.
There's not very much data that's actually as inherently
exactly through dimensions.
So I mean, data is one or two dimensions
and some data it's like many, many dimensions,
like let's say user experience design problem.
Well there's so many different angle to dimension
and issues we need to consider in user interface design,
that's not a three-dimensional design problem.
And the same is true for a lot of other knowledge work
that is really very multidimensional.
And so, showing in 3D rather than 2D
doesn't necessarily buy to that much.
There are some exceptions obviously,
there's some things are inherently 3D,
like a lot architecture building type of things,
a lot of medical problems as well
'cause the human body, you know, is 3D.
So we've got to like play in surgery or something
that's a 3D problem.
But not that many problems are like that.
And it's also not just for knowledge work,
it's also for live entertainment applications.
Some, yes benefits on 3D, like, you know,
like a shooting game or something like that
can be more and more engaging and immersive by stream 3D.
But other types of entertainment, I would say less so,
like think about something like a movie.
Well, hypothetically you could say, well,
it could be 3D and you can try like walk around
and look at the access from all the different perspectives
and angles and so forth.
But in truth that's just too much work and hustle.
And it's almost always better to take the camera angle
that a good cinematographer and director,
they decided is the most suitable and it should be a closeup
or should it be like an establishing shot.
Well, depending on you know, how the story flows
one of the others better.
And maybe from this angle gives you
the better of view of something,
I mean, from this angle, gives you more exciting view of
exactly, this is going on.
And these guys are highly, you know expert professionals
and they spent often years
on making a film that you watch in two hours.
And so enormous amount of thinking goes into
planning these camera angles,
and you get a better experience, which is what it's about
by actually abandoning the responsibility
and watching a two-dimensional representation
of this action rather than yourself,
try to move around 3D representation of it.
So I think for a lot, a lot of purposes
2D and screens, the traditional screens,
just bigger, higher resolution screens,
a higher frame rate for the videos, all those things,
there're definitely a lot of technical things
that can be improved,
but I do think that the inherent 2D,
is superior for a lot of things.
Now, some things 3D is in fact better.
We've been doing some user studies on AR augmented reality,
not the virtual reality with the helmets and all,
but just augmented reality.
And what we have found is that with the current situation,
there's a lot of interaction over it into action costs,
which is kind of just extra hustles
that producers have to go through
in order to get what they want,
for things like, let's say a virtual campus tour,
If for university, if you can't go on physical tour
of the university of right now,
then you could get a virtual tour that makes sense.
But the overhead of manipulating that is often too much.
And what we often find in these studies is that
for some of the initial experience,
it might be easier for people to watch like a photo gallery
or an edited movie, a video of the environment.
And then if they are very extra interested,
then they can get more you know,
additional information and additional engagement,
by an actual of kind of simulated virtual tour,
then what they can get the controls,
but the interaction costs, it's really a cost benefit.
This is true for all usability things
It's always cost benefit,
how much hassle for the user compared to how much extra gain
they get and in the beginning what people are considering,
many, many things they wanna look at,
it's not worth the extra manipulation overhead
of using the 3D interface so the AR interface
it's much better to actually have the simplified interface
and then reserve the fancy interaction for the fewer smaller number of cases,
but people have an additional large level of commitment or interest.
So that's of course it how it is now.
And it's like that now,
because the interaction techniques are fairly primitive
and the users are not very accustomed to using them.
So those are two problems that are holding us back.
Now, if we think 10 years ahead, 20 years ahead,
presumably we will have a better interaction techniques
you can actually see from video games,
which is one of the domains where it does often work.
Video games have been involving you know,
and becoming better at these kind of complex manipulations.
It's just not quite reached through to other domains yet
we can really learn from the gaming people.
So that would be one level of advancement
and the other level would be a more
a user kind of experience with having done it in the past,
therefore being able to do it again.
But if you're talking about what's gonna design right now,
which is what we are talking about,
not what well, you might design
in 10 years or 20 years, right now, I would say,
be hesitant in using these tools
and what you will do as a professional in the field,
I think it is something that will be growing,
because these two things I mentioned
will kind of become better.
My prediction seems to be growing at a relatively slow
rate that's not gonna be a big overwhelming thing
that's gonna take over very much,
and therefore, I don't think it's worth your time really,
our investment to become super engaged in this
unless you happen to be in one of those fields,
like maybe we do game design where it's more important,
it's worth monitoring, it's worth getting experience with,
but I would particularly say it's worth studying people,
trying to do real things, not doing a demo show.
Here see my virtual tour, isn't it cool?
Oh yes, that's looks very Sci-Fi.
But then if they're actually trying to look into,
you know I'm gonna like buy a house,
I'm gonna looking at 20 different houses
to see which few one's gonna be on my short list,
they may prefer to use a simpler technique, right?
So you wanna watch people
do something real with the interface
and that's how you really judge whether
its actually worth the large amount of hustle to do it.
And then of course also do usability studies
this is what I know, we always say,
but it's particularly true when you're doing something new
where you had these impediments to usability,
then you really wanna polish the UI more,
I mean, you always wanna do it,
but you wanna do it even more so
to really do use a testing for this type of interface.