Welcome Cloud Gurus to this new show on tech. I am Lars Klint.
And this time on the first episode, we're going to talk about augmented reality.
I bet you you'll see some stuff that you haven't seen before.
And I might just give you a tip or two on what you can actually use this stuff
for both commercially and as a developer. Like what do they actually do?
And what are they good at? What they're not so good at? Where can you use them?
Are you enterprise? Are you a developer? Are you a customer? Doesn't matter.
We'll talk about all of that. Also a bit of background about the devices.
It's all coming up on this pilot episode of Future of Tech.
Now, before we start looking at these beautiful devices,
let's just talk a little bit about the background of augmented reality.
No, it wasn't that long ago, actually,
that it started it's about early nineties, thereabouts,
that the American military had a good idea, they called virtual fixtures.
Now they want it to be able to remotely control a robot or some sort of
machinery.
And they came up with this idea of wearing an exoskeleton or like a piece of
scaffolding, um, from the user and then remotely control the,
the hands of this robot. Now, in order to do that,
you wear this thing and they overlay the actual hands of the robot digitally.
So when you control it, it looked like it was your hands controlling.
It was very cool, very convincing.
And it worked well for probably bomb disposal scenarios or something kind of
dangerous, which is what they wanted to use it for. Now,
the first kind of what we know as augmented reality experience came in the form
of phones, right? So it was very basic, very,
very simple kind of ideas that allowed you to have augmentation of your reality.
So you took your smartphone and you had a trigger point or some sort of
character or symbol that would trigger the experience.
So you point your phone at, it could be a QR code, for example,
or it could be a picture of a boat and this boat would then for example,
come to life on the phone. So you'd get this sort of 3d nice image,
uh, as an augmentation of the reality, hence the word augmented reality.
And it was very powerful and it was very convincing if you were going to buy,
say a luxury yacht,
because this would bop on top of the sales brochure or whatever. Um,
but it was kind of limited, very simple, very basic.
And it was all dependent on this kind of symbol to generate the
experience. Now,
the next thing that came along sort of probably about five years ago is what we
call Google Glass. In about 2013, 2012,
there were rumors of this new head worn device and why this was particularly
groundbreaking was that you had your hands free.
So you wore these glasses or this Google glasses,
and you were seeing through one eye, you had an augmentation of reality,
but let you use your hands and everything was voice command control.
And you could take pictures of the environment so you can get GPS coordinates,
you can get GPS, navigation, instructions,
all that sort of thing were available with the Google Glass.
It was really good for production lines. Well,
Tesla cars used it for a long time.
I think they're still using it actually for just in time information on what was
available on the production line. If they're running out,
if they're on schedule all this sort of stuff in real time, again,
you had your hands free to do whatever you needed to do.
Essentially that was a heads up display.
It didn't do much other than show some information in front of the user that was
not really dependent on the environment,
but more just of what the user is seeing. Now, just to, to compare with that.
We have virtual reality and it's,
we're not going to talk about virtual reality today as such,
but it's important to know what it is in relation to augmented reality.
So virtual reality is much, much older concept.
It actually started about 200 years ago with something called stereoscopic
images, uh, which is like the 3D glasses today. You have two images,
they overlay each other,
and you've got the sort of virtual imagery experience a long time ago.
Fast forward to about the mid 80s. And there was being used with big headsets,
large supercomputers at the time.
And it's probably like this actually today's power.
And there was both commercial experiences and there were some consumer
experiences where you can go around and shoot each other with 8 bit arrows. Um,
it was all very cool in the 80s, but it kind of died out a little bit and, uh,
has reasonably got revival with what we know as the HTC Vive and Oculus rift,
PlayStation VR, and there's a lot more devices coming.
Something like Google cardboard, you put your phone inside a piece of cardboard,
your Android phone, uh, and it effectively splits your screen up into two,
one for each eye and you, where you can look through this cardboard contraption.
Very accessible, uh, very cheap to use, very easy as a developer to develop for.
But we're not going to talk much more about virtual reality today because we're
focusing on this.
Augmented Reality.
So now we talked about the well, the background, the history of these devices,
but the present day is well, it's much more interesting, I think.
So what's happening now is that we have more of a spectrum really,
of where AI and VR fits in. So you can say one of the spectrum,
you have pure AI.
So that's where you have just an overlay of data that sits on top of the real
world. Uh, as you know,
digital assets that you can see the other end of the spectrum,
you have VR virtual reality where you remove everything the user sees and
completely reinvent what they, what the world is. Is completely designed.
But what's happening more and more is that devices are moving between the two.
So something like the HTC Vive now has little cameras that you put up behind it
that will track where you are in a particular playing space.
And the everything's moving more and more towards this that you can interact
with the environment that you're in, whether it's AR or VR.
And as these things move together in the middle, it becomes,
what's known as mixed reality.
And mixed reality is a broad spectrum that goes from one end to the other.
Some of them are more VR focused, some are more AR focussed,
but everything is kind of, um, redefined.
So what's happening now with modern devices is that devices like the modern
Android phones and iPhones are building devices that will
let you use AI as mixed reality. So they still call it augmented reality,
even though it's moving towards the middle of the spectrum from Apple it's AR
Kit and from Google it's AICore.
And these are two technologies that consumers necessarily don't know about,
but they know what they do, right? The Apple device is in particular.
I want to show you a really cool app from Ikea of all places, um,
that lets you place furniture in your room. Now it's not just furniture,
it's any kind of Ikea furniture, but it's not real.
So let's place some furniture in the room. Uh, I like this couch.
That's a very nice couch, so let's move it, oh let's see...
we can move it with here and move it over here or kind of swiveled
around. Um, we can place it there. With a little tap.
Now, if I move away a bit,
you can see that I can see the whole of this couch in real time.
It's in my space now. It's not super,
super accurate yet because the mapping of the room maybe be a little bit off.
So it, no, it doesn't know exactly where all the floors are,
but you get the idea. So that's AR Kit from Apple,
AICore from Android or from Google works very much in the same way.
It uses just the hardware that's in here. So it's your gyroscope,
accelerometer the camera. And if it fixes points,
so it finds points that it can place in the real space, such as, you know,
corners of the table or whatever it might be.
And it'll then use that for tracking.
So then means you can place things on the table, on the floor in real time.
It's very, very powerful.
VR is also going towards more of a mixed reality space.
What's happening now is that you can,
I've seen an example where you can have what's called the Magic Leap.
So Magic Leap is a finger tracking device, essentially very,
very accurate tracking of the fingers. If you combine that with virtual reality,
you can suddenly see your real fingers or real hands in the virtual reality
space. And I've seen an example where someone picks a virtual flower.
So it's in the virtual reality world with the real hand and it looks it's.
Yeah. It blows your socks off. Both of them,
the best example of that at the moment for mixed reality that I can think of and
is a bit of a passion of mine. I agree is this little buddy here, right?
So this is the Microsoft HoloLens. It's a self contained unit.
So there's nothing else needed to run an except what I have in my hand.
And it's called mixed reality because it mixes your real world and your digital
world. Very, very cleverly. So on it here,
there are some environmental cameras at the top.
There's four of them and they'll map your room or your space in real time.
So it'll give you any very, very,
very precise 3D representation of your space that will then live inside
of the whole lens. And you can place things on it.
So you can suddenly start interacting digitally with the physical space.
As an example, I have a physical table here.
If I place a digital ball on it and just sits there.
If I were to tip the table and change essentially the gravity on it,
or the forces on it, the ball would roll off.
The digital ball would roll off the wheel table. That is really,
really powerful.
All right, opening holograms
[inaudible].
This device is the only one of its kind currently,
nothing else on the market will map the environment in real time to that degree.
And it opens up a whole bunch of new avenues of improvement. So for industry,
if you're in healthcare, for example,
you can use this to overlay surgical tools that are inside the patient.
So you can see exactly where they are at all times.
Now this is still experimental,
obviously is not something that's in real world right now. Um, but it's coming.
If you're in an engineering plant, you can recognize objects.
So you can use a service called Vuforia for example, uh, it has, uh,
an API that you can use for your development and let you recognize specific
images of specific 3d objects, and suddenly have actions based on that.
For example, in a manufacturing environment, and you can do, uh,
get performance, uh, figures for that particular machine,
you can get maintenance schedules or whatever it might be that you need in that
particular scenario. Very, very powerful device. Coming on from this,
so this is mixed reality as well. Um,
this is mixing obviously digital and physical.
Now there's other devices as well,
which are these called windows mixed reality headsets.
And thank you Microsoft for being very, very confusing naming convention,
because these are nothing like the HoloLens.
The HoloLens is about $3,000 US. These are about $300. As you can tell,
there's a tether on it. You gotta plug it into a computer,
that's got to have a reasonably powerful graphics card, most of all, but also,
you know, RAM and processing power. So they're not standalone.
They do require some equipment to run. These are more like virtual reality.
They're called mixed reality because it actually tracks your movement.
It'll know where you are in relation to your computer that's running it. Uh,
but for all intents and purposes, these are virtual reality experience.
This is what's being pushed right now. This is also running windows 10.
It's using the same apps as the HoloLens to a large extent.
And the development environment is kind of almost there, not quite,
but you are developing the same way for both using unity and visual studio. And,
um, this is a good advancement in the technology,
but it's certainly not anywhere near the level of interaction that the HoloLens
can offer. So where this is going is all part of the future talk.
I guess we also have controllers that comes with the virtual reality, uh, sorry,
just say virtual reality, right? The mixed reality headsets. Uh,
and these are amazing. Like I tried the Vive controllers initially there were,
okay, these are a hundred percent accurate. Like if you wear this,
if you put this on, wear it and you look down,
you'll see exactly where the controller is in full 3D object and you can just
pick it up. Like it was there. It was very, very impressive.
So it has moved a long way for, for relatively low cost.
So that's kind of where we are now. That's kind of what we're doing today. Uh,
we've got the HoloLens. We got mixed reality headsets. We got the, you know,
the Vive is coming out with a pro edition.
That's going to be a wireless Oculus Rift as well.
So we're going to get away from the tethers, which is very important, I think.
And we're going to have a much more interactive, um,
let's say roadmap for these experiences. And it is here to stay,
this is not going anywhere. Devices are going to get smaller, cheaper,
easier to develop for that's essentially where we are today.
Well,
obviously we can't have a show called Future of Tech if we're not talking about
the future. So before we talked about it a bit, you know,
the present what these devices do, but where's it all headed, right?
What is actually gonna happen from now on?
So there's a really interesting company in the AR space called Magic Leap.
And they're a bit of a mysterious rainbow unicorn secretive Ninja company
because they've raised an obscene amount of money, about 2.5 billion dollars,
something like that.
And they still haven't produced anything that is available to consumers or that
you can buy. Before Christmas of 2017, they announced that now there was,
going to be a product coming in 2018, called the Magic Leap 1 or just 1.
And it's a really cool device. It looks kind of,
like a steam punk welding glasses type of thing that you wear. Um,
it's got a controller like a bit smaller than these,
and it's got a circular kind of dish that you put in your pocket, right.
The imagery that they've shown is very convincing,
although also looks a little bit made up.
So we're not quite sure exactly where they're going to fit in,
but it is very much a device like the HoloLens,
which I'm very excited about because I think we need a bit more competition in
this space.
The Magic Leap One promises a lot of interaction with the real world and
a much more detailed kind of gesture scenario.
It's very promising and it looks very cool.
I'm very excited to see what's going to happen in 2018 with that particular
product. Speaking of magic leap and mixed reality,
this is kind of where everything is headed. So the future of tech in this case,
is going to be a lot more of remember the spectrum we talked about before,
everything's going to move towards the middle because that's where the real
value is in the interaction and the, and the experiences that consumers wants.
No one really wants a pure VR experience because it's too fake.
You get seasick or motion sickness, whatever it might be. Um,
I never once real a pure AR because it's just data on top of us, you know,
your real world is not that interactive. We want to be in the middle somehow.
And I think that's where everything's going to go.
Couple of interesting devices that are coming is there's a meta too,
from the meta company. And this is a, you can, where you can order it.
Now it's a, again, a mixed reality device. It's tethered though,
which is a little bit disappointing, but you know, I haven't tried it yet.
I think it will be all right. The amount of gesture,
detail you can do with is unbelievably. So I've talked to gestures about, uh,
just as a lot with HoloLens, you use kind of this gesture all the time,
tap that's about all you can do.
You can kind of tap and hold and move your hand around,
but that's pretty much it it's kind of limited, but also easy to remember.
The meta two has much more detailed, um,
gestures where you can kinda hold things and move them around in 3d.
And you have a little tool box, you can grab things out of and put it's very,
very convincing and very cool.
And it also allows you actually to interact in real time with other devices like
they have this whole sort of interaction platform, which just looks very,
very promising. There's also the, uh, the Oculus Go. So this is again,
I'll talk to them briefly mentioned before the,
about the untethered version of the Oculus Rift and this has massive potential.
So it's still is a VR, um, experience,
but because it's untethered,
you can have much larger playing areas you can map out and your playing areas to
a much larger degree. And that is really exciting as well.
I'm keen to see what the battery life is though. Cause that's always a problem.
Of course, there's HTC Vive pro, which I mentioned as well, which is also the,
uh, the higher resolution. I think they're going to 4K now. And they have a,
an untethered version as far as I'm aware as well.
So it's all kind of going towards the mixed reality idea of free movement and,
and one device for everything and not having to, to hook anything up.
What does this look like in actual implementation? Like, where's this going?
What are you going to do with this stuff in the future?
Now there's different industries,
obviously that can benefit from it in different ways.
Imagine you're at a coffee shop and today you kind of look up behind the coffee
shop on the Blackboard and there's some prices on this is your latte and there's
your muffin and your banana bread or whatever it is that you buy at your coffee
shop.
Imagine if they didn't have to update that and you just needed your phone or
your glasses or whatever it might be that the AI experience is coming through.
And you look at this and it recognizes where you are cause GPS obviously,
and it then just shows you the menu, or it might be the deal of the day,
or it might know that, you know, cause cause it's you you're wearing the device.
It says, Oh, you've been here 10 times before today, you get this special offer,
whatever it might be,
little things like that that makes your everyday a bit more exciting,
a bit more effortless, I guess,
because you don't have to stand there and figure out if they got this left or
what's available. Uh, if we take that one step further,
what about you remember the movie Minority Report from about 15 years ago,
Tom Cruise, if you don't that's all right. But if you,
do you remember this particular scene where he's moving memories around in like
thin air, right? There's images floating and he's just kinda got,
and he throws it over there and it comes back into the screen and he gets one
out of the, yeah,
everything's kind of like hand gestures and he's not wearing anything, you know,
it's augmented and see it, which I think you probably would.
But imagine that like,
you'd have six screens today and you just have it in one sort of environment and
you've got your, your coding or your Excel spreadsheet or Excel.
It's a bit boring, but something cool right on the, on the screen.
And you just kind of move it around with your hands and you might have a
keyboard or you might just use voice commands. I think that's where were headed.
And I think that's very exciting.
Another technology that is not really in mainstream media,
but I've been following up on quite a bit is the use of haptics. So, you know,
haptics from your phone, if it vibrates, you know, that's a haptic feedback.
It could be that you're using your fingerprint sensors and it goes,
that's a haptic. Imagine if you had a suit on a full suit and everything,
and it was haptic or it had some sort of feedback at various places.
So if you're doing a zombie game, you get shot and you can feel it in your arm.
Or if you're, um, you know, doing surgery,
you can kind of feel the brain matter of the patient. Oh,
that's a bit disgusting, isn't it. But you know, you get my drift.
Haptics feedback, so feedback in real time from the environment that you have,
you know, virtually in or mixed reality in, um, is really cool.
And it is coming. I have seen full suits look a bit,
it looks a bit like a wetsuit and it has these sensors in it. Um,
there's also gloves that are super, super sensitive.
So everything in your fingers you can feel in real time with haptic gloves on.
And I think that's a really cool extra step.
I'm not exactly sure where that's gonna fit in.
Are people going to be gaming with a whole suit on, you know,
what's going to happen? I'm not sure, but I think it has place there somewhere.
It's very cool.
So thank you for watching this show is for you.
We want to know if you want to hear about different theme. Uh,
is there something that you want to hear about the, the,
that you're really passionate about? Did we do something that wasn't right.
Have you got questions?
Any kind of feedback is very welcome and we reward it all with holographic
kittens as well. Thanks for watching I'm Lars Klint,
and this is the Future of Tech.
The show that places me as a hologram in your living room.
Goodbye!
[Inaudible].