(energetic music)
- Hello and welcome to Microsoft Mechanics Live.
Coming up we're gonna take a demo rich tour
of the merger of AI, machine learning,
IOT and realtime analytics with Azure Databricks
and Mixed Reality through the HoloLens.
We're gonna walk you through a real world application
and then deconstruct how we can leverage the technology
in your environment.
To do that I'm joined by one of our all time favorites
on Microsoft Mechanics, Mr. Or M R Ben Walters.
(audience cheering)
- Thank you, Jeremy, great to be back.
- So over the past year we're seeing that
there's a lot more going on in the area of IOT
and machine learning, and Microsoft itself
has a comprehensive IOT platform
spanning device connectivity, management,
security, analytics.
We did a show actually with Jeff Volin
not too long ago where we turned all these capabilities
and to add to that now we can merge those capabilities
to build MR, Mixed Reality solutions on top of them.
- Yeah and this is a really interesting use case,
especially if you look at taking all that data
we have in the IOT devices out there in the field,
reasoning over that with cognitive services,
and building machine learning models
and then being able to simulate the real world
in a virtual world with Mixed Reality.
So let's jump straight on in
and I'll show you exactly how this comes together.
So what I have here is the opening webpage
for a system from a company called eSmart Systems.
And eSmart have actually partnered with Microsoft
to create this application.
And they're responsible for managing the power grids
across Europe.
They're really worried about maintaining
an up time at their power grid
and also ensuring that they keep people safe as well.
This is our field technician view,
and what this allows me to do is to be able to get
a high level view of my power grids.
I can see where there are problems and what's happening
and as I drill down I get more and more information.
Now eSmart Systems have deployed IOT sensors
across their power grid from substations to transformers
and also on those pylons as well.
And what this does is give them a view in real time
of what's happening in that power grid,
and so you can see, I can see things like grid load
I can see the temperature of my pylons.
I can also see line voltage down here in the bottom left
corner, across the day and what's happening there.
And because we have machine learning and analytics
I can also start to predict
where my line voltage is going to go
and you can see here we're actually on this blue line
about to spike over our maximum
which is really bad, could take down these higher grid.
- [Jeremy] It's not good probably
that we're seein' a lot of red here on this visual
in terms of all the things
that are about to break, effectively.
- [Ben] That's right, and it's one of those things
that we can go and provide things like prescriptive
analytics and the ability for people to be able to do things
so I could for example click on these pylons
and get a recommendation to reroute my power,
which buys me a little bit of time,
but honestly this grid's still at risk
and so we really need to figure out
how we're gonna go and address that.
- [Jeremy] And we're seein' here
we've got this voltage spike
in the lower left corner with live voltage,
so yeah they gotta get out there.
The other thing is to be able to go out there
you've gotta send technicians, right?
- Yeah, and previously we'd send engineers out,
they'd have to assess a power line,
figure out what the problem was.
But now what we're doing
is we're actually sending out drones
to be able to assess those power lines
and take a look at those as well.
And so what we have here is eSmart System
created this connect a drone platform.
These drones are equipped with a combination
of infrared and LIDAR scanners
and then they partnered those
with high definition cameras as well
to be able to run cognitive services
over the images they've taken of the power lines.
Now these images are passed through deep learning models
that are running locally on those drones.
As they start to find issues
or they identify issues in those pictures
they can then use the LIDAR scanners
and high def cameras to be able to take
a full 3D representation of the problem component
and then be able to address that offsite
before they send someone out there.
- [Jeremy] And now as an engineer
and in these cases such as, you're in Norway here
so it's not always as green and as warm
as it looks there, so it might make it tough
to send people out and it's not always easy
to kind of find the issues anyway.
Now you can use actually Mixed Reality in this case
with HoloLens to troubleshoot what's going on.
So, Ben, why don't you step into the future
and apply the HoloLens. - Put these on.
- And then we're gonna track you
to see what you can see here
with your app, as you attempt to troubleshoot
and fix the power grid.
- Perfect, alright let's do it.
- Let's do it.
- Okay so I'm gonna get you to grab this phone over here,
Jeremy, so you can project to everyone else
what I'm doing.
- Okay.
- Alright we'll see how it goes.
So now in front of me what you can see
is my virtual command console.
Across the top here I had those same feeds
that I had on my dashboard before,
of the IOT sensors bringing me
things like load, I can see the issues that are coming up.
I can also see info coming in from those drones
that are out there assessing the power lines.
So I've gone into that power line.
You can see all the images that have come across
from my drone here,
with the bounding boxes that are showing you
exactly where my problem is,
so as I said there a problem with one of my insulators,
so I'm gonna bring up that insulator.
And you'll notice now here is that 3D representation
that I was playing with earlier that you couldn't see,
and you can see with this I can now zoom this in
I can rotate that, this is a full 3D model
of that insulator that I was using before.
Now what I'm gonna do here is I'm gonna come back in
and I'm gonna select an alternative insulator
to use on my power line.
So now that I'm happy with the glass and ceramic insulator,
I'm gonna apply that to my power line,
and as I do that you'll see what we get now
is machine learning models that have been run
against that insulator, and other materials that we have,
and this gives me a view as to what is the best material
to use on this power line.
Now you'll notice we actually have an option in there
that has a zero percent failure rate
but it adds a whole bunch of load
to the power system overall
and potentially still puts my grid at risk,
but we'll notice the one that I selected
has 15% failure rate
but only adds an additional 20%,
so that seems like the one that I'm gonna use today.
So I'm gonna go ahead and create my ticket.
Swap out for my new insulator model.
And then just like that I'm gonna go
and submit my ticket to Dynamics 365,
so that my field service agent
can then use this to go and make that repair.
- Alright, very cool stuff, what do you guys think?
(Audience cheering)
Put that right back there.
So now back to the here and now,
so we saw a great example then
of how we can really get real time data
from the grid, and bring those into
you know, from the IOT devices,
bring those back to machine learning,
really deep learning from the edge for predictive analysis.
- Yeah and there are a number of technologies
really working together to make this possible.
What we've set out to do here is to give you
an entire platform with Azure
with IOT and also Databricks and machine learning,
and bring all of those devices at the edge together
to communicate and feed data back to a platform.
It also enables you to manage the health of those devices
and then apply predictive and intelligence to that.
Apply intelligence predictive insights to that
using machine learning and cognitive services
which can then also trigger workflows
as you so often see in things like Dynamic 365.
So what we have for this solution
is a number of IOT devices from our drones
to our pylons and our substations.
Then in capturing detailed images,
running cognitive services over those images,
and pushing that up into our Cosmos DB.
We then have that data fed through into Databricks,
which uses that information
to be able to build our machine learning models,
and then we can use that to test changes
before we apply them in the real world.
Of course all of that information
can then be used to render our 3D environment
so that when I'm working with those different objects
I can do that in a virtual world
before I go and submit that field ticket
off to Dynamics 365 field service.
- Very cool, very cool, so of course
in the future you'd be able to augment this solution
at the silicon level effectively using Azure Sphere.
And can you break this down though
and actually show us what the out code itself looks like
and what you had to do to build this stuff up?
- Yeah so what I have here is my Azure Databrick notebook.
And you can see as we go through here we've actually got
a number of things that we're doing.
As we scroll down I've got my code in here
where I'm loading my images from storage,
we run them through some pre processing,
we then use MMLSpark to be able to run across
those images and build that machine learning model.
And then as I scroll down here
you can see here is a sample image
that actually comes out of that MMLSpark model.
So we use all of our images,
and then as we come through
we can start to see where we detect anomalies,
and really break down those components of that image
to detect where there are issues.
And then once we have that machine learning model set up
we can then containerize that
and push that out to our drones.
- Great Ben, so you've really made this real for us
but where can people learn more
if they wanna actually get their hands on all this
and start trying it for themselves?
- So if you want an overview of AI machine learning
to get started, you can check out
the Azure essentials series on Mechanics.
Then we also have a number of other areas you can check out
aka.ms dot MixedReality, slash MixedReality,
slash GoDatabricks, and also the Microsoft.com
learn website has all of these resources
to get started.
- And of course keep checking back to Microsoft Mechanics
for the latest tech updates in reality
and in Mixed Reality.
Thanks for watching we'll see ya next time
and goodbye for now.
(audience clapping)
(energetic music)