(Mechanical music)
- Coming up, we look at Azure Stack Edge and
its growing application for AI
and machine learning workloads, with a proof of concept
that Heathrow Airport started to prevent the illegal
trafficking of wildlife parts, including endangered species,
by inspecting passenger luggage for positive matches
using Azure Custom Vision AI models
running locally on Azure Stack Edge.
All this, part of Microsoft's AI for Good program.
So today I'm joined by Matt McSpirit,
a longtime Microsoft Mechanics host,
also an expert in Azure Stack.
Thanks for joining us on Microsoft Mechanics.
- Hello, it's good to be back.
- And thanks for joining us from home today.
So, I know that Azure running locally is an area
that you're an expert in, as well.
But before we go further, for those that are new
to Azure Stack Edge, can you explain what that is?
- Yeah, absolutely, so Azure Stack Edge
is part of our Azure Stack family, which can be used
in a variety of ways.
So firstly, there's Azure Stack Hub,
which brings a select set of Azure services
to a location of your choice, enabling you
to leverage the power of Azure locally,
in a consistent manner, even completely disconnected
from the internet.
Then, with Azure Stack HCI, you can consolidate
virtualized apps in your data center, or remote site
on modern, hyperconverged infrastructure,
while easily extending up into Azure
with services like Azure Site Recovery,
Azure Monitor, Security Center, and more.
And finally, if you're deploying workloads
such as machine learning to the Edge,
and need to avoid the latency caused
by roundtripping data up to the cloud and back
for processing, you can deploy Azure Stack Edge,
placing compute power close to where you need it
so that hardware accelerated data processing
and AI inferencing can all be done locally.
Now, the big advantage here is that management
is all done remotely and centrally through Azure.
- Okay, and I know that we're starting to see
different applications of the Stack
really across industries, but how is Azure Stack Edge
then being piloted at London's Heathrow Airport?
- So, the Azure AI Customer Engineering Team,
supported by the Microsoft AI For Good program
is working with Heathrow on a pilot
to crack down on illegal wildlife trafficking
which today is a billion-dollar industry.
And what you can see here on the screen is a room
at Heathrow Airport filled with some of the items
that have been confiscated over the years,
and whether that's from animals such as rhinos
or leopards, which are sought after for their body parts,
or animals that are become endangered, such as
pangolins, considered medicinal and a delicacy
in some cultures.
And right now, today,
the UK Border Force typically profile passengers
based on flight routes that they may be taking
which are known for trafficking before going on
and searching their bags.
- Got it, so it sounds like it's pretty manual,
and not a scalable way about doing things.
- Exactly right, and in fact, even when luggage is scanned
using powerful 3D X-ray technology,
it's actually pretty hard to identify these illegal items.
So, for example, what I've got here is
a raw scanner image, and you can see
that there's a large object in the bag,
but it really requires an expert to decipher
what it really is, so they want to be able to use
machine learning to be able to do the initial detection
leading to a bag search by the agents.
- Okay, and I can imagine it would also
cut out a lot of the time that you would also take
in terms of going through all these different images,
and it would optimize that process, also,
given just all the sheer magnitude of bags,
the thousands of bags that are going through
at peak periods in the airport, but how are they looking
to potentially use technology, then,
to address the problem?
- So, the Azure AI Customer Engineering Team
has been working with Heathrow to build a solution
comprising a 3D scanner combined with AI models
created in Azure, and they're running those models
locally on the Azure Stack Edge, so here's how it works.
Once the luggage passes through the baggage scanner,
it produces a 3D BTK image file.
And this is open source technology.
It's the same tech that's used for CT scans in hospitals.
Now, each scan's about 50 meg in size,
and the scanner saves that BTK file to its local storage.
Now, at the same time, we've deployed
the Azure Stack Edge for local compute,
and this has got four containers:
sync, pre-processing, model, and broker.
Every 200 milliseconds, the sync container
copies new scans from the X-ray scanner
to the Azure Stack Edge device.
The pre-processing container rotates the image
into different angles, and it also filters the image
based on particle density to filter out objects
that are out of our range of interest
such as high-density objects like metal,
or low-density objects like clothes.
Now, we're looking for bones, and this has a medium density,
so we can quickly detect organic matter.
The model container then runs an AI model
to investigate all the objects that have particle density
similar to bones.
Now, like a zoologist, it's been trained
to determine that it's definitively a skull
of the type we're looking for, rather than another object
that might be mistaken for a skull.
And once the model has made its verdict,
the broker container manages the communication
among the other containers all on that
Azure Stack Edge device.
The AI model's verdict is stored in an output folder
which is continuously synced with Azure Blob Storage.
And we then use an Azure Logic App to update
in Azure SQL database with the results.
And this can then be read by an airport agent
on their mobile Power App to notify them
of the likelihood of a positive match
so that they can then go and investigate further.
And this whole process just takes a few seconds.
- Okay, so what was the approach then
being used to build out and train these models?
- So, data scientists built the model
using the Azure Custom Vision Service.
And this is a powerful, pre-built machine learning model
that Microsoft data scientists have built
that you can customize to your specific needs.
Now, here, we've uploaded quite a few images
to our dataset to pre-train the model,
and if I click into one of these, you'll notice
that we've notated the image to train the model.
And I can click around so you get an idea
of what this looks like.
I can also add more images to the dataset,
and you'll see here I just uploaded two images
that can be found at the bottom here.
And I'll select this one, and to train and label
the objects, I just need to place focus on it,
and draw a rectangle around it.
And in this case, because I trained the model previously,
the model recognizes the object, and it can be used
to make predictions on the dataset.
And now that I've added more data, I can go ahead
and train the model by selecting train,
and this is going to take around five minutes to complete.
So I've done this in advance, and here,
you can see the output here.
Now, there are two important performance metrics
to call out: precision and recall.
Now, in this case, we have a high-precision score
of 93.6%, which means we aren't holding up
airport security lines by falsely accusing
travelers of smuggling.
Now, at the same time, our recall metric score
of 81.7% is a measure of how well the model has performed
in identifying what we're looking for.
In this case, animal skulls.
Now, both are high scores, which is great,
but to put this to the test, I'll bring in an image
that I saved locally to see if the model recognizes
what it's seeing.
And here, you can see the model
is 99.8% sure that there's a skull in the bag.
But at the end of the day, this is an ongoing battle.
Traffickers are always going to find ways
to disguise their efforts, so we need to continually
retrain the models to ensure the high performance
of the service.
- How do you then, or what do you need to deploy,
I guess, in order to get that onto
the Azure Stack Edge device?
- Right, so once I'm happy with the performance,
I can export the model to run on our Edge device.
So here, you can see a complete set of options.
The ONYX run time, for example, is useful
if you have a diverse set of execution environments.
Now, I'm going to select the Docker container format
to deploy my ML model as the container
on my Azure Stack Edge device.
We're running Linux on this Edge device,
so I'll pick Linux as my version, and then,
select export, and this is going to take awhile,
so I've done this already.
Now instead, I'll switch over to Visual Studio Code,
to show you how we can use the IoT extension
to update the Edge device by selecting build,
and push IoT Edge solution.
Now it knows to look for a deployment template.
Since we only have one in our repository,
it selects that one automatically.
But you can see here that it runs very quickly
because in this case, it's just updating the model.
It's looking into the various Docker files we have,
and checking for any changes, but because only the model
has changed, and not the other containers, you'll see
that it's returned some layer already exists messages.
But in the background, it's built a template
with the updated Docker images, and pushes them up
to the Azure container registry so that they're available
for our Edge device to then pull them down.
Now, now that that's done, I can deploy this
to our target Edge device.
I'll look for the template that we just created,
which the, which has the configurations of each container
and their relationships, and this runs almost instantly.
And we see a message here that the deployment has succeeded.
- Okay, so, what's actually going on then
on the Edge device itself?
- If I go to IoT Hub, and I click into IoT Edge,
and select our IoT Edge device, you can see here
our four containers, sync, model, pre-processing,
and broker are all running, and we can also see
the Edge agent, which allows us to manage the containers
on the device remotely and centrally from IoT Hub.
- How do we know then that all of this is actually
up and running, and working?
- As we saw, you can see the Edge containers running,
but we can also SSH into the Edge device to see the logs.
And in fact, we can see in this section here,
the modules on the Edge device that have been stopped
and restarted, which is a good sign
that things are running.
Now, we can also look at the output
of the pre-processing module, so we're simulating
the scanner, placing an image into the folder
on Azure Stack Edge every few milliseconds,
it syncs for a second, and then, copies a new file over.
And the pre-processing container then identifies
these new files, and sends them over to our
custom vision machine learning model for identification.
Next, I can also look at the log file of our model,
and we can see the activity, where it's assessed the images,
and it's returning its predictions.
It writes a verdict to an output folder that syncs
to Azure Blob Storage, as I described earlier.
And here is our Azure Logic App that monitors the container
in Azure Blob Storage for the new content.
It lists the files, and parses the JSON,
and for every entry in the JSON file,
it populates a field that's updated in our SQL database.
This is connected to the power app
used by our Border Control agents who get alerted
of the positive results.
A switchover to the power app, you can see the experience
for the Border Control agent on their cellphone
or tablet, and as I hit enter, you can see the results
of the last runs, and I can see the results
of the most recent runs, I can view the images
of the suitcase and determine for myself
whether or not this looks suspicious,
and, or whether or not this looks like a positive match.
And if it's correct, I can confirm
by selecting item identified, and this now can be added
to the training dataset.
- Okay, so, what's the setup and management experience
then look like here?
- Well, before we set up and manage the device,
we need to order one, so here I am in the Azure Portal.
And I'm on the Azure Stack Edge getting started page,
and you can see, I just provide a subscription,
where the device will be registered for billing,
and the country for shipping.
When I click show devices, you'll see
I've got a couple of choices.
The currently available Azure Stack Edge
with FPGA, but you'll also notice here in preview
is the Azure Stack Edge with GPU,
which will ship with an Nvidia T4 GPU
to handle the hardware acceleration,
something we're all incredibly excited about.
Now, the final steps involve providing
a few more details about the resource group in Azure,
a shipping address, and that's it.
The device will show up not long after.
Now, once received, connected, activated, and updated,
you'll have a couple of simple steps to perform
in order to configure the Azure Stack Edge device
with compute capabilities.
You'll want to follow the guided wizards,
along with our docs to set up your scenario,
which, in our case, is Compute.
So under Compute, it first advises you to obtain modules
from the Azure Marketplace, or publish your own modules
or containers to the Azure Container Registry
or Docker Hub.
Now, we used Visual Studio Code for this earlier,
which made things really easy, and from there,
we guide you through connecting to
either an existing or a new IoT Hub to communicate
with the IoT Edge runtime running on the device itself.
Now, this takes a few minutes, but you can see here
my completed configuration.
Now, depending on your scenario, you might want to set up
some file shares on the device.
They're required if you want to process
and then potentially upload data
like we saw with Heathrow, and you'll see
it's simple wizard, too.
Add a name, type, storage account, user privilege,
very easy stuff.
You're then at a point where you're actually
deploying the modules or containers
as we saw earlier, and all of this is done remotely
and centrally from the Azure Portal.
- This is really amazing stuff, and a great application
of AI for Good, but what's next, then, for this?
- So, we're working to customize the models further
to reduce false positives, and apply them
to a broader range of wildlife.
We started with animal skulls, but we're now extending
the effort to identify pangolins, and bats.
And we plan to share the solution patent and AI models
we've developed with other interested airports.
And lastly, one other positive outcome
besides benefiting wildlife is, we're working
with Heathrow to look at how this new capability
can be used to combat other high-harm threat types.
For instance, weapons carried in baggage.
- So, for those people that are interested
that are watching today and want to build out
similar solutions, where would you recommend they go
to get started?
- Well, firstly, I recommend getting familiar with IoT Edge,
and you can learn more at AKA.MS/IoTEdge.
Also, you can order an Azure Stack Edge device
and try it out for yourself at AKA.MS/StackEdge.
And if you've got a project for our Azure AI
Customer Engineering Team, and AI for Good program
to consider, we've placed their contact details
in the description of this video.
- Thanks again for joining us today, Matt,
and also, of course, keep watching Microsoft Mechanics
for the latest tech updates across Microsoft.
That's all the time we have for this show.
See you next time.
Goodbye for now.
(mechanical music)