>> Would you believe me if I told you that the video
seeing behind is eight video streams,
HD1s analyzed in real time by a
tiny $100 embedded device right here?
Well, Emanuel from the IoT Edge team is here to demo us
how it's done with the NVIDIA Jetson Nano and DeepStream.
[MUSIC].
Hi everyone. Thanks watching the IoT Show.
I'm Olivia, your host.
Today, we will talk about computer vision in IoT scenarios.
Emanuel is back on the IoT Show.
Thanks for coming back, Emanuel.
>> Yes. Thanks for inviting me again.
>> So for those who didn't see the previous episodes,
can you tell us a bit about yourself and the team you work with.
>> I'm a Senior Program Manager
on the Azure IoT Edge for a team.
I've been focusing on AI for a while.
>> So AI at the Edge.
>> At the Edge.
>> Awesome. So one of the AI usage is around computer vision.
>> Correct.
>> Computer vision which is about analyzing a video streamed live,
to extract insight information like counts people,
detect objects and things like that.
>> Really, in an IoT context,
it really means transforming cameras into sensors.
So transferring raw video feeds into events that
another module or another Cloud service can
make sense of without sending tons of data.
>> So very concretely,
so we have the power of the Cloud,
you can have infinite things up there,
in terms of computing storage and so on.
So what would actually be the need for
analyzing and running computer vision at the Edge?
>> Well, although raw video feeds are taken by
cameras are taking up a lot of bandwidth.
So they could be very large file size.
Also, you may not want to share
all those raw video feeds for
privacy concerns or sensitive data being in them.
So most of the time,
it's much more efficient and
cheaper to do all the processing locally,
next to where the cameras are,
and only forward events to other services.
>> So this is all good.
We've been talking about bringing an intelligence as
applied to the Edge for some time already.
So when it comes to computer vision,
what I'm hearing, let's imagine,
I'm a manufacturer of these devices.
What I'm hearing is, change your hardware,
put very smart cameras that have the computing size,
like you have the Qualcomm camera here as one of our deaf kits.
>> Yes. So these types of camera.
>> Yes.
>> In many cases,
it makes sense when you want to
constantly monitor a candy dispenser.
So when you've got machines that
you expect them to be very spread out,
then you don't have a lot of choice,
and it's new equipment.
Then going for a new camera that can
do the AR processing is a great choice.
But there are also lots of applications
where existing cameras are already deployed,
and they are not capable of running AI algorithm.
>> They're just capturing the video.
>> They're just capturing videos.
So these types of cameras.
It's just a basic RTSP camera.
It captures images and sends them via an RTSP feed.
>> Okay.
>> It's not capable of running any AI algorithm.
>> So our audience have heard about IoT Edge.
So I guess there's a solution using
IoT Edge for actually doing something right.
>> Yes, of course. So when you've got many existing cameras,
or a lot of cameras on a dense side,
then instead of changing all your cameras,
bringing one new IoT Edge device is a common approach.
>> Yes.
>> So this type of Jetson Nano device that we have here.
>> You might have heard about the recent announcements
surrounding NVIDIA collaborating with Microsoft,
or computer vision for the Edge.
>> The EdX platform,
which is a set of guidelines and tools for
hardware manufacturers and software developers
to build those applications,
where all the algorithm is running on
the Edge Server, on Edge Gateway.
>> Okay. So here actually,
the Jetson Nano DevKit is
a very cheap board that can do very advanced AI?
>> Right. It cost about $100. It can run.
It's pretty impressive in terms of what it can do to
run the deep neural networks AI computer vision.
So personally, I think of it as a recipe pie for AI.
>> Okay.
>> Because of it's price point,
because of its AI capabilities ,
it's a really fantastic device to get started with.
>> Well, how about we look at how that works.
>> Of course it runs IoT Edge.
>> I wouldn't expect last from you.
So show me a bit about,
what would it take to actually make something
happen here without the Edge,
and how computer vision actually
really would work in that scenarios we have,
like several video streams going into that device,
and that device to extract the insights directly at the edge.
>> So I'll demo in
a minute about how many video streams it can process concurrently.
Because at the end of the day,
to lower your hardware cost,
you want to be able to process as many video streams as possible.
So the smaller your algorithms are to
process them as a higher number of video streams are better.
So that's why we've partnered with NVIDIA.
So they've got a toolkit called DeepStream.
>> Okay.
>> Which is a toolkit to
help you build your own video analytics pipeline.
>> Okay. Without being a data scientist?
>> So it's not so much on the AI side.
>> Okay.
>> It's more for developers to
stitch together a very efficient video pipeline.
>> Okay.
>> It assumes that there is
a Data Scientist that gives you an AI model.
>> Okay.
>> So you gives this AI model to DeepStream,
and you can configure DeepStream
to ingest as perfectly as possible a high number of streams,
to process some streams and not others,
and to run a first AI model and then the second one,
to have cascading AI models.
So this is the types of applications of DeepStream.
>> Okay.
>> It really shines when you
need a super high-performance like real time.
>> Okay.
>> Most of real-time cameras are like 60 frames per seconds.
>> Yes.
>> Most of them are HD. So 1080p.
So processing 1080p 30 times per second is quite intensive.
So you have to write your code in a really efficient way.
>> Yes.
>> This is one of the things that DeepStream helps you with.
>> Okay.
>> It also helps you connecting to large number of cameras
when you want almost real time,
maybe one frame a second,
but over 50 cameras.
>> Okay.
>> Or when you need multiple AI models.
So first, you want to detect that there's
a worker in the construction site,
and then from this worker,
you want to see if he's wearing his helmet or
his safety equipment like gloves or glasses.
>> Okay, fair enough.
>> So that's really where DeepStream is super powerful,
and it uses really all kind of
optimizations that are built for you.
Of course using NVIDIA GPUs to get to that point.
So it does things like zero in-memory copy.
>> Okay.
>> It does things like pushing as much as it tends to the GPU,
while avoiding all the access conflicts.
It does things like tracking
algorithms so that you can do inferences,
and most of the time but sometimes instead of doing inferences,
doing tracking, which is a bit cheaper computationally.
So all of these things are things that DeepStream builds for you.
>> Okay. So is it like a set of APIs,
set of very simple [inaudible].
>> Yes, it has some APIs.
So instead of pipelines, it's plug-in based.
It's based on just rumor for people [inaudible] video development.
>> Okay.
>> Just rumor has been an open source
video tooling for a while now,
it's based on this tool.
So it's lets you develop your own set of plug-ins,
and NVIDIA gives you a lot
of trivial plug-ins that do those things efficiently.
But you can write your own on top of it, it's very modular.
So one thing that we've partnered with them is to have one
plug-in that sends all the output messages out of DeepStream,
and forwards all of that to IoT Edge.
>> Okay, awesome.
So in a form of an IoT Edge module.
>> So we've also partnered with
them to of course containerize DeepStream.
So DeepStream can be containerized and push all those messages,
at one of those plug-ins,
within the containers, that pushes all those messages to IoT Edge.
>> Okay. Got it.
>> So to make things easier for Azure customers to get side with,
they actually published their DeepStream offer
into our Azure Marketplace.
>> Okay.
>> So this making it really easy to get started with DeepStream.
>> Okay. I think you have VS Code open.
>> Right. So let's do a demo.
>> Let's look at what it looks like.
>> Okay. So, yes.
I'm in the shoes of a developer,
DeepStream is SDK again,
so like bagging developers.
So I've just opened an empty IoT Edge solution.
>> Okay.
>> VS Code is the IoT Edge developers natural choice.
>> Yes.
>> So here's four of these anti solution.
What I'll do is just add a new module.
>> Okay.
>> DeepStream from the marketplace.
>> So that's an interesting thing,
like embed a new module,
add a new module in VS Code directly from the marketplace.
You don't have to go and find the URI or not.
>> Right.
>> [inaudible]
>> So here I can,
is there a start from a blank canvas in C,
C#, Python and so on,
or other module from the marketplace.
>> Awesome.
>> So this is the option I'm selecting now,
going through all the marketplace offering
and DeepStream SDKs right here.
>> Nice. This is available today.
>> It's available today.
>> The latest Azure IT tools in VS coding have all that.
>> So I'm selecting Jetson,
so there is one versions called Deep Zone,
which is NVIDIA's embedded class of devices.
>> Okay.
>> Another Container for several class NVIDIA GPUs.
>> Okay.
>> So study things the Jetson one,
and now my empty solution
as, of course, adjacent as before,
but now it this new and if you get
deep steam SDK coming from the marketplace.
>> Okay.
>> With all the default settings
preset like entry points and host config to run.
This is to get access to
the Nvidia GPUs and the working directory, right?
>> Awesome.
>> So I could just deploy that and push it on
my Nano and get started with it right.
>> Okay.
>> So I went a bit further
like to push the boundaries of the Nano.
So by default this module from
the marketplace tries to process four video streams.
>> Okay.
>> Let's go to eight.
>> Eight. Okay.
>> Which is the maximum of these Nano, right?
>> Okay.
>> Also by default it doesn't push the output to the screen.
>> Okay.
>> So I've changed the Config File of DeepStream a little bit,
which I have in this solution
here just to gain a little bit of time.
>> Okay.
>> So here's an example of a deep steam file, configuration file.
So this is my source videos.
>> So you are feeding from local MP4 file?
>> Yes I'm simulating eight video streams for now.
>> Okay.
>> But it could also be like live at this P cameras.
You just change the URL and it will be RTSP streams.
>> Okay.
>> Right. Then, so this is my source my list of sources.
>> Okay.
>> At the end, I've got my Sync which by
default is a fake Sync like no display essentially.
>> Okay.
>> Number two is a display. Right.
>> Okay.
>> So displaying on screen.
>> Okay.
>> Auto Sync here is to say and send all the output to
message broker and which has its own configuration file.
That is as you get by
the name to send all the messages to the Azure IoT Edge runtime.
>> Got it. Right. Okay. That's by default or?
>> That's by default. The author
in the marketplace as is by default.
>> Okay. Got it.
>> Right, and so this is like
all settings that you can change and the trick here right.
These last settings I'll point out in
this configuration file is the AI model itself.
>> Okay.
>> So this is one that is by default as a sample.
>> Okay.
>> But of course like data scientists could give you
one and you could include that into your own video analytics.
>> Do you know what the format has to be?
>> Yeah. So in the end,
DeepStream relies on console RT
which is Nvidia inferencing engine.
The good news is that Tuncer RT supports a lot
of model formats on mobile networks.
So most likely, you
don't need to change anything as a data scientist.
You can use the same thing.
>> Okay.
>> It supports onyx consent and support.
So you can build and use
Azure Machine Learning and custom vision to build
your models and then deploy that and have them run the DeepStream.
>> Usually you have this fine which is local in the machine
and let the module actually just leverages locally right.
>> So this is more like the developer cycle.
So you typically add the file
locally on the machine so that you can change any iterate quickly.
Once you're ready to go to scale,
you would embed that into the DeepStream container.
>> Okay.
>> So you'd base your container of the one from the marketplace,
and add one more layer with your AI model.
>> Got it.
>> So that you can control that at scale.
When you've got an update to make to your AI model,
you can just update
this last layer on your docker container which is your base.
>> Okay. Easy.
>> Right.
>> Straight forward.
>> Right.
>> Yeah. Then it's just IoT Edge right?
>> In the end it's just IoT Edge.
So really deep steam in
IoT Edge complement each other really well.
>> Yeah.
>> So DeepStream takes care of fluxes of performances.
There's a video processing and output messages.
IoT Edge takes care of the connectivity to the Cloud,
takes care of the security,
takes care of deploying the container,
and managing that at scale right.
>> Yeah.
>> So it's a complement to choose us very well.
>> Yeah.
>> Right. So this is to give the Config Files that
I've tweaked so that we can switch to the actual demo.
>> Okay. Let's see how it looks like on a device that's running.
>> Right. I've already deployed DeepStreams there,
to save the download time.
>> Okay.
>> So here you can see the entire application running.
So IoT Edge is running.
DeepStream as a container has been deployed via IoT Edge.
It's processing eight video streams,
eight different video streams.
>> In real time.
>> Real time. So it's like those are ten ATP feeds.
>> Yeah.
>> All at 30 frames per second.
>> Awesome.
>> It's striking cars and people and then
sending messages to the edge runtime.
>> Yeah.
>> Then I've got around my IoT Edge runtime to forward like that.
>> Upstream and then in applied
to can do the counting and have them like that.
>> Yeah. You totally do actually first filtering
and business logic.
>> Okay.
>> As an extra module on your IoT Edge device itself.
To batch your messages to do
bit more processing on the event itself not on the images,
and then forwards and aggregate to the account.
>> Awesome. I love it.
Pretty straightforward.
>> Yeah.
>> Impressive. I remember we talking about,
"hey you can't run that many feed,
and analyze them on a tiny device." Well, here it is.
>> Right.
>> I love it. Actually I
want to see this messages going up to the Cloud.
>> You don't trust me, huh?
>> I do trust you. I want people to see them, right?
>> Okay.
>> So let's switch back to the pieces.
>> Right.
>> So in [inaudible] , I can seal
my edge devices that I connected to my IoT Hub.
Yes it's Azure IoT Edge extension.
I can start monitoring
all the messages flowing from the Edge device,
from one edge device to Edge Hub itself right.
So it's preparing the monitoring drove, and here,
you can see all the messages being
sent from the device of this account.
>> The objects have been detected.
>> RIght.
>> Some information about them, right?
>> Right. So it sends you essentially the bounding box for each.
It will send you the type.
A car, a person has been detected.
>> Okay.
>> The bounding box about where Cisco are personalized there.
>> Very impressive, Emanuel. Thanks a lot.
I think that was very insightful.
We've been talking about this kind of things.
I remember last year at Build,
what we're actually talking about
these multiple video feeds analyze on
this edge device, tiny edge device.
>> Right.
>> It's reality. You guys can actually try it out today.
If you want to learn more about computer vision and DeepStream,
you can go to aka.ms/IoTShow/DeepStream and you'll get
all the information about this demo using
the Nvidia Jetson Nano Dev Kit,
and the fantastic demo that Emanuel put together.
>> Yeah.
>> Thanks a lot for this incites, Emanuel.
>> Yeah. Thanks.
>> See you soon on the IoT Show again.
You as well. See you soon. Don't forget to subscribe.
[MUSIC]