>> You're not going to want to miss
this episode of the AI Show where we talk about
a brand new component of
the computer vision service called spatial analysis.
It's very timely, it's very needed. Make sure you tune in.
[MUSIC]
>> Hello. Welcome to this episode of the AI Show.
We're going to look at a brand new service
inside of the computer vision service called spatial analysis.
We've got a special guest with us today.
Hello, my friend. Why don't you
tell us who you are and what you do?
>> Hi there. My name is Adina Trufinescu.
I'm a Program Manager.
I'm working on computer vision,
which is on Azure Cognitive Service.
I'm here to talk about spatial analysis. Supper exciting,.
>> Awesome. Why don't you give us
a rundown of what it actually does,
what's the purpose of the service?
>> First, I want to tell you about computer vision,
which the existing services for computer vision are image-based.
Basically you pass an image in,
the AI is running and it's giving you a result.
Spatial analysis is a new feature on
the computer vision service and it is actually video-based.
We run computer vision AI on video streams.
They can be live video streams or
they can be recorded video streams.
Then there are actually three use
main scenarios that we're looking at.
The first one is people counting which
enables you to count people in a video.
This is mainly used for scenarios such as
maximum occupancy or when people shouldn't be in a certain area,
this is how you can monitor for safety scenarios.
The second one is social distancing.
We actually call it social distancing
because this is what resonates with people,
the actual operation namely
the documentation is person distance where we
actually count the people in the video stream
and we detect when people are either too close or too far.
The too close scenario is
the interesting one for social distancing.
Then the third one is the entry and exit in physical spaces.
Imagine that you have a camera and in the field of view,
you want to monitor a certain area in the physical space,
and you designate that area with a polygon or with the line and
we are giving the events when
a person is crossing in or out of the polygon,
or a person is crossing the line and the line is directional,
so this way you can tell when the person enter or exit a space.
>> This is really cool.
Those three scenarios seem pretty basic.
What are some good use case scenarios for a business to use these?
I know you mentioned them briefly,
but maybe you can elaborate a little bit more on that.
>> There are many industries and use cases,
but the ones that we are focusing on based on the input from
customers are retail manufacturing and real estate management.
For retail, retails they want to
know where people move through the store.
They want to know where people spend time in front
of a display or how long they wait at
the place where they check out and buy things, in the queue.
Based on this information,
they're looking to understand how to modify the store layout,
or more recently how to help people apply social distance.
Retail is a big industry.
Our own dynamics connected store have
already a SAS application which is using spatial analysis.
These works, we know that it works for retail.
When our developers want to build their own business logic,
which is more custom to the needs of their customers,
they can use spatial analysis for retail.
In manufacturing plants, they have
all these areas where it's not safe for people to actually be in.
Basically this is where,
with a camera and the video stream,
they can monitor when people passing through a certain zone,
which they call forbidden zone,
and basically they can notify an alert,
the safety personnel or the people
actually involved, the employees involved.
Then by saving these data over time,
they can actually create reports and meaningful information
that it helps them
create like guidance for safety of their employees.
Then the last one is real estate management.
Especially with the back-to-work situation,
companies are looking to be agile and to figure out
how do they help their employees stay healthy and stay safe.
This is where social distancing comes
into place and also maximum occupancy.
This is where companies are
looking at solutions to figure out the number of
people is actually constrained to
something that is actually healthy and safe.
>> Here's a question, because obviously you're
taking video of people.
How do you manage privacy concerns?
>> That's a very important one.
There are two angles to look at it.
First of all is the technology angle.
Spatial analysis, it's a docker container,
it draw us at the edge.
Our customers, they want control over
the data and they want the data to stay within their premises.
They also want to make sure that
the network workloads in terms of
video leaving their premises is something that they control.
This is where we started with
spatial analysis as the Docker container.
Like most of Azure services,
they start with an Azure service in
the Cloud and then it moves as a container.
Because of these customer input where
they're really concerned about having
control over the video data and
then meeting the privacy requirements,
we decided to start with spatial analysis
at the Edge with a Docker container.
That's the first aspect.
The second aspect is the responsible AI aspect.
When we released these technology into the world,
we just don't put it out there.
We actually release responsible AI guidance.
This is guidance which is based on
user research and societal research.
When we worked with our customer
in private preview to collect their input,
to look at the use cases,
basically we came up with a set of guidance of
what represents responsible use of these AI.
Then basically, what we hope to achieve is
that our developers can build these business applications and
use spatial analysis while upholding the dignity of
the end users involved with this technology.
>> This is all really cool, but I feel like I want to see,
get a taste for how it works. Can we do that?
>> Absolutely. Let's go for it.
What you see here is
a video in a space that mimics a retail space.
Then you can see the people are being detected.
This is the people counting operations.
The people count here is six.
It doesn't change because we have six people at the same time.
Notice here there is a trail behind each person.
This is to show that the person is being
actually detected across multiple frames.
These one runs at 15 frames per second.
This is an unnatural counting people.
In these case, we do not have a zone designated,
we're counting the people in the whole frame.
Moving on, let me show you social distancing.
This is the same camera feeds from the same location.
Notice here that we detect people
and then we measured the distance between people.
To actually measure the physical distance between people,
the cameras they have to be
calibrated to the ground floor and this way,
the measurement between people is
actually accurate and relevant in the physical space.
Then the last thing I'm going to say here is that we
provide the information whether
the people are too close or too far,
and then you deciding
your business application how to react to that information.
Then the last one is entry-exit.
This one is another retail space.
Notice here we have an area or zone in blue.
That zone, it's an imaginary one and it is being
drawn using a frame that you collect from the camera.
Then you throw that zone,
in this case around the area where people wait at
the checkout stand and then notice there
that there is a time that the person is waiting in lines.
The way we achieve that is
actually when the person enters into the zone,
we have the timestamp of that event.
Then when the person leaves the zone,
we have another timestamp.
This way we're able to calculate
the time that people wait in line.
This is another use case for entry-exit.
I do not have one for line,
but hopefully this gives you a good idea of these operations.
>> This is amazing. As a software person
or as someone that wants to implement this,
how is this implemented in Azure, for example?
>> Absolutely. Let's go to the Azure portal.
What I have here is
an IoT Hub service and use that in Azure that we set up.
To deploy the spatial analysis container,
you need Azure IoT Hub and you need an Azure IoT Edge device.
The device is that we support they
required to have an NVIDIA T4 GPU and we
recommend to use Azure Stack Edge with
GPU or you can use your own device as long as it has a GPU.
The Edge device shows up here.
It's already pre-configured and connected to this IoT Hub.
What I want to show you here is
the spatial analysis container
which is being deployed as an Azure IoT Hub.
I'm going to go there.
In the Module Identity Twin,
we pass actually the configuration for the operation.
Let me scroll here and I can show you here these configuration.
You can see the operation ID.
In this case, it's spatial analysis person count.
You can see the video URL that is being processed.
In this case, it's a 3D coordinate URL.
You can see the GPU index on which you're running.
That's why I mentioned that you need a machine with a GPU because
of the computational needs of the AN model.
Then if the machine has more than one GPU,
you have to load balance the cameras
across the GPUs to not overload a single GPU.
Then last but not least the zone configuration.
There is this JSON packed in here neatly
where you see the polygon and then you can
see here that the type of event that you
require is this person count and you can
also give us a threshold telling us
that if the AI confidence is not high enough,
just discard the event.
Then just to show you
a simple web application that is receiving the person count event,
which shows here in real-time
the number of people in the video going,
in this case between 0-2 people.
>> Well, this is actually really
cool and it's actually really timely.
Are there customers that are using it right now?
>> Yes, we are working with ANC,
a company in India who's working to create
a pandemic compliance solution using these exact skills.
Basically, they are looking at
deploying this in their office spaces
to help with obviously the pandemic compliance.
Then we have a bunch of
other customers in the real estate management.
Then last but not least,
I've already mentioned about Dynamics Connected
Store and their own retailers.
>> Well, where could people go to find out more?
>> Go to aka.msspatialanalysis.
That container is published on Microsoft Container Registry,
anybody can get it.
The access to the container functionality is gated,
meaning that we need
your information to give you access to the functionality.
It goes back to the responsible AI considerations.
We want to learn about use cases.
We want to make sure that the use case is
followed the responsible AI guidance.
Then once we approve,
the use case will be in contact with you and then
you'll take it from there and use the technology.
>> Well, awesome. Thank you so
much for spending some time with us.
For those that are watching,
thank you so much for watching.
We've been learning all about Spatial Analysis,
a new service inside of Computer Vision on the AI Show.
Thanks for watching and we'll see you next time. Take care.
[MUSIC]