>> Hey friends, Azure Cognitive Services
offers the most comprehensive set of
pre-built AI capabilities that you can
include in your applications including speech,
vision, language, web search, and decision.
Now they are available in containers,
so you can use them when the data
that you have can't make it to the Cloud.
Christina is here to show me how today, on Azure Friday.
[MUSIC]
>> Hey friends, I'm Scott Hanselman and it's on Azure Friday.
I'm here talking with Christina about Azure Cognitive Services.
We're going to do an update about some of
the great new stuff that's built in.
>> Great. So Azure Cognitive Services gives you
a comprehensive suite of pre-built AI capabilities.
What that means is that a lot of developers want to add
AI capabilities into their apps or websites that they're
developing so that they can give more value to their customers,
but a lot of times, they don't have
data scientists or extensive expertise,
you need to build a really good machine-learning models.
So what we do is we build
these pre-trained AI models
that all these MSR researchers have contributed to.
So these are bleeding edge AI models and then
the developers just take those and then put it into their apps.
>> These are legit,
these are not little cute demo things that I can play with,
these are like real models I could put in production?
>> Yes. So we have a bunch of these capabilities,
spanning different categories like
vision, speech, decision, language,
and search, and we're adding more constantly.
One of the newest ones we have is receipt understanding.
So I have just come back from
a conference and I have to do with expenses now,
and a lot of times when you do expenses,
you have to take a picture of your receipt,
put it on your expense tool,
and then also fill it in just every single detail.
>> It sucks.
>> Right. So one of the things that we want
to do is make that easier for people.
If you're ready to have your receipts,
then you don't really have to type it all over again.
So that's what we want to do.
So I'm going to use this tool called Intelligent Kiosk,
which is just a bunch of pre-built demos built-in,
but like these APIs are available in real life as well.
So let's see.
So this is a receipt from a few years ago that I had.
So a Michael's receipt for some craft stuff I bought,
and then within seconds,
you see that it got them merchant right,
the Michael Store address.
>> Hang along, you're hovering over it,
it's telling you where it found the word too.
>> Yeah. So like Michael Store 8696 because that's the store name,
the address, the phone number,
the date and time, and it also normalize everything.
So a lot of times when you have receipt,
sometimes they do 15:01 or sometimes they do 3:01 pm.
So you can normalize everything so that when it's in your system,
it just knows exactly what it's talking about.
Then the subtotal, the tax, and the totals,
they got all of the detail just from just one picture.
You can also do this with,
so this demo I just did it from an image that I already had,
but if you wanted to do it with a camera,
then you can just, from camera, let's see.
So this is the receipt that I got very recently.
So I have this receipt and I'm going to take a picture of it,
it's covered the selfie camera,
but let's see if this works. It worked.
>> Twenty-five, 34, did you hand write that?
>> Yeah. Well, that actually worked.
My handwriting's really bad.
>> So it found a sideways handwritten receipt
and figured out the title?
>> Exactly.
>> Is the total, that's fantastic.
You're going to make the Microsoft
expense reporting people look bad,
it's like we don't have that yet.
So the customers and the public are going to be able to
play with this on Azure before we do on MS expense.
>> Right. So there are going to try to build it into MS expense.
>> My goodness, that's so awesome.
>> Yeah. So that's one thing.
So we have all these cool capabilities that we have.
But a lot of times, the problem is with customers,
they don't want their data going into Azure,
for regulatory reasons or
for just privacy reasons, they are concerned about.
So they don't want their data going to Azure Cloud.
So what we have for those customers
as Cognitive Services on containers.
So you can deploy a container in
your own datacenter or wherever you want your container to be,
and so your data never leaves
your compliance boundary so
that you can run everything to same model.
So it's not a stripped down version
of those models, is the same model.
So you can call the same API,
just point it to your local host
or whatever your datacenter is instead of
Azure Public Endpoint and you can get the same data so
that you can be compliant with your regulatory requirements.
>> That's such a really important thing,
I want to really make sure that people get that
because I didn't even really know.
I understood that, but when you explain it to me,
that means that my own privacy concerns changed.
I would want the app to tell me,
''We're using machine learning,
but your data never leaves here''.
I think we all have an impression with the public,
the non machine learning expert
public like yourself have the sense that,
''They must be streaming live video from my camera of my receipt."
Then some genius AI in
the Cloud is figuring this out in real-time.
These are highly trained,
very condensed models that can totally run on your laptop.
>> Exactly. That's a really good point
because for example, for IoT devices,
it may not be the regulatory requirements you're concerned about,
but maybe the Internet connectivity,
maybe you don't have great
gigabyte Internet connection all the time.
So in those times if you have
a local model that you can call against,
so you don't have to worry about
your Internet going off for a couple of seconds and
those couple of seconds being
just that one time that customer sends data.
>> I might want to run this on a Raspberry Pi or I'm like,
when you say the Edge, it could be a really tiny device.
>> Exactly.
>> Cool.
>> So what I want to do now is show you how that works.
So first, I'm going to use the same app
to demo a real-time crowd analysis with.
First, I'm going to run that against the Cloud.
So right now, the endpoints are the Cloud. So here-
>> So I'm part of the crowd?
>> Right.
>> I'm in the crowd.
>> So it detects your faces and it shows you that neutral,
happy, and neutral smiling.
>> My face is always neutral.
>> Yeah.
>> Okay. So that's actually going to the Cloud?
>> Right. So now,
I'm going to go back and what I'm going to do.
So currently, I have a Docker container running on my device.
So if I call that Docker container,
it's not going to be connected to the Cloud.
So Face APIs which is the one we use,
I'm going to change the end point to localhost.
>> Okay, localhost 5000.
>> So it's going to be all within my computer.
So I did that and now I'm going to go back to the same demo,
and it works just the same.
So it detect your faces
and it's a little slow on the emotion uptake.
>> Let's see.
>> Yeah. There we go.
>> There you go and that was coming from this Window right here?
>> Yeah. It's coming from the Docker containers.
So you can see that you can run the same models just
on your Edge or on
your Cloud service and you can get the same results,
it's the same models, but without having to worry about
intermittent Internet connectivity or any any of that.
>> You could put that in airplane mode,
and it would work just fine?
>> So the billing end point actually does go to Azure.
So it does have to have at least intermittent connectivity,
as in we still have to know if your using it or not.
>> Okay. So you're reporting usage,
but you're not sending any of my data up to the Cloud?
>> Exactly. So like Christina called Face endpoint at this time,
but I don't know what picture you use to send that and point up.
So your data, like the PII
and all of the data that is sensitive
stays on your compliance boundary.
>> That's cool, the personal identifiable information
is completely private.
>> Exactly.
>> That opens things up for a lot of people,
and that's just two of the cool things that you can do.
You have this demo app, but there's so much more beyond that,
forms and search, and speech detector, on and on.
>> Exactly. So going back to this slide,
we have all of these really cool capabilities.
I think we have 15-20 GA services already,
and we're constantly adding more so that we can empower
the engineers who are developing
their apps to do more with just simple APIs.
>> I'm telling you, just the receipt one has me excited.
If I can have that on my phone and take pictures of receipts,
this going to be a total game changer.
>> Right.
>> All right. Fantastic. Thank you so much for showing me that.
>> Thank you.
>> All right. I am learning about
all the exciting stuff that you can do
with Azure Cognitive Services today on Azure Friday.
[MUSIC]