>> Welcome to another episode of the AI show.
Today we're going to be talking about
the Intelligent Edge with
Anusua Trivedi. How are you doing my friend?
>> I'm doing good, how are you?
>> Awesome. Tell us a little about the project.
>> So, today we're going to talk about real-time edge AI.
It's important nowadays to not
only work on applications but
work in a way to actually give it
access to every user and empower every developer.
So, what we're trying to do here
is show you how we build a model on
Azure and how can
we actually deploy it on the Edge device itself.
>> I see. So this is Machine Learning
but instead of putting the models in the cloud,
it actually works on the devices that you're looking at.
>> That's exactly correct. Yes.
>> Awesome. Tell us about the project.
There's got to be a use case that you have.
>> Yes. So when we were thinking about AI applications,
the first thing which struck us was healthcare.
So, it has a profound effect on everyone's life,
and what if we can
develop an AI product which affects healthcare?
So, we were thinking about the example of skin cancer.
What if we can actually develop an app which
will enable us to put us in a priority queue?
So, take the example of third world countries.
I come from India, to take the example of my country,
the patient and the doctor ratio
is like super weird there.
For one doctor you have
thousands and thousands of patients.
So, how can the doctors prioritize
the use cases which are super essential?
And cancer is very, very time-sensitive, as you know.
So, we thought about,
okay, what if we can actually create a suggestion app,
kind of an AI assistant for
the users or even can be used by the doctors,
which will be able to detect the stages
of skin cancer and
suggest if you need to go and see a doctor.
>> I see. So it's not a doctor.
We don't want people to be confused
about whether this is a doctor.
>> Is just a prediction app as AI does.
>> It's just predicting. That's right.
>> Yeah, so AI does predictions for you.
And here, it tries to confidently
say if you need to go see a doctor or not.
>> Awesome. So, tell us a little about how you went
from idea to actual model?
Or maybe we should start with a demo,
just so people can see what it is.
What would you like to do first?
>> Yes, let's start with a demo first.
As you see here, on my screen,
I have two apps.
I have photo version of the app and I
have a video version of the app.
Let me first start with the photo version.
What I have done here, I've already
preloaded this app with different types of images.
Let me choose one of the images and see what it gives me.
So as you can see, for this image,
it's telling me I'm all clear.
So, I can still be confident by not going to a doctor.
Let me choose another one, as
you can see here it's suggesting me,
you should surely go see a doctor.
But what if we can make this app real time, right?
For example, what if we can feed it
a video feed and actually predict something cool.
So, if you see here, I have
an image on which I
am trying to predict and it says that,
"Okay, this is a melanoma,
it's undetecting, and you should go see a doctor."
>> So, let me see if I understand what's
going on, as you're doing this,
because traditionally I'm used to cognitive services,
I would send something to a REST API and it would send
back whether you should see
your doctor or not, for example.
In this case, the video is sending the pictures
to a local thing onboard.
So, it doesn't even have to be connected to the internet.
>> Yes, for example, in my phone here,
I don't even have internet working.
>> I see.
>> So internet is turned off.
For example, if I just hold
it on my hand and I'm trying to.
So, if you see here, it's telling me I'm all clear.
It's telling me I'm all clear,
I do not need to worry about it.
That's my own hand feed, I'm just feeding it.
So, it's all live feed.
So, what I'm doing is,
I'm just giving it all live feed.
In the phone I have a trained model deployed
and it's just processing in
real time and just giving me the information.
>> So for those that are new to Machine Learning,
why don't you describe what a model
is and how you can actually have
it fit on a device or work in a device.
>> Yes. For that let us go to my screen and see.
>> Awesome.
>> As you can see my screen here,
I have the Azure Machine Learning up and running.
You can run it anywhere,
but because Azure makes your training processes easier,
I'm running my model in Azure Machine Learning.
Here, as you can see I have my training code.
Here, I have developed,
going a little bit deeper into the code,
I am showing you here I have a keras model,
which has got three layers.
I have two convolution layers,
as you can see here, two to the convolutions.
And I have a max pooling index in there.
So, this is exactly what my model is and
have trained it on a research data set.
The dataset is called is it cancer data set.
So, it's an open data set,
I've just taken the data set and trained my model on it.
>> I see. So just to be clear,
this is the part where you actually train the model.
>> Right.
>> And the data you got, from a data set that has
pictures of cancer pictures and non-cancer pictures.
>> It's a labeled data set of melanoma and it tells me,
"Okay, this image is
a carcinogenic image and
this is a non-carcinogenic image. "
And it tells you whether you have melanoma or not.
So, I have taken- it is a classification algorithm.
So, what I've done is I've taken the images,
I have trained a keras classification model
on top of it in Azure.
>> Okay, so as I'm looking at this, it's,
effectively, you're using a convolutional
neural network with one layer.
Well, one convolutional layer is a max pooling dropout.
And then, you run this, I suspect,
somewhere on this particular data.
So, tell us about that.
>> So, what I do is I have actually trained my models.
I have uploaded the data.
I have uploaded the data just in my local machine,
I can upload it in Azure Blob storage, wherever I want.
And I have actually preprocessed
data and create three tensors.
As you can see here, I have a training tensor,
a validation tensor, and a test tensor.
it's a very, very Machine Learning specific thing.
What you do is, you divide your data set
into your training test and validation data set.
First, you train on your training dataset,
then you do your validation on the training code,
and then you bring in your test dataset.
Something which the model has not seen at all.
Then you test it. So, with that,
let me go to the scoring
and show you how that's doing it.
>> Let's do that. So as you're going
there, just to be clear,
you train it on this bulk of
data but you got to know how well it's doing.
So, you got to use
the validation data to see
if everything is going according to plan.
Then you use this stuff that it's never seen
before and say is this general as well.
>> Right. Exactly. So what I'm doing here,
is basically I'm loading
the model and just what I'm saying here is,
take a test image,
what you have not seen, and then run that test image.
And I'm saying, to load your model,
you load it from this schema.json So,
I've trained my model,
I've saved my model schema and my model weights.
I'm loading it up and what I'm doing is I'm
actually giving it a test image, trying to test it.
>> Awesome. So this now,
we've shifted from the actual training,
to this is how you predict the thing.
>> Right.
>> Got it. Okay.
>> Where the whole prediction of the app is coming from.
So, here what your saying is, my model is saying, okay,
using TensorFlow backend for keras,
and it giving me, if you can see,
two flags, two numbers.
If you see the first flag is for the non-malignant,
the second flag is for the malignancy.
If you see, the coding logic goes in this way,
whichever is bigger is the flag, the predicted flag.
As you can see here, my malignancy flag
is much higher than the non-malignant flag.
So, that's why the predicted label
comes out to be malignant here.
>> I see. So maybe if you're predicting between
different kinds of melanoma it
might be a larger vector with- okay. Got it.
>> Right, this is a binary
classification that we are doing.
A classification of two types of things.
Suppose I had lesions, moles,
different types of dermatitis to be categorized,
I would have a multi-label classification problem.
That is, I have multiple lebels for
multiple different types of problems.
>> And in this case you will
train a different model that would have
multiple outputs and you will
give it different data from it.
>> Right, it just depends
on how you're training your model,
how you're defining your model.
And the model that I'm using could be
used for multi-label classification also.
All you need to do is just change your number of
labels and your label list and it will work seamlessly.
For binary as well as for multilevel classification.
>> Fantastic. So when you're building this and this
is going to get to the heart of what a data scientist is,
because I understand Machine Learning models from
a mathematical perspective but I've
never actually done anything as a data scientist.
How did you know to use
a convolutional neural network and how
did you know to set it up the way you did?
Is there just like a trial and error.
>> It's mostly a trial and error process,
specially if you're training your network from scratch.
If I would have been using
something called transfer learning,
which is very common for image classification,
I could have avoided a lot of complexity.
But because healthcare images
are very, very domain specific,
you need to have your hyperparameters
adjusted correctly to make your model learn very well.
For example, if you see here,
in the training code I'm opening,
it's giving me an accuracy,
a training accuracy of almost 90 percent.
So, that is a very high accuracy.
If I would have been using transfer learning,
where I'm taking a pre-trained generic model.
Pre-trained on image net or something like that,
and I'm trying to modify it for the healthcare,
it would have got me to around 80,
85 percent at most.
>> And that makes sense, because
a transfer learning has been trained on a generic set of
images and these images
they're generally not at the generic set.
>> You're right, it's very, very domain-specific.
It has different types of information,
color information, age information,
dimension information embedded inside.
So we want to learn all of those,
you need to create things from scratch.
Hyperparameter optimization is the hardest thing
when you're training a model from scratch.
So, I had to spend a lot of time while
training on the hyperparameter optimization part.
>> Why don't you describe what a hyperparameter is.
That way people that are looking,
that maybe are new to data science can understand.
>> So, hyperparameters, for example,
let's take the classification example
because hyperparameters can be huge.
For classification models,
hyperparameters are your learning rate,
your initial learning rate.
Your learning rate decay.
That is because you're doing convolution,
you're just learning in the forward pass and then
you're decaying it when you're doing the backward pass.
So, there is learning rate, there's learning rate decay.
And there is also something called the batch size,
so you're trying to learn in batches.
So, you cannot just give all of your data and
your memory would be overflown.
So, you would need to identify what is your batch size.
Another thing you need is the optimizer.
That is actually defined for describing your loss.
For me, I spent the most time
on the optimization part of the loss.
I have been using
RMSprop but it was not converging my model.
So, I had to change to AdamOptimizer
and had to change the values
of the AdamOptimizer to make my modern
converge and come to a 90 percent accuracy.
>> I see. So the hyperparameters are
sort of the extra values that you put,
the secret ingredients into the actual model.
>> Yes, that's a secret sauce of the chef.
Usually, there's a lot of trial and error to do that.
>> It's a lot of trial and error.
To train the model with Azure Machine Learning,
it took me very less time.
But to build the model,
that is where I spent a week to bring
my hyperparameters in tune with
each other so that they are working in perfect sync.
>> That's awesome. So, this makes perfect sense.
The thing that maybe is a little
bit more unclear to me is,
how do you take these models?
Because I saw how you did the prediction or
the inferencing in here with Python,
but how do you actually take this model
and put it onto
a device that can do Azure Machine Learning?
>> Right. For that, let me show
you a conversion code that we are doing.
So, here, is you've seen the code.
What we're trying to do, we
develop the keras model, okay?
We're loading up the keras model.
What we have done is we have
installed Core ML in Azure Machine Learning,
and I'm trying to convert
the keras model to the Core ML format.
So, let me define a little bit what Core ML is.
Core ML is basically Apple's package of compression.
It actually helps you compress
your model weights in
the format with all the dependent informations,
so that you can run it on any device.
So, imagine something like sudo apt-get, right?
So, imagine you have
a deep learning environment, your dependencies,
your model, everything inside,
your workbench, or your Azure Machine Learning,
or wherever in your local machine.
And you want to bring all of them together,
as well as you want to compress the weights
a little bit because when you train
a Machine Learning model, it's huge.
>> Right.
>> Right? So, you would want to compress it so that
your phone processor can run it seamlessly.
>> Right.
>> So, all of these steps are taken care of by Core ML.
So, all I needed to do was call Core ML converter,
and through a few lines
of code like three or four lines of code,
it was able to take all my care
as dependencies from my environment
along with the model and compress it and
create a full package, Core ML package.
So, when it creates a Core ML package,
it's basically the name.mlmodel.
>> Right.
>> Right? So, anything star.mlmodel is basically,
you're getting a Core ML package.
>> And that makes sense because
when you're doing inferencing,
you're not going to need to have
anything having to do with the learning rate,
or the decay weight,
or whatever optimize, you don't need that at all.
You just need all the numbers that are getting
multiplied together and then how they're
going to be output.
>> Exactly. You're so right.
So now, once we have converted that,
the main reason why we are using a Mac today is we put
that Core ML model into Xcode and compiled it.
That produces a star.mlmodel C file, okay?
So, that produced a compiled version of the Core ML.
Now, we can drop it on any application and run it.
For us, we're using Xamarin.
And let me show you the Xamarin.
>> Let's do it.
>> So, as you can see in my screen here,
I have the Xamarin app uploaded.
As you can see in the resources,
I have dropped the skin cancer_mlmodel.cfile,
the compiled one, which I have compiled using Xcode.
Here, as you can see after compilation,
it has created different versions of it.
When I first created the model,
it was all just a model, just scareus.HDF5 file.
But, when I have compiled it,
it has created all the separate files
so that my app can effectively read it.
>> I see.
>> That is what Core ML does.
It takes every dependency.
So, it takes, okay, what is the shreds?
What is the shape?
What is the architecture of
my model? What are the bindery?
What are the DLLs? What are my dependencies?
So, everything comes along with the model.
>> I see. So, you should think of
this as importing an assembly,
or a Azure file, or some kind of
package that you're going to call to do the inferencing.
>> Exactly.
>> Okay.
>> This is very, very developer-friendly thing,
and you can actually take any model convert it to
Core ML compiled version and run it on any app. Okay.
>> So, if you have a data scientists
in-house that has made some amazing model,
used keras, or TensorFlow,
or something else that's pretty
standard, you can ask them,
"Hey can you export this to an ML model,
and then use it in your application?"
>> Exactly. In fact, what we have been doing
is we are a team of data scientists and developers.
So, when I produce a model and give it to my developer,
my developer knows to go to the Core ML,
just convert it, and just drop it onto the app.
>> Yeah. Because it's again,
just an assembly like you would,
like a PDF library.
Nobody knows how PDFs are written
but we know that if you call library, it will make one.
Same thing here except it will
detect whether or not someone may have skin cancer.
>> That's right.
>> Amazing.
>> Yes. So, once you have that,
you go to the ViewController.csv file, and there,
as you can see, this is a sample app you
can find in the Microsoft GitHub.
>> We'll make sure to put links. Yeah, we'll put links.
>> Yes. So, here,
as you can see, I have just
pointed it to the model file name,
and going down below in the logic,
where you're actually doing
the calling of the prediction function,
you're saying, "Okay if my ClassName is benign."
ClassName is basically the label
of the predicted label that you are getting.
Then suggest that you're all clear.
However, if the ClassName is
malignant as you were seeing on my Azure ML workbench,
it's suggesting to you to go see a doctor.
>> So, that's it?
I feel like a lot of times when we talk about
Machine Learning it's like voodoo magic.
>> Right.
>> And when we look at the app initially,
it felt a little voodo magicish.
But now, that we've gone through, we saw the model was
a convolutional neural network built in keras.
A lot of time was spent
on making sure the hyperparameters were okay.
Once they were done, you did
some inferencing in Python to make sure it
was like your internal check then you export to ML model,
and then the ML model then goes
into a Xamarin app, and you're done.
>> Yes. That's as simple as that.
In fact, the Xamarin app part is super easy.
And we'll put links in there.
You take the sample app,
you just changed three or four lines of code and
you have your own specific app running.
So instead of, like a skin cancer,
if you wanted a fruit classification,
you can very easily do that.
>> That's amazing.
>> Yeah.
>> I mean, the thing that I like about this
is that we often,
and I spent a lot of time
studying Machine Learning models,
and when we're talking about optimizing leads,
and she's like, "I had an optimizer of my own."
"Oh, that's really cool, how
did you figure out your batch size?
How did you know the learning weight was 0.1 instead?"
That to me is exciting, but then actually seeing from
beginning idea all the
way into an application, it's pretty amazing too.
And in short amount of time, we've only been
spending we're 15 minutes in probably.
>> Yeah.
>> Awesome! So, where can people go to find out more?
Or, is there anything that I'm missing?
>> No. You are not missing anything.
So, we do have Azure GitHub link
and the whole project in the Azure docs.
So, we will give you the link below in the video.
>> Awesome. Well, this has been amazing.
Thanks so much for spending some time
with us. Thanks so much for watching.
You've been learning about computing at the edge,
and we saw an entire example that's pretty
amazing in just a couple of minutes.
Thanks so much for watching. We'll see you next time.
Take care.
>> Thank you. Bye bye.