>> You're not going miss the next episode of the AI show,
we talk all about computer vision for digital scientists.
Make sure you tune in.
Hello and welcome to this episode of the AI show.
We're talking all about
computer vision for digital scientists with
two special guests and the coolest part.
Well, let's start with you Randy.
Why don't you tell us who you are and what you do.
>> My name's Randy Price.
I worked at the LSU AgCenter and
we've been interested in newer things
coming up called digital
agriculture and also goes what digital science.
>> That's cool, Alicia.
>> Hi, I'm Alicia Moniz.
Before joining Microsoft, I was AI Microsoft MVP,
and got the opportunity
to co-authour one of the cognitive services books and
I'm delighted to be able to get hands
on with some of the new features in version 4.0.
So thank you for having me.
>> For those that do not know,
Alicia and I went to the same school and graduated the same year.
We are products of the same university.
I'm so glad you could be with us Alicia.
>> Thank you.
>> Let's start with the problem. Randy, can you
tell us what we're talking about today?
>> What we're talking about is we got a lot of
tools for seeing things in the field.
We can take a drone and fly up real high and make
a map and we know where a bad and a good areas,
but we just really can't diagnose it from that.
So on the next slide
we'll show you where you can also take those drones and
fly way down low to the ground and we do this
on the filled routinely like
all the time and get really close up images.
Now the next step is we either has to send somebody out
there to see what the actual problem was or what we would like,
is some of the people who are good at AI and machine vision
to build up some software that will
analyze these images automatically.
Things we're looking for are differences in color,
we're looking for holes in the leaves,
damage and stuff like that.
>> That's cool. So that sounds like an ask.
What is it that you're looking for when you're flying over?
Because obviously this is an agricultural problem,
but as you're taking pictures,
what is it that you're looking for and what alerts do you want?
>> A lot of times you're looking for physical damage.
I mean, when it comes to worms and
some things like that and certain types of bugs,
they will actually chew on the leaves.
So you'll see that physical damage.
Now, there's another thing we look
at and answer in terms of color,
it isn't always how green it is.
Sometimes we do what's called an index or an indices,
where we'll take a ratio of some of
the colors we found like green minus red is a really
good one and that tells you how
well the plant is doing biologically.
If it's low, unlike nitrogen or water or something
like that I'll get more of a yellow tans to it.
There is something called chlorosis,
and you'll see that in some of the soybean plants,
we're going to show you,
where it'll get a real yellow spiny look
on the leaf and is from lack of
some nutrients like manganese or iron.
>> You need software because so many fields a bill
are so large that it's better to have a computer
analyze images and tell you where there might be trouble spots.
Is that the problem we're looking at?
>> That's right Seth. We have been
working on a camera system that we
put on a tractor and there's
some other people doing the same thing.
We actually recorded a field where we recorded
a high res image of every plant in the field,
65 gigabytes of data in one time.
Not that much is going to get bigger than that,
but it's impossible for me to look at that 20,000 images that we
recorded to find plants that have problems or damage,
and we would like machine learning
algorithms that could do that automatically.
>> That's really cool because now we're
taking the problem of agriculture and we're
moving it over to the digital space.
So Alicia, how did you help him out with that?
>> Thank you. It has been fantastic working with
both LSU AgCenter and Dr. Price on talking
through the use case and talking through possible solutions
and being the computer vision enthusiasts that I am.
I thought it would be a fantastic opportunity for us to
walk through some of the new capabilities and the
Visuo Studio that was released
and walk through a little bit of Python code using
Azure Data Studio and we can go ahead and take a look at
what the actual classifier looks like via CustomVision.ai.
As a refresher or for some of our researchers who are
attending or not as familiar with the cognitive services.
What we'll be talking about today is the custom vision service.
Cognitive services will let you perform
some predefined activities via the computer vision API.
But what we're actually going to do is we're going to go
ahead and create our own data set and
use that to train our own classifier and
that'll be done via the Custom Vision API.
I've done this in the past four event detection.
So it's full objects and event detection.
What we're doing is we're trying
to ask a binary question, yes or no?
Is something occurring or is something not occurring?
Now, we can go ahead and try out these tools
without any programming on our
own via the newly announced Visual Studio.
If you go to the featured tabs,
there's remove background from images icon on
the right hand side and that is what we're working with today.
Randy and I were looking at some of
the pre-processing activity is that is
generally performed on the images.
We get better results from the images when
there's a starker contrast between the colors of
the leaves and not
having a brownish background at the bottom of the image.
One of the newly released features in cognitive services
4.0 is the ability to remove background from images.
I do have a bit of
a notebook that I can go ahead and share with you,
to walk through how that code looks like.
You can see that I've already
created variables for my acquainted key,
for my cognitive services.
You can go ahead and install the Azure AI Vision SDK.
What we can go ahead and do is we can use
URL reference to the image or we can pass the image locally.
>> Here we're just sending the file,
the image analyzer function within the sdk.
What this does is we've already
created our cognitive service in Azure.
It's waiting for us to pass either an image
or a text file so that it can return back the result set.
What this function is expecting is,
it'll expect a JPEG or a PNG file,
and it'll perform the manipulations and it'll return
back file locally for you to continue processing.
>> Is this part removing
the background currently or what is this part doing?
>> What this part is doing is,
it is sampling between
pixels within the file and depending on the contrast,
it'll go ahead and remove the parts of
the image where it's determining it's not part of the foreground.
>> Got it. It's like a preprocessing step for doing
other things in the Custom Vision, is that right?
>> Yes. Most research and analysis that involves images
requires preprocessing just like
we need to preprocess for outlying data.
There's usually some type of cleaning that we'll have to
do against binary data.
The same thing goes with some of our image data.
We really ask the owners to go in there
and filter out some of the outlying information.
When we're building our training sets,
we had take into consideration what are
the use cases that are likely to occur
and then prefer to use those for training
versus not as common use cases.
>> That makes a ton of sense.
It's like preprocessing the data to
make the actual task on the images,
which is discovering challenges with agriculture much easier.
>> Yes.
>> Got it. Cool. Now,
once you do this pre-processing,
what's the next step?
>> The next step would be to build
a data model that's able to help us.
It's interesting because when we look at biased,
I know we have been joking previously.
Now everybody knows I have astigmatism.
Sometimes when I look at variations
between smaller items in images,
it's a little harder for me to differentiate between color.
When we have these types of human conditions that affect
our perception of data or media assets.
For me, my eyes get tired [inaudible],
if I'm part of the group that's
labeling and classifying the training datasets.
I have higher accuracy in the morning
and lower accuracy in the afternoon.
It's just because my eyes are a little strained by then.
But if you have a computer that
is able to go through and do the classifications,
then you won't have that bias
affecting some of these automated activities.
>> That's cool.
>> Data augmentation is one of these new tools coming out that are
really helping researchers to expand their testing abilities.
Usually we say data out is only as good as data in.
Sometimes I'm limited by the data I have
access to as far as what can I do research on?
Data augmentation is one of those things where we now have
a little bit more ability to create these scenarios.
Our friends at Open CV helped
me with generating some testing images
and they took a few images and extrapolated using their software.
They were able to give me a range of
colors for me to use so that I
could build my tree testing use cases.
Then I could go ahead and use those to perform
my quick tests against existing images
that Randy and his team are collecting out in the field.
I can go ahead and open up that model,
and we can take a quick look.
>> As you're getting that open.
You're basically creating new images that have
different shades of colors to augment the data you're
using to train your model and to test it, is that right?
>> Yes. Sometimes we don't have data for all the used cases,
so we'll have a set of data at our availability.
But because we have a [inaudible] set,
we're really limited in gaining
insights about that existing set of data.
This is a common issue
or challenge with machine learning algorithms
is you get to know more
about the datasets that you already have access to.
How do you build a machine learning model that is more into set?
>> That makes sense. What are we looking at here?
>> What I went ahead and did was,
I established three different levels as far
as the range of yellow chlorosis,
that the different colors were representing,
so I did level 1,5 and 10.
There's different things that once you go through testing,
you're like, okay, well this might work a little bit better.
What I can do is, I can go ahead and upload
one of the images provided from the fields.
Then I'm definitely trending more towards the level one.
The probability range is
probably a little higher than I'd like it to be.
There's different thoughts that I've had as far as using
the probability percentage to help me build my range
versus building the range by
uploading 10 different levels of image variations if that's fair.
But right now we're looking at three times for
three levels and definitely
trending more towards level line which would be the lowest.
Looking at my training dataset,
the yellow might be a little overly
exaggerated when compared to some of the field images.
I think that definitely affected my levels too.
>> Got it. What did the levels represent?
>> It's common in scientific research for scales T1-10.
When they're defining problems
statements they try to create that unit of measure.
If we're verifying a condition,
what level of condition would that be?
>> That's fine.
>> One would be green and 10 would be extremely yellow.
>> Got it. That makes sense.
Once this is all figured out,
are we just trying to figure out
the yellow content in the images to
indicate whether a plant is
struggling or not struggling
or what is it that we're trying to find?
>> Sometimes it is just that yellow color.
Like we have some fields we were
looking at some yesterday and today
that they had the yellow chlorosis in certain parts of the field.
Sometimes it's as simple as just identifying
how much area has that and doesn't have that.
That was a field we were going to use for research,
then when we looked at we were like,
maybe we don't want to use that one,
because of the difference that exist in it.
In the next steps we'll make a map of it,
which these algorithms will help us do.
Then hopefully we can treat that and treat it
site specifically in the areas that need treatment.
>> That's engaging to me how we're using these drones to figure.
Usually it would've had to be someone was walking and looking at
all these leaves and there's not
enough time to do that. Is that right, Randy?
>> Yeah. Again, there's some farms in Florida that encompass
like probably 75,000 acres.
Just in order to drive to one side of the farm,
to the other, you can have an hour and a half, two hour drive.
The more you can digitize it and get it in a computer
and get people to look at it the better off we're going to be.
>> That's amazing. Alicia,
where can people go to find out more about this?
>> Definitely, you want to pull out
Serge Retkowsky's GitHub and
he has some amazing work books out there,
and they're very direct and to the point,
you can get up and running with
computer vision or CustomVision very quickly.
Definitely check that out and customvision.ai
has a really easy to
use UI where you can go ahead and build your classifier.
Definitely Division Studio.
I do want to call out that there are direct links to
the api documentation within
Visual Studio so we definitely look out for those lengths.
>> Well, this has been impressive.
Thank you so much for your time, my friends.
>> Thank you for having us on.
>> Thank you so much for watching me learning
all about computer vision.
For digital scientists a real use case
that actually affects lives.
Thank you so much for watching
and hopefully we'll see you next time.
Take care. [MUSIC]