Hello and welcome to Microsoft Mechanics Live!
Coming up we're joined by Julia White
for a look at Microsoft's approach to AI.
And we're gonna also make it real
by taking a look at some of the practical applications
before breaking down how you can leverage AI
for your own apps and services and online experiences across the areas of
knowledge mining, cognitive services and machine learning.
But please join me in welcoming Julia White CVP of the Cloud Enterprise Group.
Thank you so much.
It is awesome to be back on Mechanics after a year or so hiatus.
Thank you.
So we're seeing lots of innovation we're seeing
lots of buzz around AI it's being used a lot that term I think in the past we've
covered deep learning that a lot of custom ml models stuff that data
scientists do and you know a lot of us afraid a scientist but on this show we
wanted to actually give everyone a real-world set of practical ways that
they can actually integrate AI into their own apps right I mean this is
really important because we don't want AI to just be in the hands of a select
few our focus is really on making AI machine learning easier and more
accessible for everyone to use and that's really the key focus and for the
past decade we've been integrating AI into a lot of our cloud services one of
the early implementations actually you remember was in exchange online of
distinguishing what was good mail vs. junk mail is just a really early example
but of course AI is an integral part of being maps and being search and curse
the official and biometric authentication with Windows hello also
using AI and then infused into our office 365 experience right with
PowerPoint designer or word grammar checker even and then we also use AI
pretty pervasively around security right and using all of our advanced threat
protection czar underlying that our cloud services is using a ion or
security technology now we've taken all that technology that we've used
internally for all of those applications and turned into Azure AI services that
you can use in your own applications and this include many of the pretty
sophisticated pre-trained ml models that we've now turned into Azure cognitive
services and we've also enabled support for the most popular AI frameworks so
you can train your models using a number of different frameworks in a very open
approach on that front and of course for those people who need some
very highly specialized you can also collaborate and build your own custom
machine learning model using Azure as well so regardless of whether you using
a pre trained model or a custom ML model you can always of course get the AI
compute you need with Azure and then go ahead and deploy and run that either in
the cloud and/or on the edge in that area right so we're doing a lot in terms
of democratizing AI and really making it available for everybody to use so why do
we get to do this and actually make it real and show people a couple of
demonstrations what we can do I love it all right so great let's get started
with an example of knowledge mining now this is about quickly getting insights
and all of your existing information within your organization so for example
I'm assuming many of you who have been around for a long time your businesses
have been and you have a lot of data you're sitting on but it's probably
sitting in things like file cabinets or scanned in some archive it could be
written it could be pictures it could be videos so what if you could use AI to
add structure to that data and then have it indexed and easily searched or even
more powerfully you can discover different patterns or relationships
within that data by surfacing those insights so let me give you an example
so I'll make switch here - we have JFK example so about 25 years ago after
Congress mandated that the jf the documents related to the JFK
assassination became Declassified so last October they released 34,000 pages
of documentation 34,000 pages of is a lot I mean that's that's probably how
how high would that stack up I'm gonna say least to the top of that wall right
probably so first of all we took that enormous mistake of data paper and we
digitized it and then we put it in Azure and then we applied as our search with
cognitive services where we're able to intelligently annotate and index that
information to make sense of it find relationships and surface new insights
then we built a simple web app on top of it so you can literally interact with
this data and it's a site you can all go to as well but let me show you how it
works so I'm sitting out at the the web experience again on top of that data
repository that's been searched index I'm gonna go ahead and search on Lee
Harvey Oswald hopefully everyone knows his relationship to the Drake ok
assassination so we hadn't search on that now you can
see over here on the Left adjure is kind of pulling out different key entities
based on the contents of the document so you see things like the CIA
and if JFK of course and Lee Harvey Oswald it all makes sense over here on
the right you can see a variety of different documents and you can fact see
that his name is actually highlighted it could be like typed information here
written in this example I'm scanned and other things and even if I using optical
character recognition or OCR that cognitive skill we can annotate that
text document to make it searchable even though it was handwritten and now if I
scroll down you can see it's capturing a picture too so again because if you're
using computer vision the cognitive skill I can understand the content of a
picture as well so if I click on this you can see it recognizes that's Lee
Harvey Oswald you see the name is actually highlighted here and you can go
down I it's actually scanning over this you can see it's giving me an annotation
again using that OCR capability within the image itself so kind of close that
out now the next thing I can show you is also about the relationships that's
found between the information so if I click here on the graph site it's
actually pulling together a graphical representation of the relationships
across specific terms that are found across this whole body of documentation
so if I grab Oswald here for example you see kind of a cluster of information
circling around him like Harvey and Lee of course that makes sense but the
interesting thing is you see this connection right here Sylvia Duren right
if I pull that out and a big collection of things around
Duren now if we dig into this document here our ticket to this here in
searching and I can see I'm finding documents that let me to understand that
this is the interrogation of Sylvia Durand by the Mexican government and we
explore the documents you see that she actually wrote to the Cuban consulate in
Mexico and was Lee Harvey Oswald's linked to the Russian KGB all just
became discovered because this graph can become advanced out of this it's pretty
amazing which are able to find with this azor
search with the help of those cognitive services and if you want to try this at
home you can go back to this web app it's available publicly at AKMs slash
JFK files and also this is a public and available on github two really cool
stuff of course you can reverse engineer some of this work on github so really
cool but you're also able to take document types that weren't searchable
this is paper like this is handwritten paper leveraged as your search with
cognitive services to really allow us to index all of these things how do I make
them searchable but also structure how easy is it to harness these types of
capabilities well let me show you how easy it is Jeremy okay oh can I take in
that document all that scanned imager you put it in add your blob storage and
I've now hooked up my Azure search here in the imager portal to that storage and
all I have to do is literally set up search I go down here and I can just
select the cognitive skills I want to add to my search and I can just say okay
it's literally that simple but you remember we were using OCR in that
search I did so let me go ahead and choose OCR and when I do that you'll see
additional skills are generated so I can also pick additional attributes because
of that turning on so and that JFK of course we use natural language and other
things too so all of them literally it's that simple okay so we've seen a lot of
different categories here in terms of the cognitive services that you
mentioned earlier but many of these are pre-built ai models that came about
really as we started developing various services in Microsoft so how did these
come about yeah I mean that's right it's basically where our core engineering
teams along with Microsoft Research have made several breakthroughs across the
area of vision language speech and search and for example we recently
received the highest score for our machine learning capabilities through
the Stanford question and answer data set called squad and it's a test of AI
reading comprehension around hundred thousand questions under their related
answers so pretty impressive what we're doing alright so beyond integration with
intelligent search how can we use cognitive services API is in our own
applications well I mean basically every application can benefit from cognitive
services in some way but let me give you a specific example so we're here in
Florida and it's famous for many things but it's including oranges of course so
let's just take the example of an orange farmer currently many orange growers are
not differentiating their produce they're sending all of their oranges for
juice production but maybe they want to build a model that creates a sorting
facility that uses AI that can decide what's great a which and they sell those
oranges at a more lucrative retail channel or what's the lower grade B and
they send those those oranges off produce production so what if they could
use this AI to differentiate oranges by using computer vision and sharing the
oranges that are sorted to grade a go in one category and go be great B in the
other category and using intelligent edge to do that for them so
it have to be a pretty pretty smart system because bad and good oranges are
pretty close looking right you think right but let me show you how you can
actually train a model to do that let's try it so I'm out here in a now a within
my computer vision area and I have already pre-loaded a group of oranges
into the site and without any trading it's already identified that it's a
group of oranges in a pile and has confidence of eighty-five percent so
that's lutely just loading out the image it's publicly available and thanks to
everyone participating is getting to me that confidence I mean it's like
beautiful oranges here as well but the model hasn't yet been trained to decide
what is a great a orange or a grade B orange but that of course makes sense
because there's the orange far more you'd want to decide your your
categorization of what's the threshold for great a a grade B right so I want to
do my own custom training on that so that me do that now I'll go over to my
custom vision AI and I've got a project I've started here on orange
classifications so go ahead and click in here and I've got a number of images
loaded to help start training my model so you can see there's labels on some of
these oranges and they are they little ugly I know dad for that orange but you
know yeah and I can't all be beautiful on the inside they're delicious
yeah so if I click on great a you'll see it's seeing that these are what I'm
defined and categorized as grade a level oranges and if I I've started doing my
training on grade B so you can see some of those but I want to go ahead and add
another image to train that model further so I go ahead and just add
images and this is how I started the model and got it going so I'm going to
add a new grade B Orange open it up and I'm gonna go ahead and tag that as a
grade B orange again not simple and I'll upload that file uploading it it's gonna
retrain and then with you know within a few seconds I can go ahead and have that
updated to my models relearning so let me do a quick test to see if that how
it's working so I'll go to my quick test area and I just browse from my files
I'll grab I'll start with an A grade a orange okay drop that in and I see it's
giving me the probability that's a 99.5 percent positive that this is a great
babies perfect looking orange I think I've ever seen right I mean I guess it'd
be hard to get that one wrong right but let me go try a great b1 here yeah let's
grab a bead and load that up and see how it does so
this case is 97.7 for some Potter's of you see it looks a little different if
they don't pick up on that level of detection now again this is just a great
example of how you can get started with AI for your own uses I didn't wasn't
doing any coding I just load it up and train the image and you can try this
today on Azure it's on custom vision AI where you can get access to this
experience and if they move forward we're gonna continue to make cognitive
services API is even easier to use for a great example just here at ignite we
launched our unified speech service then now combined speech to text text to
speech translation and the ability to train the model based on unique custom
speech mounted patterns all as part of our Scott custom speech API these are
really great ways to leverage the AI effectively that Microsoft is built and
tested and our AI services are even getting smarter and smarter all the time
as we look at more images so what do you want to what if you wanted to build out
your own models and train those to work with your own data how does that look
yeah so for the folks who want the custom build their own machine learning
models and the more sophisticated needs Ashur provides a complete machine
learning platform that you can take advantage of and I'll just show you kind
of one part of that here okay go into getting back to my add your portal and
I'm in my data bricks notebook and one of the great things about as your data
breaks as it enables both data scientists and data engineers to
collaborate on shared projects within this interactive workspace so you can
easily pull in structured and unstructured data you can prepare it
connects a called data wrangling if you clean it and get it ready and train for
my AI models right within the workspace so what I'll show you here is actually
example of a shoe retailer and they've built a shoe recommendation engine such
that way and a customer comes in searches they're shown something it's
highly relevant so it's to hopefully drive up sales and results but right now
the Shuji realtor wants to then take that recommendation engine and use AI to
make it even more highly predictive do we do anything then to help with the
types of algorithms and data models that you might use to make these types of
recommendations well yeah actually one of the things we also against this this
week at ignite is a capability called automated machine learning and that's
basically using AI to train AI in a sense if you think about it that way but
essentially we test all of the different models using AI to see which algorithm
performs the best has the most act results and then automatically
recommends that to you so we'd even expedite that process of it you can also
use you know simple sequel like querying to usually explore the data within spark
here in Azure data bricks and also get a sense of the current recommendations
engine and sakura see okay let's look at that alright so I'm down here now first
of all start and this is the clickstream results so right well let me show you
that what this means essentially this is the frequency of clicks over the period
of time now you see all the clicks are happening kind of at the end which means
that's when the shoe is going on sale but I get a clearance out of inventory
so that's probably when it's the cheapest price that we're gonna sell it
rate which isn't great for my sales right when I'd really wanted to be lots
of clicks in the beginning when this she was new and hot so this gives me a clue
that something's not right so I'm gonna go ahead and bring the data
scientist into the notebook didn't see what you know she's and so she can
confirm what people are seeing in the different searches so as I scroll down I
can see that this is the current results she's providing it like here's the
target shoe and then here's the related results these woolly slippers for
open-toed shoes okay so I thought that we would actually to make the
recommendation look more like that first shoe that's surprising that they're
recommending bad I'm not getting a lot of clothes look comfortable but I don't
think you want to wear those in public all right so what I'm gonna do is I'm
going to scroll down here I'm going to optimize this model by bringing in a
deep learning framework in this case tensorflow package that can help provide
recommendations it's called deep image feature Iser and effectively breaks down
the different attributes or the feature of every shoe and the Claddagh log so
things like open-toed heel color size type more so that i can really get a
granular assessment of that now once i feed those attributes back into my
machine learning model i'm gonna prefer on the same search again and see what
kind of results we get if i scroll down here you see there again is my target
shoe and now you see I'm getting very similar recommendations around that
target and even just to make sure it works on different types other than
these open-toed in different colors of the same shoe you can see even it's very
close on the other open-toed models yeah so you can see the accuracy the results
around again I have now loaded up a different Salado heel I can see that I'm
getting very similar results so I feel much more confident about the
recommendations making so then once I see that I've got
a recommendation engine that's working well I can then finally take that and
distribute it in Azure cosmos database and I can integrate that into my machine
learning model that's running the operational database of my web and
mobile applications so this recommendation is now going to be
showing up across all of my customer experiences it's a really powerful stuff
three awesome ways to start to apply AI in your own day-to-day workflows and
experiences but where should people go to learn more well luckily you can check
all everything that I showed to you out today it's all public so the JFK demo
they add your cognitive services the vision API and you can also sign up for
a free 14-day trial of azure data bricks you can see that workbook and do that
data wrangling and AI prep as well and the power of all this is that the more
people who use the open frameworks to participate in this the better the AI
innovation gets it really is a kind of open community approach and we're all
going to benefit from where we can harness Thank You Julie emotion we keep
tracking this on Microsoft Mechanics having it hit subscribe hit subscribe
now follow us on Twitter thanks for watching and we'll see you all next time