>> You're not going to want to miss this episode of the AI Show,
where we talk all about scaling
your AI/ML practices with MLOps and Azure Machine Learning.
This is a three parter,
Part 1 of 3,
make sure you tune in.
[MUSIC]
>> Hello and welcome to this episode of the AI Show.
We're talking all about scaling
your AI/ML practices with MLOps and Azure Machine Learning.
This is part 1 of 3 and I've got
my good friend Abe with me. How you doing, my friend?
>> I'm doing well, how about you?
>> Fantastic. It's been awhile.
We talked about MLOps before.
I'd love to get the download on MLOps again,
but AI is changing a lot, right?
>> It sure is. There's a lot of new stuff happening.
I'm super excited and as you can see,
there's a lot of breaking news in the past,
I guess we're recording in May now,
so a lot has happened.
As you can see Microsoft has been super busy in 2023.
Things along, partner with open AI,
Copilot for almost everything.
So you're seeing a Copilot for Office,
GitHub, VSCode, Bing,
Edge, I can keep name in SharePoint,
OneDrive, the list can go on.
You're probably thinking to yourself,
how is Microsoft releasing all these AI models?
>> Yeah, because I've done machine learning stuff
and it's actually quite hard.
So how does this kind of innovation,
how do you keep pace with this kind of thing and how
do you do it yourself if you want to do it?
>> The big part of doing this at
scale is it's always thinking about how can
I adapt this to
some MLOps or some pipeline
to quickly release these models for production.
That's what at Azure Machine Learning
we partner with a lot of teams
internally to help them with
MLOps and you're probably asking, what is MLOps?
If you haven't heard of MLOps already, I'm going to tell you.
So basically what MLOps is,
is how do we get these ML models to production?
So as you saw all these opening AI models,
how do Microsoft get into production so quick?
So MLOps has three main pillars.
So it's the people,
it's the process, and it's the platform.
It's how do you bring together all
three of these major pillars to make sure things are automated,
your software infuses ML and
delivers continuous values to your customer in a quick time.
In the people, process,
and platform aspect of things,
Azure ML does help with
the platform setting when Scott will talk about processes,
we're coming up with the MLOps accelerator.
But also getting the people
enthusiastic about MLOps is a big thing.
What are the key benefits of MLOps here?
The real key benefits of MLOps
is if you have these three things working together,
you're going to increase automation,
collaboration, and reproducibility, which would then equal
faster things go into production
and in a secure manner that's super auditable.
So that's the big thing about what MLOps is,
if you haven't watched the video we recorded last year,
we'll go into more detail about this,
so I recommend you check this out.
>> Yeah, so a question I have before we
get to the next stuff is I have
found this a lot because I
talked to a lot of people about machine learning.
I have found that there is no amount of platform or tool
that will cover up bad process or
people not buying in to the process.
Is this your experience as well?
>> That is definitely the experience.
It's like any software.
It's like if you're like everybody use git and you have somebody
on your team using like OneDrive or Dropbox to put it in code,
it's really not going to work. Yeah, exactly.
So when git first started everybody to buy into git,
how do I do git pushed git commit,
and now the tools and platform are lot better.
So people have better processes.
People are educated, so I would assume or most practices,
most companies are doing the same thing.
You're upskilling people, MLOps, on like,
how do I use these tools to get stuff to production faster?
So you're right, the people is super-important.
People need to know what they're doing,
how they're doing it just like they had to do for Git PR,
it's GitHub and things along those lines to collaborate quickly.
>> The other question I have is machine-learning,
because I come from the DevOps world.
I've done DevOps stuff before.
But machine learning is by nature just different.
Would you say the process is different from DevOps to MLOps and,
and if so, what does that difference look like?
>> Yeah, the process is slightly different and luckily,
you like we've thought through this over the past year
to really make sure we can conceptualize it.
So we've talked to a bunch of customers over the past year to
really talk through what this lifecycle looks like.
What is the difference between
machine learning and the DevOps world?
So as you can see on the screen,
what we've come up with is
three big pillars and a surrounding managing loop.
So as you can see,
like over the past year or two years and Azure machine learning,
we've talked to many customers and validated this hypothesis.
What we see for enterprise machine learning lifecycle
it's usually starting from prototyping,
then I go to training and then I go to operationalization.
So basically what that means
is most data scientists, when they're working with stuff,
this is why it's a little bit different than
the normal DevOps is basically work in your dev branch.
They prototype, they work on Notebooks,
they work on the hypotheses.
Then you can't just release things to prod.
You have to then train that model somehow.
So there's a loop in the middle of like,
how do you train that model?
How do you scale up? How do
you compare experiments, pick the right model?
When you've picked the right model,
how do you then prepare that model,
put it in the right container,
maybe start operationalize that model,
deploy it to prod and when that thing is deployed to prod,
how do you then monitor things,
promote things from the Dev stage
into proud when it's going to deployment.
When you think about things that way,
that's how we look at the real ML lifecycle.
It's not just a linear process,
it's more cycles that can go back and forth and then it's always
surrounded by some managing loop
because nobody wants their data leak.
Everybody wants everything to be in
your company or in your enterprise safe and secure.
>> Yeah, and as I'm looking at this,
this is actually really cool because up here,
the process is different when you
start building a machine learning model,
you really are doing experimentation.
But then when you move into
the training phase where you want this thing to be repeatable,
your data size increases,
the amount of people that you're working with also increases.
Finally, the other thing,
and I'm glad you brought this slide, it's one of my favorites.
I use it for some other things.
But when the difference between machine-learning software and
other software is if you
get the thing to work in regular software,
it just works but machine learning models could stop
working over time too and that's
what makes the weirdness even more weird,.
>> Exactly and you can see the loop backs and the top and bottom.
It's like you might have the greatest model to start with.
We need to monitor that model and see
how you can retrain that model.
So at the bottom you can see there's
a retrain and if things go really bad,
you got to start the project,
all over again and start this whole loop.
So you're completely right.
It's not like a static thing.
Data changes, the AI,
the model gets smarter, it learns more things,
there can be skews in the data, skews in the model.
So definitely that's something you
got to take into account and that's why
these cycles are all cyclical.
>> Well, a, this has been an amazing process
that you've been talking to us about.
But one thing I would like to ask you is,
maybe you can give us a preview for
part 2 because we've talked a lot about a lot of concepts here.
But I feel like how can people,
if they're getting started want to do something like this.
>> Yes. So just a little sneak peek of what's coming up next is,
I don't like to talk too much about all the process
I want people to start doing so next,
we're going to talk to some folks on our team that I work with,
they work with a bunch of customers,
how they scale up the ML practices and we're
going to talk through what are the right process,
what is the right architecture?
What they've heard from customers,
maybe a little bit of an accelerator,
and how you can get started yourself.
So I don't want to give away too much.
But I'm going to let you stay tuned
and listen to what they have to say.
>> Awesome. Just a reminder,
if people want to learn more about this, where can they go?
>> If you want to learn more about it,
like lucky for you we just released
the new MLOps module on the Microsoft Learning website.
So if you head over to Microsoft Learning website,
the link should be on the screen.
You should be able to start upskilling yourself and MLOps,
then you know how to use this for
your enterprise or maybe even your own small project.
>> Awesome. Well, thanks for being with us today Abe.
>> Sweet, thank you so much Seth.
>> Awesome friends. You've been learning all about how to scale
your AI/ML practices with MLOps and Azure Machine Learning.
Part 1 of 3.
Thank you so much for watching and
hopefully we'll see you next time. Take care.
[MUSIC]