>> You're not going to want to miss this episode of
the AI show where we make our AI,
create an image from tags that you can post to Instagram.
We'll show you how to make
those discrete steps on
the next episode of the AI show. Make sure to tune in.
[MUSIC]
>> Hello and welcome to this episode of
the AI show where I've got a special friend with me, Santos.
Why don't you introduce yourself.
>> Yes, my name is Santos Pulay.
I'm a Program Manager in the Azure Machine Learning Team.
I worked with the pipelines feature.
>> Fantastic. So before we get into pipelines,
let's talk about a workflow for
a typical machine learning scenario
because I feel like this sets
the context for what we're working on.
Tell us what a good workflow is?
>> At a high level, a machine learning
workflow will have three major phases;
the data preparation phase,
and the training phase,
and the deployment and scoring phase.
So in the data preparation phase,
you have data ingestion,
validation and transformation etc.
In the training phase you will use
leverage features like automated machine learning,
hyper-parameter tuning, and the custom model building.
In the deployment phase, you deploy score and
monitor your models that are ongoing.
In the overall mature workflow,
you will also have retraining as well as batch predictions.
>> The thing is like, and I have done these people,
if you put this all at in one script,
it takes for ever and
then you like an hour in and then something breaks,
and that's not a good thing and then you
have to start all over again and it's not very good.
>> That's when you break these down into module channels.
You can break it down to two phases.
Like data proportion could be
one phase and training would be another phase.
You will have different steps for individual phases.
You can use pipelines for individual faces or
you can use as an Uber orchestrator for the end-to-end scenarios.
>> So let's start. You mentioned the word pipelines.
We talked about the standard machine learning workflow.
How do pipelines help with that?
Tell as what pipelines are and how they help?
>> Sure. So even if you're just having a single-step,
to run a single model training,
you will start realizing the advantages of using pipelines.
Then you start from exploration to iterations.
So usually, when you need to iterate many times
over to perfect your machine learning model.
So for example, one of the things is reuse of the steps.
A step is a unit of computation.
So when you really want to redo the steps over and over again,
we have something called reuse of steps.
So that means that reuse of computations that are already done.
In that case, you are saving a lot of
compute course just by reusing the step.
In an interplay scenario,
we're talking about 30 percent savings on the compute cost.
>> That's amazing because when I'm running, here's an example.
I do Tensorflow stuff all the time,
and I'm a computer vision person,
and so what happens is,
I crunched through all these images,
and then the training breaks
then I have to go and re-crunch to all these images.
If I'm understanding you right,
you're telling me I can put
the crunching of images into a single step,
and that data will just be there unless something changes?
>> Absolutely. So if the input and
parameters of the step remains the same,
we're not going to rerun that step unless you
specifically want us to tell you, tell us to do so.
>> I see and that was my next question.
How do you know when to run it again,
and you're saying that when the input and
the parameters are different then you need to rerun otherwise,
just reuse that stuff.
>> Exactly.
>> Awesome. So how does one actually set this up?
You talked about the optimizations
and how it's actually pretty powerful,
but is it hard to do?
>> It is not very hard to do.
You can start with Jupiter notebooks and you have
constructs to build up individual steps in Jupiter notebooks,
or straight to Py files.
So once you have finished
building a pipeline for an individual phase,
you can just publish that pipeline and get a rest endpoint.
So what that means is that then the color or the
teams who are using your data processing phase for example,
they can just recall into that rest endpoint.
They don't need to know the details of the pipeline's building.
>> I see. Let me see if I understand this right.
If we're putting these pipelines out to a rest endpoint,
that means anyone can call them and change the parameters,
and if the parameters are changed then
only the steps that need to run are up.
>> That's exactly right. So if you are
authorized to call into that rest endpoint,
yes, you can call into rest endpoint.
You don't need to know how it was built,
and what are the optimizations that are built into it.
You only need to know what parameters you need to
use to run into that workflow.
>> So if I'm understanding this right,
and then maybe we can get into a little bit what this
looks like inside of Azure Machine Learning Service.
So basically whenever you're doing machine learning process,
the first thing I need to do is separate it into discrete steps.
These steps then I can package it into a
single like Python file for example,
then what I do is for each of these steps I
create a set of pipeline things and then those tasks can run.
>> Yeah. So the steps
can be stitched together using data dependency.
So if you say a step is going to produce
some data and the subsequent step is going to use that data,
pipelines will automatically figure that
out and wait for that step to finish,
or the data to be available before the next steps happens.
So you would construct your steps in such a way that
there are certain model of channels that output a certain data,
whether it is ingestion data, or transform data,
or the output of a trained model, which is a model,
then you move on to the next step
as to what to do with that output,
whether it is a data or a model.
>> That's interesting because
implicitly what happens is, when you build these steps,
all of a sudden there's like a graph
of things that need to happen,
and you're saying that the graph is
implicitly constructed by the data dependencies?
>> Absolutely, or you can say,
like if there are no data dependency,
you can actually force some steps
to wait for one of the previous step to finish.
So there are other constructs that enabled you to do that as well.
>> So there's implicit graph of tasks,
and then there's an explicit one.
What's cool about this is if you have
certain ones that have no dependency,
if you set up your computer correctly these can
run on different nodes at the same time.
>> Correct. By default,
all the nodes that have all the dependencies
satisfied or no dependency will run in parallel.
You can make them in sequence by explicit constraints.
>> All right. Well I feel like we need to look at this now.
>> All right. So here is a graph that actually is built
for creating an image from a pump,
and that image gets published into an Instagram post.
So this graph generates
an Instagram post every time the pipelines runs.
>> So hold on. This is crazy.
So you're saying that this will actually from some text
create an image that you can put out to Instagram.
>> That's exactly right.
>> Let's go to this task because that sounds
crazy and I want to see what steps there are to do that, so let's go through that.
>> All right. So here we are looking at
the three data sources on the top
that are the data sources for the creation of the poem first.
So it is automatically generated
based on the data that we are feeding into.
>> I see.
>> Then we are pulling them into the text to image step,
which means that once the data is generated,
it is going to generate an image
based on the text that is provided.
>> I see. So I'm looking at these and
there's database looking symbols next to things,
and check-marks next to the other.
Can you describe the different kinds of
steps that exists in a pipeline?
>> So these are called nodes.
The top ones are beta nodes.
So there is no computation involved.
It is pointing to data.
The ones that have check-marks are
libraries which we call modules or steps.
The green check-marks indicate
execution of that particular module has finished.
>> I see. Now I'm getting a better sense for this.
These can be used not just for training,
but they can be used for generation for any series of
computation that require like GPU type compute.
>> Exactly. So for example
during data processing you might be using Spark,
and for training you might be using CPU or
GPU compute clustering enamel compute,
and for deployment, you could be using ACI or AKS.
So pipelines can stitch together
these heterogeneous compute options using
different frameworks if you will.
>> That's amazing because each of
the steps are discrete and I imagine that
the output of each step stores it
somewhere so that it knows this is a discrete set of data,
and then you can reuse that data everywhere.
>> That's exactly right. The output is
stored into users data-store,
where the user can construct
a pipeline data object where you
can store that into the data-store.
Our back-end knows how to go and finish that,
and reuse in the next step.
So we'll keep that around for reuse
as well for the next pipeline that comes along.
>> Now this is amazing.
Why is it such a good business tool? What do you think?
>> Obviously, collaboration is a key.
For different engineers working on
different areas of the machine learning workflow.
They can work independently,
yet they can stitch together,
the intuit scenario in a single pipeline.
That also gives an idea about what others are building.
Again, people don't have to wait for
translating their data that they produce into the next person,
or the model that the data scientists
produce to helps engineer to actually go and deploy it.
>> So everything is all nicely wrapped up into one spot.
>> Exactly, and business obviously care
about the cost savings as well.
So reuse is a big cost saver.
They can also look at
the reports and monitoring that we have and say,
well this particular project doesn't have the ROI that we are
expecting so that they can make
educated business decisions as well.
>> Well, this has been amazing.
We've learned all about the machine learning workflow.
We've learned about pipeline steps.
How to do stuff in pipelines and separate your stuff out,
and then we learn about pipeline data
which actually can be reused,
so you don't have to reuse all your
compute and we've learned all about
how amazing it is to use Azure Machine Learning Service pipelines.
Thank you so much for being with us.
>> Thanks.
>> Thanks you so much for
watching and hopefully we'll see you next time. Take care.
[MUSIC]