PRIYANKA VERGADIA: So you've built your machine
learning models, and now it's time
to take them to production.
What do you need to do?
Stay tuned to find out.
[MUSIC PLAYING]
Welcome to AI Adventures, where we explore the art, science,
and tools of machine learning.
I am Priyanka Vergadia, and today, we're
going to talk about managing production machine learning
pipelines.
As you've seen through [? years ?] of AI Adventures,
machine learning is a lot of fun and is
much more than just a novelty.
Machine learning models power so many technologies in our lives,
and there are plenty of new and better models just waiting
to be applied to the real world.
So how do you transition from experimenting
with a model to actually moving it to production?
Well, just as you need a quality CI/CD pipeline
to maintain a robust code base at scale,
you need a machine learning pipeline
to help you monitor, transform, evaluate,
and serve your models in production.
What are the different steps involved
in a machine learning workflow?
Well, first, you ingest the input data set and split it
for training, evaluation, and testing.
Then, validate it.
Identify and remove anomalies from that data,
or handle redundant features and empty values.
You might have to address the detected issues,
maybe finding more training data for certain categories.
Then, you do feature engineering to transform the data
and enrich the features to improve
your training, at which point the model is
ready to train and tune.
Once trained, evaluate the model to make
sure it generalizes well enough to use in production.
When the model gets the green light,
you push it to production.
This production environment needs
to be powerful enough to support the rate of inference required
by your application.
Now, when does this ML workflow become a pipeline?
Well, it will be automated, and the automation
is necessary, especially if we want to deploy
these models in production.
So the question now becomes, how do we
automate the machine learning workflow tasks
to create an end-to-end pipeline for deploying
our models into production?
Here is a very high level architecture of how it works.
TensorFlow is running on TFX, which in turn is running
inside Kubeflow pipelines, and that's
running on GKE, leveraging the Google Cloud Services.
You can create pipelines with similar functionality
using both Kubeflow pipelines and TensorFlow extended SDKs.
We have an entire series on Kubeflow,
which I have linked below.
In this video, let's focus on TensorFlow
extended, or TFX, which is an open source,
end-to-end platform specifically designed
for scalable, high performance, machine learning tasks.
Here's how it works.
A TFX pipeline is a sequence of components implementing the ML
workflow tasks, and they are built
using open source TFX libraries, which
can also be used individually.
This is sort of a canonical TFX effect pipeline.
You can think of it as hello world pipeline.
These orange components come out of the box with TensorFlow
extended, but you can create your own components as well.
Once the model is ready, you deploy it
into a serving cluster in TensorFlow serving,
TensorFlow.js, TensorFlow Lite, TensorFlow Hub, or even
AI platform predictions.
Now, what is a TFX component?
It is essentially a task that we perform in our machine learning
pipeline.
It has a config, and it takes input from the metadata,
except for the very first component
which is ingesting the data set from the original data source.
It takes those inputs using an input channel, does its task,
and generates the output artifact
and then uses the output channel to put those back
into the metadata, and that's how components
communicate with each other.
The upstream components will produce an artifact
that the downstream component depends on,
and the data flows through the pipeline
by flowing through these dependencies in the metadata.
Now, what is metadata?
Well, it allows us to trace the lineage of the artifacts that
are produced, for example, what data is the model trained with?
Metadata can also be used to compare previous training runs.
It also allows tracking and reuse
of previously computed outputs which
saves quite a bit of time, especially for larger data
sets and models.
If the input to the component has not changed,
we don't want to rerun the component.
We should just be able to pull the last result from the cache
and reuse it.
Now, let's go back to our standard components
and understand them a little bit more.
First, off, ExampleGen helps us ingest our data from CSV files,
BigQuery, Presto, et cetera.
It splits the data and saves it off as TF examples and TF
record files.
StatisticsGen will make a full pass over that data
and calculate statistics about it.
That's important to understand our data, if it is new
and if it has changed over time.
SchemaGen then determines the schema
across our data, such as what are the types of our features,
and for the categorical variables,
what are the valid categories?
Example Validator takes both of those results and looks
for problems, things like the examples
that have the wrong type in the particular feature
or a categorical value that should not be there.
Transform is where the feature engineering is done.
It will convert the feature engineering
that we want to do with a graph that becomes part of our model.
Trainer takes that graph and the model
that we have defined using TensorFlow
and trains that model.
Evaluator takes that trained model
and does a deep analysis of the performance of that model.
It looks at not just the entire data set but also
the individual slices of a data to try
to understand how performance varies across different parts
of our data.
It can be used for fairness analysis,
like seeing if your model is performing differently
for different types of model users.
InfraValidator makes sure that we can actually
deploy our model to our serving infrastructure.
If both Evaluator and InfraValidator
agree that, yes, we can deploy our model,
our Evaluator has checked to make sure
that the model is actually better than what we have
in production, then the Pusher will push it
to one of our deployment targets.
So that was a quick overview of TensorFlow Extended which
helps us engineer our machine learning pipelines,
so we can deploy our models in production.
In the next episode, we will see TFX and [INAUDIBLE] pipelines
in action in cloud AI platform pipelines.
Until then, I would love to hear the topics that
will help you get better at machine learning and AI.
Drop them in the comments below, and don't
forget to like and subscribe.