- [Moderator] Thank you Mattei, Clemence and Sean.
And it looks like we do have a few minutes
for some question and answer
with Mattei, Clemence and Shawn.
So gentlemen,
let me read off some questions for you.
So the first one is for Mattei.
Which components of MLflow, Mattei,
are the most widely used?
- [Mattei] Yes, so the most commonly used
and the one that people start with
is usually experiment tracking.
But after that, we also see a lot of use
of model packaging and the model registry
as a way to, you know, to share models
and keep many versions of them around
and send them into different deployment environments.
- [Moderator] And the next one is for Clemence.
So, generally people are curious what's next in the roadmap,
in the areas of model governance
and the model registry.
- [Clemence] Mm-hm.
Yeah, so, on the model registry,
there's a couple of interesting items.
Right now we usually start with an API first approach
and as actually mentioned earlier in my talk,
we have some customers that actually use the model registry
to train tens or hundreds of thousands of models
and deploy them.
That requires a lot of sophistication
and they use our API surface
to manage that CI/CD pipeline.
So we're slowly making that easier
and we're actually planning to provide hooks
in the model registry so that you can more easily integrate
the model registry with your CI/CD systems
and manage the deployment life cycle of these models.
So in short, I think a lot of the work
that we are investing here
is to make, making it more accessible
and more easier to actually manage
the envelopes lifecycle in the model registry.
And then there's also a couple of other,
bigger roadmap items
that I'm not going to teach here.
We're actually gonna talk about them
at the Spark Summit in June.
- [Moderator] Another reason for you folks to sign up
for the Spark Summit in June, which is online
and will be free.
The next question is for Sean Owen.
So Sean, how does the notebook
integrate with system hardware?
So someone wants to know, for example,
GPUs and CUDA integration.
- [Sean] Yeah, right, on Databricks,
Databricks is gonna manage all the hardware for you.
So when you start a cluster, you can for, if you like,
choose a version of the runtime that supports GPUs
and provision GPU instances.
And everything else is taken care of for you.
So CUDA will be installed,
TensorFlow will be installed for you
if you're using the ML Runtime.
And even more than that, Databricks will do things like,
if you like, scale up the cluster
and scale it down according to load,
and even shut it off when it's not used.
So you generally don't have to worry about the hardware
or getting the libraries installed, that's done for you.
- [Moderator] So the next question is for Mattei,
and, Mattei, generally-speaking,
one of the areas
that people consider
to be a main bottleneck,
at least particularly for deep learning,
is distributed training.
So how does one go about distributed training
of a machine learning model that is not MLlib-based
in Databricks?
- [Mattei] Oh yeah, great question.
So we do support distributed training of TensorFlow
using the Horovod package
and also using the distributed training strategies
that are coming out in TensorFlow itself,
in TensorFlow 2.0.
And it's also possible to do this for PyTorch.
PyTorch has been adding more support
for the distributed training as well.
So we've done a bunch of work
to make sure that the machines are configured
with high-performance networking
and with high-performance shared file systems
so that your distributed training job can run on them.
- [Moderator] Next question is back to Sean,
Sean, if all the modeling steps
are being done in the notebook setting,
so how do these transformations get tested?
So how am I sure these transformations
are ready for production?
- [Sean] Yeah, great question.
I mean, obviously you wanna test your model in some sense
before you turn it live.
So Databricks supports that in a couple ways.
Number one, a notebook could encapsulate
some testing code.
It could load a model, it could run some smoke tests,
test it against a golden set,
and that itself could be scheduled through a job
in Databricks to run regularly,
or indeed could be triggered from an external system
like Airflow or Jenkins, if that's your,
the tool you're using to orchestrate and run CI/CD.
So I would do Databricks as a place to run code
that can be triggered,
can be run from many different places, flexibly.
- [Moderator] Next question is for Clemence.
Does Hyperopt in Databricks
differ from the open source version?
And Clemence, you might as well explain to our audience
in general, if they're using an open source tool,
say TensorFlow or PyTorch,
how do Diversion and Databricks
differ from what's available?
- [Clemence] Yeah, that's a great question.
So in general, we really want to provide you
with the best open source tools on Databricks
and just make sure that they work out of the box.
So the short answer is,
when you use something like TensorFlow, PyTorch,
or Hyperopt on Databricks,
the libraries and packages themselves are not different.
So like, we're not changing the APIs
or like we're not changing the way
that you're used to be using them in other environments.
Now, in addition to this, we usually provide improvements,
in some cases in performance,
or in like distributing these workloads on Databricks.
I'll give you one example, specifically for Hyperopt.
So the Hyperopt API itself is exactly the same.
We just package it up with the machine learning runtime
so that you don't have install it.
But then we actually extended Hyperopt
with this functionality that Sean showed earlier
that's called the SparkTrials.
And it actually allows you to distribute
your hyperparameter tuning trials using Spark.
So like, each one of the spark workers
just runs a trial in parallel,
and that gives you a performance advantage.
So we usually provide the standard API,
standard open source tools,
and then provide additional benefits on top of them.
- [Moderator] Next question is to Sean.
So Sean, in addition to notebook based model development,
can someone also leverage Databricks
to implement CI/CD best practices?
So this is Git, building deployment pipelines and so on
to automatically retrain and verify models on a schedule.
- [Sean] Yeah, I think a similar answer.
So if you've got some code in a notebook
that trains a model,
you can trigger that on a schedule easily
with something like the job scheduler built into Databricks.
You can also train it externally as well,
or trigger it externally
through a tool like Jenkins or Airflow.
And the MLflow in the model registry
has a role to play there.
For example, if you've registered a new current model
as a staging candidate for your production model,
you might have a job that grabs that from the registry
from MLflow, runs some tests on it via Databricks
and then updates its state to production, for example,
to indicate that this has passed test
and this is ready to go.
So yes, you can absolutely do that.
You can also use Git to back things
like your notebook revisions as well,
but that's a little bit separate.
But yes, Databricks is a place where the code executes
and the models are managed
and it can flexibly be run and triggered and driven
either internally or externally.
- [Moderator] And with that,
that's all the time we have for this live Q and A
and a quick reminder,
all these gentlemen will be at the Spark Summit in June.
So make sure you register, too.
So if you wanna try
some of the things you saw in the previous (mumbles)
and as well as the demo, go to databricks.com/try
where these tools are available.
As a reminder, you can continue to submit your questions
via the Q and A panel, as we carry on with the presentation.