- Hey guys, I'm Avi and I want to share with you
our achievement in our technology breakthrough regards
in my last role as Head of Data Science in the world
of ethic and advertisement so let's start.
So a little bit about myself I'm currently
the Director of Data Science and AI at First Digital Bank.
It's a new bank in Israel maybe you are going
to hear about soon.
And I have an experience of 10 years at AI, big data,
machine learning, deep learning in different use cases
and usages and also within the relation systems.
So what is the problem that we fight literally?
The problem is that we got a billions of events in our data,
millions of clicks for example in the impressions
on different kinds of ad units for advertisement purposes.
And we used different kinds
of more classic machine learning tools to build a classifier
to predict if someone's going to click or not click.
It's called CTR on the specific ad on specific requests
for publisher and we have a lot of painful trade-offs.
We had a skewed feature we have a lot of skewed...
The main skewed feature was the ad unit because
every ad unit performed in different way behave
in different way with different label data
for every ad unit.
And when we tried to build one model for example
one XGBoost classifier model for this goal,
we got a very low AUC an accuracy for this model
in many types of ad units and requests.
And we thought how to handle this in which research analysis
we need to do.
So the research we done is the first assumption that we had
and we want to check and research on is to split the data
to smaller data sets per ad unit.
Because we saw that every ad unit has a different behavior
between each other and when we built in our lab and model
for each ad unit and for each data set for this ad unit,
we got much higher results in accuracy and AUC.
So we understood we want to build a model for every ad unit
and it caused new problems and issues.
The first problem that that we got,
is to train 20 models for 20 ad units in a much,
the fastest we can and when we run it one by one by other,
we finished to plan all the models in eight hours,
and then we ran the variation and so on and decided
if we want to deploy to production or not.
So it was only the absolute because well futures
with new values that we didn't try that came,
with data in the last eight hours in the model
for example campaign ads new campaign ads that started in
the last eight hours and the model was obsolete for those
new volumes in the features.
The new issue here was when we tried to run it on the master
of the class that we prepare the data for the model because
it was new data you start to prepare the data prepare
the data set for the model and on a cluster
on huge clusters.
And when we collect the data to the master and then run
there the model but it didn't use the cluster
the cluster power only used the master.
Also if you have like auto-scale and the other instances
where already there we want to for example use the cluster
run all those models and this would scale a bit of weight
So other ideas that tend to it was to use for example,
Airflow to run the models on Docker and we found ourselves
finding pipelines for each type of model.
And it was very problematic because we have different kind
of different amount of data for every model for example
we had a one ad unit with 100 million rows and we propose
for me a sample of 10 million and audio was like 1 million
rows and we used all those rows.
So the difference was from one million to 10 million.
And if you want to optimize it
you can choose different types of instance,
a Docker instance into one of those models.
So and all those types of things can change in one month,
it can change from one million rows in a specific ad unit
to five million rows.
So we don't want to maintain this all those pipelines
for all those 20 models.
And we thought maybe how to continue using our
own technology we already used and how to we can leverage
them to solve these problems.
So we did a bit about the machine learning model life cycle
and how we want to in the end want to run it in
distributable and scalable way.
So we have the first step it's like the data preparation,
data cleaning if you want to do for our old level data,
we want to prepare for different kind of formulations
and maybe futures, for different kind of models.
And after it where the data transformation for
a specific model to prepare the sophisticated data set
for planning and validation in this pipeline after
we confirm the data for specific model dependent
and validation data we're starting to train the model.
And after it's in circles we wanted to do
a hyper parameter tuning and again and again after we
are getting the best model that we can,
after doing validations see that we have good resolve
of our metrices.
After we of course doing the prediction that we want to do
is more a offline example, we're doing the provision
on the model and then we're starting to do
a model evaluation because we want to see if it's kind
of a positive validation step that we want to see that
the tool data that came there's the predictive data
and check our metrices again on this post validation.
So before I'm jumping through our solution to
this main problem.
I want to share with you a little bit about the background
of our tech solution both all the technologies that involved
in this solution wherever there we still want to use Spark.
We are preparing the other data and since Spark 2.3
we have integration with Apache Arrow to improve
the serializations between the Spark dataframe and
the Pandas dataframe for example and they
also add different kinds of tools like Pandas UDF that
we want to talk about in a second.
So Apache Arrow helped us a lot to improve and optimize
the memory usage and the CPU and or GPU usage.
The Apache Arrow is doing all the segregation
in much better way in terms of memory.
And also in terms of CPU computing, because it uses a much
more suitable data model to save the time in memory
and do the compute on the CPU.
So all of this is learning on the background and also
this integration with Apache Arrow between Spark
and Apache Arrow come with new features for Spark.
For example Pandas UDF it's gonna help us for example,
to run a groupby on the data on specific columns
and pass a Python function that we want to do on
the group of data.
And this calculation the Python function will run all over
the cluster all across the instances of the clusters
in walkers of Spark.
And of course running it in this bit way caused a cluster.
So it's new way to combine between to conform
between Spark dataframe to Pandas dataframe
and run the Python code on the Pandas dataframe it costs all
the cluster of Spark.
Quick example, you can run in the groupby yes you could
have groupby you can pass your if you want to do groupby
I can give her a teaser if we are going to want it on it
on a ad unit column and then paste to apply the function
you want to run it can be any Python code or for example,
algebras code to train and validate and do
the post validation on the model.
So here we come our solution the most important thing that
we want to achieve here is to first of all the improvement
in accuracy by running 20 models in stable way
but also save to find most simple way to do this
in more generic way for example one to one like 500 models
or 20 models for very different uses.
So in this way we can support like 500 run 500 models
in parallel and without any much code changes,
just change the groupby column that we want to run
the models on.
So here we can do a more deep dive on the data preparation
for in our solution.
So we have here mass compared to our last solution it's much
the same where using Spark to prepare the sample
for the model.
The main difference here that we are for example going to
use 10 million rolls of sample for each model we want
to prepare for each ad unit we want to prepare
a model for.
So instead of using 10 million roles for 28 units
we are going to use 10 million rolls for each ad unit
and it's going to help us a lot to prevent
the imbalance feature that we have in data it's simply
the ad unit because if the ad unit
performs an X much different from each other.
So to do this we can prepare the data set
for all the ad units we top off 10 millions rolls for each
and then run the Pandas UDF upon this dataframe and pass you
the groupby the selected features that you want to run model
for each value.
So here we want to put ad unit in our use case and in
the apply we can pass you the the CTR function is going
to prepare for us the CTR model to run eligibles model
to run the validation and of course run the post validation.
So the post validation it's little bit different because we
are running it as the first step in our pipeline.
For example if we ran yesterday model and we want
another one a new one today we first of all run
the post validation for all the models in the same way
we just want to run for each ad unit we want to run
the post validation for it's model and send the same
same metrices and results on those models.
So we're turning all the model life cycle train
the data preparation the planning the validation
and post validation in the same way by using Pandas UDF.
And the main difference here it's that the Pandas use
the S function that we want to first to apply,
after we apply is going to run the course all the cluster
within all the instances in the cluster.
So by running all those models in distributed way all
the life cycle of all the models in distributed way
by leveraging the same acknowledged we order the use Spark
and eligibles for example it can also support
a TensorFlow with a TensorFlow connector.
We can run all those models in one hour in parallel
across the cluster the main achievement here is
that we don't need to handle all the maintenance.
The Spark already doing all optimization
in resource management and for example, we saw in
the cluster we saw that in specific machine we
have four models that runs the course the walkers
of this instance in the same in parallel,
because every model had like two million rolls
but in the same when we look in different instances in
the cluster we saw that we have one instance
that runs one model with 10 million rolls
because we passed in the Spark submit we passed
more dynamic parameters.
In this way we use the cluster in most efficient way
instead of for example manage a pipeline for every model
and always to think to choose the right instance for
this model Spark already set in the same way that
it handles data it can handle it in this use case also
the model in the same distable way.
In this way we can support not only 20 models we can support
like 500 models and we already started to work in it
for example run and model for each campaign and for
each brand and ad unit.
So we'll be glad to have any feedback from you guys,
thanks a lot.