MARK MIRCHANDANI: Companies looking
to make predictions require vast amounts of data,
complex calculations, and rapid responses.
Join us as we talk to LiftIgniter
about how they built the machine learning platform
on this episode of Stack Chat.
Thanks for joining us.
Tell us a little bit more about LiftIgniter
and how you built your platform.
VENKAT VENKATARAMAN: Thanks for having me.
LiftIgniter is a real-time personalization platform
powered by machine learning models running in the cloud.
By real time, we mean end-to-end latency
in the order of about 100 to 150 milliseconds.
And this includes the network latency as well.
Now, this is a real challenge both in terms
of scale and computation because we
track a number of signals by users
in order to provide them with the most relevant
and personalized recommendations.
Our world view is that each person is unique.
And at every moment in time, our platform
is optimized for the most relevant results
based on all the signals we gather up until that point.
MARK MIRCHANDANI: Can you show us the architecture
you built to keep this moving?
VENKAT VENKATARAMAN: Sure.
As an ML company that crunches and stores
huge volumes of data, infrastructure cost
has a large impact on our business.
We focused on three main factors--
pricing, latency, and reliability,
just like anyone else.
To emphasize these, we used three main techniques.
Auto-scaling based on workload, lets us make sure
that we have the flexibility to scale out
heavy workloads on demand.
And we use a combination of auto-scale groups
either for preemptible instances and for on demand, which
leads us to the second point, which is the flexibility
to allow us to use preemptive instances as
needed, and thirdly, the option to move
stale but useful data into cold storage, again,
for cost reduction.
When it comes to speed, network latency usually
has a huge part to play.
But Google's network has been pretty snappy
in our experience.
We have been able to keep components that chatter a lot
close by within the same region with enough redundancy.
With Google's Footprint, we're able to create and run services
closer to the edge so that all the customer-facing services
perform really well.
And we have also been able to avoid
any outages that'll affect our SLA, thanks to Google Cloud.
MARK MIRCHANDANI: Can you tell us a little bit more
about how the recommendation system was actually built?
VENKAT VENKATARAMAN: Sure.
The diagram is actually a simplified architecture
diagram.
We are not showing the load balancer, and proxies,
and things like those.
But it just focuses on the main components and the services
that we have in our architecture.
The requests to our cloud are either
going to hit the front-end APIs, or what
we call as the model servers.
The model servers are the ones which
for anything that has got to do with user
activity and returning recommendation results,
that's taken care of by the model servers, where
all the magic happens.
For everything else, like, for example,
updating or creating new inventory items,
or making other API calls will actually
hit our front-end services.
Now looking at the front-end services, when an API
call comes in, a bunch of processing
happens on those front-end machines.
And the front-end services, they push the data
into data storage.
We create either files or other data,
push it into some data storage like Google Cloud Storage.
We also have other data stores.
And GCS would actually trigger an event
into Cloud Pub/Sub where we have a lot of subscribers listening
on messages on those topics.
And a subscriber could pick up those, do further processing,
and an example shown in the diagram
is where we have an inventory system which
does further analysis and processing based
on what the trigger is.
The model servers are the ones, like I said before,
which handle all of recommendations and the user
activity tracking.
They do a lot of processing in-house--
a lot of magic happens here, a lot of heavy lifting happens
here--
pushes the data, once again, into data storage.
A similar mechanism happens, which triggers an event
into Cloud Pub/Sub.
And we have subscribers listening
on those topics, which pick the data from GCS,
do further processing.
And as shown in the diagram, this data
is now pushed real time into BigQuery
where our dashboards are fed off of BigQuery for analytics.
The key thing here is while we can move things
into BigQuery in real time, BigQuery can also
get a little expensive, so we want to keep access to BigQuery
only when we really need it.
So we have an offline processing which takes data off
of BigQuery, does further aggregation,
pushes it into Cloud SQL, which acts like a caching layer.
So the dashboards would either hit Cloud SQL our BigQuery
depending on how far back in time they
want to go and gather data.
The box at the bottom shows our Spark cluster, which
runs off of Cloud Dataproc.
And the Spark cluster is basically
used for tuning our models and retraining it
for better accuracy.
We run it on a nightly basis.
We gather all the data from GCS.
We have the models running all over again.
And if the results of those models with the tune parameters
are better than what it was before,
we make the necessary changes and push those configuration
into the cloud.
MARK MIRCHANDANI: Now, it's been a while since you
architected this originally.
If you could go back and do it again,
what would you do differently?
VENKAT VENKATARAMAN: We started architecting our platform
around 2013, 2014 when machine learning was still not hot.
Big data was still the thing of the day.
It wasn't as hard or widely available as it is today.
If we had all the libraries and frameworks that are available
today back then, then some of the
build versus buy decisions would have been different.
We also enabled augmented intelligence
by giving our users the ability to influence recommendations.
Sometimes they may implement rules
that can get a bit too restrictive
and end up returning less results.
To help enable the customization for these users,
we've been researching indexing solutions
to give better results.
Part of this was investigating something
like Elasticsearch, which was good,
but it wasn't performing as fast as we
needed because of a really tight latency requirements.
We're currently looking into other ways
to build this and keep serving improve recommendations.
MARK MIRCHANDANI: Thanks so much for sharing the details.
VENKAT VENKATARAMAN: My pleasure.
MARK MIRCHANDANI: If you'd like to try setting up
Pub/Sub notifications for Google Cloud Storage yourself,
check out the walkthrough in the description below.
Thanks for watching, and make sure to subscribe
for more great Google Cloud Platform content.
We'll see you next time on Stack Chat.