Welcome to another episode of 'This is My Architecture'.
Today I'm joined by Aveek from Zillow.
Hey, Aveek. Hi, Adrian, thank you for having me.
Thanks for coming in.
So tell us a little bit about what Zillow does.
Of course. So Zillow is the largest real estate platform in the country.
We get the most amount of real estate traffic on our website.
And we do everything from selling, buying, providing rentals,
and providing home loans.
So all things real estate, come to Zillow, right? Zillow.com
So, tell us a little bit about the interactions that you have with some of your customers.
Of course. So one of the things that we do is connect our potential buyers
to real estate agents through phone conversations.
And that's where this architecture comes in, in helping our customers as well as our agents.
And here, we have a near real-time pipeline
that allows us to run machine-learning models on top of these conversations, to generate insights.
I can't wait. Let's dive in. Sure!
So it all starts by calling AWS Transcribe,
which converts the phone call speech to text,
and one of the important things in here is we use, we take data security very seriously at Zillow,
so we use the redaction API in AWS Transcribe,
and that scrubs off all PII content, for example, name, phone number, email address, from these transcripts.
So, once that is done, these transcripts are sent over to Firehose,
and we have a limit of one minute of buffering on this Firehose,
and every one minute we are spitting out a file containing these transcripts onto S3.
So, one file for one minute.
So, you've got an async pattern to deal with all that volume and all that data sitting in S3.
Tell us a little bit about what Lambda is doing.
Of course. So, this Lambda is to handle the trigger that is generated off of S3 for this event,
and this Lambda then pulls this container information from ECR
and creates a job and submits it to AWS Batch.
Right, so you've got your container registry maintaining all of the different models that you have.
Now, tell us a little bit about how you're using Batch.
Of course. So the Batch is where our machine learning inferences run.
We use transformer-based architectures for our machine learning models, so previously, of the R deep learn models,
and AWS Batch helps us centralize between ECR and ECS
and helps us scale up and down depending on our job load.
For example, when we submit a job, this job is put in a job queue over here,
and this job queue triggers the scale-up and scale-down in ECS.
So when there is a lot of load and we get a load of traffic, say, for example, in the mornings,
our cluster scales up, and at night, when there are no conversations happening,
we scale down all the way to zero.
So that's a clever way of using Lambda and Batch together to scale your EC2 container service.
So, what sort of savings have you seen there?
So just by choosing this architecture and not going with something, for example, Cron-based
on a Kubernetes, or doing individual services for the models,
because we run multiple models in a single container through AWS Batch,
we are saving up to 75% of the compute cost.
So, once you've done that inference, you've matched some agents, what happens next?
So what happens next is we put these predictions into 2 different channels,
one of the channels being an SNS topic.
So the downstream teams can then subscribe to this topic,
and then, in near-real time, within 5 minutes of the call ending, give these valuable insights.
And we also send it off to another Firehose
which buffers it for a longer duration and then puts it into a data lake,
secure data lake on S3, so that our data scientists can run analytics with Spark and Presto and what have you.
Great, so you're using SNS to give near-real time data availability.
Tell me a little bit about what the developer experience is like.
Of course. So, we use GitLab pipelines and GitLab bespells.
Our applied scientists, they build their models and they push their Python code
over here, to GitLab.
And the model dates go to S3.
And we take both of these and run in GitLab pipelines
and we push the new image to ECR, so this image tag would plug in
to a particle of GitLab commit and a particle of model date version.
So you've built a pipeline for your data analysts and they don't have to
ever deal with any of the infrastructure.
That's correct. So, if your model is not doing well in production, you get an easy way
to roll back or roll forward.
This is a great architecture.
Thanks so much for sharing it with us today, Aveek.
Thank you so much for having me.
And thank you for joining us on 'This is My Architecture'.