Welcome to another episode of 'This Is My Architecture'. I'm Aarthi.
And joining me today is Xin from Zillow.
Xin, thank you for joining us today.
Thank you for having me here.
So Xin, everybody knows what Zillow does.
Talk to us about what your team specifically does at Zillow.
Okay. I'm part of a Zillow artificial intelligence group.
So our group mostly doing a lot of advanced NLP machine learning
speech to text solutions to help our internal business units.
So internal business units,
so what problems are you specifically trying to solve for them?
So one of the problems is called speech analytics.
So we have lots of phone conversation data with our customers.
So there's a natural need for how to utilize and analyze the speech data
to do better quality control and help our customers.
Okay. So why don't we dive into the architecture
of how you do the speech analytics?
Sure. So Zillow currently working with different telephony systems.
So first step we have is we will grab
the phone conversation data into a dedicated S3 buckets.
From there, it triggers a Step Function execution.
So this is, we call the audio ingestion process.
So you've got the data ingested.
What is the primary role of the step function here?
So step function has a very important role
in the what we call audio analysis phase.
So for all the audio data, first they will go through the AWS Transcribe.
Basically, here is speech to text.
We get the transcripts. After you have the transcript,
we also call Comprehend.
Here we can generate the sentiment for all the utterance of the customer, for example.
And on top of that,
we can run many other models.
In the end, all the results, intermediate results
will be pushed into a S3 bucket.
So your Step Functions, you're using a sequential pattern here
to go to Transcribe and then move in to Comprehend.
There's a lot of consumer data here, right?
How are you taking care of your consumers,
like, privacy and compliance?
Yeah. I think we have two different approaches here in place.
First, AWS Transcribe is very nice to have a PII reduction functionality.
So we turn that on, which means all the consumer PI information,
which are in the transcript will be removed.
The otherwise, Zillow has a dedicated team to make sure
all the processes data store internally follow the CCP compliance.
Got it. What is the role of this intermediary S3 bucket
that you have here?
You've processed all the data and you're storing here.
Like is there any other information that goes into this S3 bucket?
Yeah. As I mentioned here,
we simplify this beside this Transcribe and Comprehend,
actually we run lots of data enrichment
to get lots of information about the call.
And also we could run customized NLP models to get more insights.
Also, the information will be pushed into a Data Lake location.
So here the analyst data scientists can run analysis modeling on top of that.
Awesome. So how do you then take the data
from that intermediary level to your end users?
So here we do have another process going on here.
They will grab the data from the S3 bucket.
They then are pushed to a Firehose. Firehose acts as a buffering layer.
Eventually, data will go into the Elasticsearch cost.
Why did you choose Elasticsearch as your final destination?
So this product is mainly to serve internal customers,
the ability to search for phone conversations
is very important to them,
and they want to search by different features.
Here, Elasticsearch acts as a natural fit for this purpose device.
So another benefit of Elasticsearch is
the autoscaling functionality we have seen.
So we've got data in the Elasticsearch, right?
So your internal teams, your business units,
how do they consume this data?
So now this product is for internal customers.
Currently, we use AWS Cognito
to do the authentication and user management,
and they're on the UI.
They can do search or their search criteria will be in the back end,
they will be transformed into a search query.
The query will be run against Elasticsearch.
Besides that, they can also grab certain call URLs to replay that call.
And in that case, we generate a preassigned URL for the users to interact with.
Okay, so depending on their search criteria,
whether it's a phone call that they're searching
or keywords that they're searching,
they can look into the Elasticsearch from the UI.
Exactly.
This is a neat architecture.
Are there any enhancements that you are planning into this architecture?
Yeah, definitely need.
So one of the things we are thinking about here,
they are more in this area.
So as you can see here, as we add more functionalities into the system,
not only search, grab URL, comments, and many other more.
Naturally, you will make a lot of sense to simplify the architect here.
Here where I think about using the AWS GraphQL or AppSync.
You will simplify the design here.
So first we have we will have a proxy GraphQL as a proxy.
Then in the back end, we can add lots of resolvers as we need.
We expect here there will be efficiency benefits
as well by switching to that.
And that can also simplify the Lambda functions as well.
Yeah, sure.
Xin, this is a neat architecture of simplifying the use case
for your internal teams of how to get this analytics
from the speech that is all ingested.
Thank you for sharing this architecture with us today.
And thank you for watching 'This Is My Architecture'.