Welcome to today's session of This Is My Architecture.
I am Kwunhok from AWS Solutions Architect.
Joining us today is the date strategy lead of Ztore - Dennis.
Can you brief us on the background story of Ztore, Dennis?
Sure, Ztore is a Hong-Kong based E-commerce hub.
It was founded in 2015,
and has more than 250K users.
We mainly sell groceries,
particularly those special items like food and cooking oil.
These years, we focus on machine learning technology.
It can help us distribute our products to our users more efficiently,
and also help users find products that meet their needs.
Why don't you tell us about
the type of service you use to apply machine learning?
Sure, the tool we use to get machine learning up and running
is called the recommending system.
The system uses a very simple method.
It will train the new model by batch daily,
and then generate each user a recommended result directly
before finally saving the result in the DB.
API will then read the results being saved in the DB.
Is this Architecture behind us what you're talking about?
Exactly!
Now, let's invite Dennis to introduce to us how this architecture works.
Sure. First, we will use the Scheduler from CloudWatch
to schedule a P2 easytool instance.
It retrains our MM model everyday.
It also provides the service of
generating a recommended result for every user, and writing it on DynamoDB.
When our user logs onto our website,
our load balancer will pass their IDs
to the API in our Microservice.
Our application will then
find the corresponding recommended results on DynamoDB.
Our application is run by Fargate of ECS.
Did you choose Fargate to avoid managing the infrastructure?
We did indeed. The recommended system is deployed and maintained
by the data team of Ztore.
Our data team is formed mainly by a group of data scientists.
Therefore, we don't have staff who can focus solely on data faults.
We do our best to shorten the process of development,
and also make sure MM Model works smoothly with the basic infrastructure.
That's why we choose to develop the system ourselves.
What do you think is the most challenging part in this architecture?
For us, the most challenging part is the possible massive fluctuation
from the traffic of our website front end.
We need to ensure its scalability.
Therefore, we chose DynamoDB as the core of this architecture.
DynamoDB is a serveless key-value database.
Its read/write capacity can be auto-scaled.
That means we don't need to maintain a DB's instance ourselves.
Plus, when our traffic has a spike,
it can auto-scale the capacity to meet our needs.
We know that you use the auto-scaling of DynamoDB to manage the database
but how about application? How do you ensure the scalability?
We use ECS deployed by Docker for application.
It has an auto-scaling feature as well.
Based on the API loading of the Microservice,
ECS will scale out the number of Container.
Our Container will then run with the server-less Fargate Engine.
We won't need to maintain the cluster under our Docker.
Auto-scalable then becomes more convenient.
We have actually conducted a stress test for our system.
Even with the same config, we can take up to 100 time more data than we have now.
It can be automatically scaled.
We built a lot of CloudWater Dashboard and alarm
to help us monitor.
The whole system's overhead became very low.
This way, our data team can focus on enhancing our model.
That's great! You've been running this stack for a while now,
do you think there's any improvement you can make?
Soon after we deployed this stack, we came up with a second Use case.
This is the cart to item recommendation.
Based on the items users put into the shopping cart,
a recommended result will be generated and sent instantly.
This is a bit different from the architecture I mentioned earlier.
Because after a model retraining,
a list of recommendation results will not be presented.
A whole MM Model will be the output.
Our application will need retaining,
in order to load this new model.
After training this model,
how do you deploy it to the application?
We have considered many methods.
For example, is it possible to place the parameter write of our model into a DB?
Or can we push a "post to persist" model into the application?
We worry about application overloading with these methods.
Using some CICD tools is another common method.
After each training session, it will rebuild the application before deploying it.
Based on the features of ECS,
we came up with a plan that suits us the most.
After it has done the retraining every time,
it will write out S3 of the model.
Then it will signal ECS to deploy the whole application.
The new application will be able to read that model under S3.
You mean every time there's an update,
and a new model comes out, it will enter ECS and update.
And ECS will load a model through S3, the new container task?
That's correct.
How do you ensure this update doesn't contradict your production?
We do that through the rolling deployment feature of ECS,
which means whenever we redeploy,
ECSh will first produce a new container,
and wait for it to slowly load the new model.
After that, ECS will conduct a health check on the new container
to make sure it's healthy and ready.
Only then will it redirect API's request to the new container
and cue back to the old container.
The whole process does not affect the service level
and is also automatic.
That's amazing!
If you do this training by batch daily,
have you ever considered doing some real time model training?
Sure we have, our next goal is to do a real time model training.
However, we are still designing the model right now.
We expect the requirement of computation to grow rapidly in the future.
The next challenge we will face in our architecture
will be maintaining a low cost and easy architecture,
while handling the original computational requirement.
How can AWS help you?
We are now evaluating the distributed processing tools of AWS.
For example, Tools, Frameworks and EM.
We also try to use AWS' new tools to personalise this service.
I hope Amazon's Personalise can be helpful to you.
Thank you, Dennis, for sharing this architecture with us,
and thank you all for tuning in to This Is My Architecture.