Hi. Welcome to 'This is My Architecture'.
I am Ehsan, and today I am joined with Sarath from Alef Education.
Hi, Sarath. Hi, Ehsan.
Thank you for having me today. Thanks for your time.
So tell us more about Alef Education before we start looking at your architecture, please.
Okay. Alef Education is an e-learning platform that wants to revolutionize education
through digital transformation.
Amazing. So I think you are doing a lot of great stuff
when applying learning to students.
So tell us more about the architecture, how you do that.
Okay, Alef Education is a one-shot platform for students, teachers, and authors
to work together to build a personalized education for capable students.
We start with collecting information about students who are learning in the platform.
There are about 120,000 students learning through the platform as of now.
And we are collecting events
that compose learning on their performance in the platform
as well as how they are scoring in their assessments.
How many data points that you collect here and you put in S3?
We are collecting about 100 million data points
that are active for a learning experience.
Amazing. And?
And we are also collecting information about how authors are publishing this content,
Alef, through an authoring event.
And this contains about 65,000 data points that are inserted into the Redshift cluster.
Okay, so you have learning events from students
and then authoring events from Redshift.
Exactly. Okay.
Then we are also using a Spark Streaming job
to transform this unstructured data that we have in S3 into Redshift.
And in Redshift is a scalable data warehouse
that allows us to merge both unstructured and structured data.
And we have about three terabytes of data
collected from all the students who are learning in the platform.
So all the data will be in scalable data warehouse-- Exactly.
And it will be ready for the next processing step. Exactly.
What would you do next?
Because we have consolidated all the data that is done by the students,
teachers, and authors,
we are able to leverage the data that is there in the Redshift
to build a personalized recommendation system for students to learn.
Okay, and that would lead you to the next part of the architecture.
Exactly.
We are collecting this data for training this into a machine-learning model
that is passed through an EKS cluster.
This EKS cluster is also running on top of Kubeflow
which is an open-source implementation for training machine-learning models.
So you train the model with an EKS,
but how do you make sure that this is the right model
and the best model that you need to develop?
That's an interesting question.
So we are using identifying different parameters that we can optimize
for training the models,
and once we are fine-tuning it,
then we are passing this through a SageMaker hyper parameter tuning job
where we are using different objective metrics to optimize the model
and select the best model out of it.
Can you give me some examples of these metrics?
We are using a one-versus-all model
for classifying this into objectivizing the parameters.
We are using F1 score as our primary validation metric,
and then we are also using recall as a secondary objective
so that we are selecting the best model with avoiding the false-positive read.
Perfect. So all the hyper parameter tuning is happening here.
Exactly. And then once you are happy with the model,
what do you do next?
So SageMaker allows us to provide the best model through parameter selection,
and then the best model is bundled into a docker pull
and published to an ECR repository.
So the ECR will be the repository for the docker
as a container registry managed by AWS.
As a container--yes. Okay, awesome.
So once we bundle this model,
then we allow the students to use this model,
and we publish to an inferencing cluster in EKS.
So as of now, we have all the data coming all the way
through SageMaker into ECR. ECR.
So this is the training part. Exactly.
How do you leverage this recommendation engine from the student's perspective?
Because they have selected the best model that is saved ECR,
it gives us an opportunity to publish this model into an EKS cluster
where students would be exposed to this machine-learning model
for personalized recommendation.
And how many events do you have per day
maybe during the e-learning process?
We are also using about 65,000 events per day that are collected for recommendation.
And this is actually autoscaled
based on how the students are using this in their classroom.
And when the cluster is not used, as in the students are going offline,
then this cluster is autoscaled down.
So there will be some peak times and some--
Some down. Yeah. Okay, amazing.
And the autoscaling will scale up and down accordingly.
Exactly.
This is, again, passed through the inference cluster.
Okay, so now we close the cycle of training and inferences,
but I can see CloudWatch here.
What is the role that it plays with this machine-learning process?
CloudWatch provides us an important role in monitoring
how we are doing at training an inference.
So we are using logs from the training cluster
to collect metrics about how we should be training and optimizing the model.
For instance, if our model is diverting from what we are collecting in the form of data,
it gives us an option to use the early stopping
to stop and retrain the model based on other objective functions.
And this is for training part.
And what does it do with the inferences part?
We are also collecting the inference data
for all the machine-learning models exposed
and using this to track how the students are performing
when the model is published in the production.
And we are using the model objective to monitor the accuracy.
If the accuracy falls behind, we are using model leniency techniques,
and there are CloudWatch alerts
that are notifying about the state of the model.
Okay, awesome.
That's great to see how you apply machine-learning for recommendation engine
to help the students' learning experience.
Thank you, Ehsan. Amazing.
Thanks for your time, Sarath,
and thanks, everyone, for joining this episode of 'This is My Architecture'.