Hello. I'm Taehyun Kim from 'This is My Architecture'.
Today, I invited Yungon Park from I-ScreamEdu to share a story about the I-ScreamEdu Architecture.
- Hi. - Hello.
First, please introduce I-ScreamEdu.
I-ScreamEdu runs a service called Ice Cream AI Home Run.
Students use our learning machine which is a tablet PC to learn.
Our team is running a service called AI Student Record that collects data from our learning machine
and provides appropriate advice to students based on the data.
Yes, I am curious about the AI Student Record Service Architecture that you just mentioned.
Please explain in details.
Yes, any students' actions in our learning machine are sent to API Gateway in JSON format.
This transmitted data is processed in the Firehose stream.
When the data is collected at a constant price, the Lambda function is called,
and data conversion is performed based on the data in DynamoDB and Aurora DB.
The converted data is stored in the S3 bucket.
As you said, Firehose processes data.
At that time, DynamoDB and Aurora seem to be utilized.
Please explain what data is stored and how it is used.
The app information installed in the learning machine is frequently viewed, and it needs a fast response time.
To meet these requirements, I chose DynamoDB, which can be searched by Key Value.
As students' learning content or content related to exams are often searched
through the index, the data is stored and queried in Aurora.
Yes, so how will the processed data be processed later?
When a new object is created in S3, the Lambda function is called to proceed with the data classification.
When you open the newly created object, there is the attribute that lets you know when each data was created.
The data is classified based on its attributes and the classified results are uploaded back to the S3 bucket.
The classified data is saved based on data types with year, month, and day.
It's processed in Glue based on that data.
AWS Glue provides various functions such as crawler ETL scheduler, etc.
How is I-ScreamEdu using Glue?
We first crawl the classified data through Glue.
ETL operations are performed based on the updated data catalog.
For ETL work, we use the PySpark script provided by Glue by modifying it for our purpose.
It operates according to the certain schedule.
The result of the ETL work is saved in Redshift.
Yes, it seems to be stored in Aurora, final data or Redshift.
Also, it seems that you are using the Step Function for processing data in the middle.
Please explain in more detail.
Yes, Redshift is used by us as a data warehouse.
It is also used in the AI Student Record for evaluation data analysis and content usage analysis.
In order to provide a smooth service to the AI ​​Student Record, the DB in charge of the service is divided into Aurora,
and the parts for analysis are performed by Redshift.
Step Function is used for data analysis work.
Dozens of Lambda functions are intertwined, and these Lambda functions are arranged in order,
and the data is read from Redshift and the result is saved to Aurora.
Data analysis results saved by Aurora can be viewed by students in the learning machines
or in their parent apps or our website.
Thanks for your explanation. Finally, please tell us about the future plans of I-ScreamEdu.
As students spend more time studying at home and more content is added inside and outside our company,
the amount of data that we have collected has increased.
In order to respond to this increased data, we are improving our infrastructure.
In addition to Home Run, we are building data pipelines and analytics systems
for other services within our company or other organizations.
I look forward to seeing I-ScreamEdu in the future.
Thank you.
This has been 'This is My Architecture'. Thank you.