I’m Keita Tokuda from Tsukuba University.Today I would like to talk about our attempt to
predict student performance in abacus-based calculation using matrix factorization techniques.
This work was conducted as a collaboration with David Kaschub, Takuma Ota, Yasunobu Hashimoto
at Digika, an EdTech company in Tokyo, and Naoya Fujiwara at Tohoku university,
and Akihito Sudo at Shizuoka university.
This is the outline of today’s talk.
Firstly I would like to introduce the iPad application for the abacus-based calculation.
That is how we acquired the students’ learning history data of calculations. I would also
like to give a brief introduction on the matrix factorization technique for user modeling,
and why we chose this approach in analyzing the educational data we had.
Then I'll talk about the model and the experimental results, and conclude the talk.
The abacus is a calculating tool.
It has been used for thousands of years, at least in East Asia.
This is how the Japanese abacus, or Soroban in particular, looks like.
Soroban is a quite familiar tool among Japanese people.
It is also included in the curriculum of the compulsory elementary schools.
Soroban is made of multiple vertical columns with freely-moving beads attached on them.
It is designed to represent integers with the positions of the beads in an intuitive way.
This is an example of the representation of the number, 38, by soroban.
The beads are counted when they are shifted towards the horizontal beam at the center.
Three lower deck beads shifted towards the horizontal beam represent the number 3.
The upper deck bead represents the number 5.
Altogether, represent the number 38.
It is widely known that an advanced learner of Soroban is able to conduct mental calculation
at remarkable speed and accuracy.
The calculations shown here are the examples describing the level the student at Digika
aim to attain,
Mentally calculating these operations within 10 seconds at age of 5-8.
We believe that learning Soroban is a mean to acquire a remarkable ability in mental
arrhythmic calculation.
What we like to achieve additionally, is to incorporate the data-driven approach in the
education of abacus-based calculation, so that we can predict each student‘s performance
more accurately, and characterize each student’s strength and weakness more precisely, and
realize automated adaptive curriculum, based on data.
However, it may not be an easy task.
The progression of the learning status of students may be a complex process.
Development of an appropriate method for estimation of each student’s learning state should
be an important issue.
To the authors’ knowledge, data-driven approach has not been adapted in education of the abacus yet.
One of the reason that has hindered data-driven approach in education is the lack of large
data.
We have previously reported an educational system developed by Digika, called SoroTouch,
which enables learning abacus-based calculation on iPad.
As you can see, a student can conduct abacus-like calculation on SoroTouch by changing the positions
of the beads on the touch panel screen.
The data of detailed operations of a student on this application is recorded in the cloud
server in real time, whenever and wherever the student uses it.
As a result, this system has accumulated students’ learning history data of hundreds of thousands
of times of calculations, so far.
We could use this data for the student modeling.
As the algorithm for the student modeling, we utilized the matrix factorization technique,
which has been very successful in user modeling, especially in development of recommendation
systems for entertainments.
As many of you may know, matrix factorization is a mathematical procedure to decompose a
given matrix into two matrices.
Let’s assume the matrix on the left-hand side contain the data of the students’ performance.
Specifically, the (s, t)th element of the matrix has the value of the sth student’s
performance in the tth task.
Then, MF aims to approximate the matrix by a product of two matrices on the right-hand
side, one consisting of the latent distributed representations of each student, and one consisting
of the latent distributed representations of each task.
Once you acquire the latent vector for a given student and for a given task, respectively,
you can estimate the unknown performance of the student in that task by calculating the
inner product of these vectors.
As in the problem of characterizing human preferences in entertainments such as movies,
it is expected that student state in learning calculation should be represented by multiple
latent factors.
For example, a student might have his or her strength in addition, and weakness in subtraction.
Another student might have his or her strength in subtraction and weakness in addition, at
A task in calculation could be also represented by multiple factors such as the number of
addition, or number of subtraction, or number of carry-ups, and so on.
The advantage of MF is that it infers a distributed representation for each student and each task
without detailed prior knowledge about how to construct the latent features.
This is the specific MF model we utilized in this study
In this model, the estimated performance of the sth student in the tth task is expressed
by a summation of four terms.
Namely, the global average term, the student bias term which takes a specific scalar value
for each student, the task bias term which takes a specific scalar value for each task,
and the student-task interaction term denoted by a dot product of the sth students’ vector
and the tth task’s vector.
The illustration below shows how the same equation can be described with a summation
of four matrices.
The baseline methods used in this study to compare the model‘s performance in prediction
task are the two models.
The first one is the global average model, which is the model one takes only the first
term in the MF model.
The second one is what we call the BUI model as in the former studies, and it is the model
that omits the interaction term from the MF model.
This is the objective function that is used in the optimization process to infer the latent
representations of the students and the tasks.
It is expressed in the commonly used form.
Namely, it consists of the residual sum of squares, and regularization term with the
Frobenius norm of the yielded two matrices.
In this study, we used the value K=150 for the dimension of the latent vectors, and β=
ten to the minus third, for the regularization parameter.
For the quantification of the students’ performance, we utilized the percentage of
the correct answers.
Namely, the (s, t)th entry of the matrix was calculated by the ratio of the number of the
correct answers the sth student gave to the tth task, to the total number of the answers the sth student gave to the tth task, multiplied by a hundred.
In order to discard noisy data, we have conducted pre-processing,
and we discarded any data of a student who solved less than 10 different task,
and we also discarded data of any task that has been solved by less than 10 different students.
We also discarded any student-task pair with n_st^total less than 40.
As a result, we obtained the matrix with the dimension, 2442 by 626.
The number of the pairs where the data is given, is 115223, which is the 7.54% of the
all the elements of this matrix.
The randomly chosen 95% of these points were used as the training data, and
the remaining 5 % was used as the Test data.
This is the result we obtained with our analysis using MF.
The left scattergram shows the predicted values of the performances of students in the test
data, plotted against the true values.
As you can see, there is a correlation between the model prediction and the true value.
The right bar graph depicts the comparison of the model performance to the baseline models.
The root mean squared error values, or the RMSE values, calculated with the predictions
in the test data are compared between the MF model and the baseline models.
As you can see, the MF model outperforms the baseline models, suggesting that the MF model
successfully acquired latent factors that determine the interaction of students and
tasks in calculation.
This is the conclusion of our current study.
Thank you very much for your attention.