My name is Magnus Kaspersen, and I’m presenting the paper “The Machine Learning Machine:
A Tangible User Interface for Teaching Machine Learning.”
The paper was written by myself and my colleagues from Aarhus University; Karl-Emil Bilstrup
and Marianne Graves Petersen.
This research is part of the multidisciplinary CEED project at Aarhus University, and is
supported by a research grant from the Villum Foundation.
Our motivation for this work, comes from the widespread use of machine learning in society.
It is everywhere, and affects us in several ways; from recommendation algorithms on Facebook
and Google, to machine learning systems supporting decision-making in law enforcement and the
judicial system.
However, because machine learning is able to connect information in data-sets to large
for human comprehension, the inner-workings of such systems are too complex for most,
if not all of us to understand.
Because of this potential to affect our daily lives, it’s important for children and young
people growing up to understand machine learning and become able to take a critically informed
stance about its use.
Prior research on using Tangible User Interfaces in teaching situations suggests that they’re
effective in making otherwise intangible phenomena, such as machine learning, tangible and explorable
for students.
Thus, our goal with the machine learning machine, was to take the ML process, and make it tangible,
explorable and understandable to users
And now to the main attraction of this talk; the Machine Learning Machine itself.
Overall, the machine learning machine works as a typical image classifier, similar in
concept to Google’s Teachable Machine: Users provide a data-set of two types of images;
these could be pictures of the sun and the moon, and goes on to classify these as belonging
to one of the two categories.
The machine learning machine itself consists of two interactive devices; the Trainer for
training machine learning models, and the Evaluator for evaluating these models.
In addition, the machine learning model itself is represented by a physical artefact and
students use simple pen and paper to create data for the machine learning machine.
To make the machine learning process tangible, we have simplified the machine learning process
into four distinct steps, which are made tangible through interacting with the machine learning
machine.
First, we have Problem Formulation.
Here, users need to define a problem, that is solvable using machine learning.
In the case of the machine learning machine, this entails finding two classes of motifs,
that users would like the machine to be able to distinguish between.
Next, users go on to create their data-set.
Here, they use pen and paper to make drawings of the two motives.
For good results in distinguishing between classes, users need to produce between 5-10
drawings of each class.
Next, they go on to the training phase, where each image is shown the Trainer and classified
as belonging to one of the two classes.
This creates a machine learning image classifier based on the data.
Finally, students can evaluate the ability of their model to recognize new drawings,
by using the evaluator.
Here, data that has not been used for training the model, is used to evaluate the predictive
abilities of the model.
Based on the the results, the above steps are repeated to improve the model’s predictions.
Here, we see how this process looks from the perspective of the users.
Just, a small note: These pictures are from an intervention we did, with the machine learning
machine after the paper was accepted for TEI, and is not part of the paper.
In the first, top-left image, we see a group of children drawing suns and moon in order
to have the machine learning machine recognize between them.
On the next, top-right picture, we see a child insert these drawings into the machine learning
machine in order to train a machine learning model on them.
The bottom two pictures shows the students placing a new drawing in the evaluator and
pressing the “predict”-button on its front.
This shows the students two numbers, which shows the classification of the model.
Based on these predictions, students can reflect on the performance of their models.
It might be good at recognizing moons, as in this case, but might perform worse when
looking at the sun.
In that case, students can go back to the drawing board and create more drawings of
suns in an effort to increase the model’s performance.
A part from teaching the basic process behind creating machine learning models, the machine
learning machine can also be used to teach critical aspects of machine learning as discussed
further in the paper.
Here I will give a short demonstration of how, the machine learning machine can be used
to demonstrate unintentional biases.
To demonstrate this, we would ask users to create a model based on a data-set of suns
and moons as seen here; with all suns drawn in one color, and moons drawn in another color.
Next, we ask students to evaluate the models ability to recognise moons drawn in the same
color as suns.
Here, students will see that the model performs significantly worse; this is because the model
did not actually learn to differentiate between drawings of suns and moons, but only between
colors.
Based on this realization, a classroom wide discussion can be had about where this issue
might occur; one example could be the racial bias in machine learning systems used in the
judicial system in the United States, which has been show to discriminate against black
people.
To evaluate the Machine Learning Machine, we conducted a small, preliminary pilot study,
with two male teenagers aged 15 and 19.
Even though, the study was only preliminary and no generalizations can be made from it,
some interesting findings did occur.
First, we found that the slow approach to interacting with machine learning, and especially
in evaluating the models, was beneficial in terms of spurring reflections during use.
The participants reflected on the difference between classifications when using the Evaluator,
and used this to figure out how to improve their models.
This ties in to our second finding, that the iterative nature of the machine learning machine
process allowed the users to transition between ready-to-hand and present-at-hand interaction,
which meant that they were in kind of a continuous circle of reflection and doing.
Finally, we found that while the participants were able to reflect on machine learning while
using the machine learning machine, it was difficult for them, to connect their experiences
with machine learning applications in everyday life.
We expect that this was due to the generic tasks, we asked them to do, and we plan to
further evaluate the machine learning machine with more situated tasks.
All in all, I hope, that you found our work interesting and inspiring.
There is more to be found in the paper, so I do encourage all of you to read through
it, and please don’t hesitate to contact me, if you have any questions regarding the
work at any point in time.
Thank you for listening!