Hello everyone and welcome to the presentation of Exploring How Game Genre
in Student-Designed Games
Influences Computational Thinking Development.
My name is Giovanni Troiano, and I will be presenting this work on behalf of my
colleagues Qinyu Chen, Angela Vargas Alba, Gregorio Robles, Gillian Smith,
Michael Cassidy, Eli Tucker-Raymond, Gillian Puttick, and Casper Harteveld.
We want to thank the CHIorganizers for giving us the Best Paper Award
and we also want to thank them for still giving us the opportunity to disseminate
our work.
This work was a collaborative effort between Northeastern University,
Universidad Rey Juan Carlos, and the TERC Institute, and it's part of the
Building Systems from Scratch project, which is funded by NSF.
Let me start by providing context on the Building Systems from Scratch project.
In this project, we implemented a STEM curriculum in various middle schools in
the Boston area
where we leveraged on game-based and constructionist learning, to have 8th
graders learn about climate science and systems thinking.
The students were asked to program and design games in Scratch and ultimately
developed in computational thinking
which we assessed using the Dr. Scratch metrics here.
Let me explain to you how the Dr. Scratch metrics work.
These metrics are based on observable coding practices in Scratch, namely how
the Scratch blocks are used to develop and design a program.
These practices are then translated into computational thinking dimensions
which in the case of Dr. Scratch
are seven, and they are: abstraction, parallelization, logic, synchronization
flow control, user interactivity, and data representation.
Dr. Scratch will score each of these dimensions on a scale from 0 to 3,
depending on the computational thinking complexity which reflects the complexity
of the underlying programming practices.
For instance
you will get one point in logic for using an "if" block, or you will get two points
for using an "if/else" block.
When you sum up
the seven dimensions, you can get a maximum of 21 points in computational
thinking.
This work has three main motivations and objectives:
First
we wanted to understand the impact of game genre on computational thinking
development
that is important because, as game based curricula emerge in constructionist
education,
we want to have a good understanding of how design choices of students impact
the way they learn about computational thinking and programming
Second
we wanted to unfold the underlying programming routines by looking at block
usage in Scratch.
In fact, the Dr. Scratch metrics right now only account for the first instance
of block usage.
However,
how many, for instance, logical blocks are used in storytelling games opposed
to action games? And does that make a difference in the final score for
computational thinking?
Last but not least
we wanted to understand the educational implications
of looking at game genre and impact of design choices
For instance
is there like a particular game genre
that is "favorable" to computational thinking development?
In this study, we collected 404 games designed by 8th grade students as part
of the Innovative game based STEM curriculum.
The students
designed their games in Scratch to personally reinterpret climate change
topics of their choice
For instance
the game at the top
it's a reinterpretation of the ice-albedo feedback loop phenomenon,
which becomes here a Pong game
The one at the bottom
it's called Government Simulator.
it's a game that asks players to act like
politicians and take decisions that can impact either negatively or positively
the environment.
We could assess using Dr. Scratch 391 games for their computational thinking
proficiency
namely their score from zero to 21 in all the seven CT dimensions
or the score from zero to three on each individual computational thinking
dimension.
Then we could analyze 325 games
for their computational thinking development based on a decile analysis
(ndr., decile works like quartile, but has ten time intervals instead of four)
that means that we kept track of the evolution of the score for each project
from their beginning to the end.
the context in which we carried out the assessment was of course the game genre
To identify the game genre
we used both the tragic game design (TGD) model
and the categorization of game genre in HCI proposed by Heinz and Law
Furthermore
we analyzed the underlying programming routines
of the student design games by looking at block usage in Scratch
After multiple rounds of coding
we could finally identify 10 game genre in our student design games.
You will find more detailed descriptions about each game genre in our paper,
but I'm going to provide a few here to give you an overview and a sense of the
game design displayed by students in our data set.
Pong games
were the most popular in our data set with 96 Pong games designed by students.
Albedo Pong is a perfect example of such a pong games.
The climate change elements here are well integrated in the game mechanics
where an ice paddle will shrink, every time a sun ray (a tiny yellow ball) hits
the ocean surface after bouncing back from the sky.
This game will represent the ice-albedo feedback loop phenomenon
and it shows how breaking it can eventually lead to global warming
Pat Earth instead is a pet simulator
where the player is asked to feed and give a shower to planet Earth.
However, by the end of the game the player will soon learn that the
indiscriminate consumption of food and water can actually be harmful to the
planet
Carbon Clicker is a simple clicker game, which shows players how the misuse of
technology
can lead to
a massive production and release in the air of CO2
and thus be harmful to our planet.
Finally the shooter game Ice Cap Hero! transforms Alien Invaders into CO2
Invaders
shooter games are also the games with the highest computational thinking score in
our data set.
and the ones that used most blocks in Scratch.
Now, let's see if game genre actually did have an impact on the final score in
computational thinking proficiency as assessed by Dr. Scratch in our student
design games.
The lowest score was produced by quiz games with a mean score of 13.4. Puzzle
and Qhris game showed the larger variations, and Clicker and Maze game
showed the smaller variations We performed a one-way Anova on the data
set to find that game genre does have a significant impact on the final score in
computational thinking proficiency. A post-hoc tests showed that this
significance can be attributed to To quiz game
This means
and the results suggest that designing Quiz games may not be advisable or may
not be good from a computational thinking learning perspective.
Now, let's see if game genre also has an impact on the computational thinking
proficiency of each individual dimension
Shooter games scored the highest in abstraction with a mean score of 2.18.
I just want to remind you
that the maximum score of Dr. Scratch
in each computational thinking dimension is 3.
Storytelling games scored the maximum in parallelism (or parallelization)
Maze games scored almost the maximum in synchronization
with a mean score of 2.97
We performed a multivariate Anova to find a significant impact of game genre
on the computational thinking proficiency of each individual dimension
a post-hoc tests revealed that Quiz and Storytelling
make this significance in the data set appear
as they scored the lowest in logic and parallelism consistently
with the score in each of these two dimensions being always lower than 2
Now, let's see how game genre differently impacts the computational thinking
development in student-designed games.
As we can observe
most of the computational thinking increase in all the dimensions happens
quite early in the projects specifically within the first three deciles.
In our data set, that means four days of work out of two weeks of curriculum,
so we found this to be a time critical period for computational thinking
development means that within the first three or four days of work
computational thinking profiles will consolidate and then leave space
for more specific or refined game design practices later on.
Now, if we observe the individual dimensions
of computational thinking and how their score evolves over time we can also see
some interesting patterns.
For instance, in Swipe Elimination, logic at the beginning
is not very strong,
But then it boosts quite a bit towards the end of the project to improve
towards proficiency
In Puzzle games, parallelism is not very strong
at the beginning
to become them quite proficient towards the end of the project.
In Platform, the same thing happens
with synchronization not very strong at the beginning
quite proficient at the end.
In Storytelling instead, we see that logic is not strong at the beginning but
it doesn't really improve towards the end very much.
So
in this kind of game, we do not see the same pattern that we see for instance in
Puzzle or in Platform.
Finally
we can see that Simulation
is the game genre that has the least increase in all the computational
thinking dimensions
within the first three deciles
but then it grows quite a bit towards the end of the project and then it becomes
quite a bit proficient in all the competition thinking dimensions.
Now, let's take a look at the results for block usage.
While the use of block seems roughly distributed across game genre,
we can still notice that, for instance, in Quiz and Storytelling game
there isn't much frequent use of control and logic blocks (green).
This means that
for students designing such a games
may prevent them from becoming proficient in logic and may prevent them from
learning how to use logical and conditional statements in programming
practices.
On the other hand, Shooter and Simulation games make frequent use of data blocks
(brown)
meaning that for students
designing these games
may have a higher chance
compared to other games genre to lead them to proficiency in data
representation.
While we could not find any significant impact of block usage on computational
thinking
we could still find a small
but yet significant correlation between block usage and computational thinking
this suggests that future work should further explore and scrutinize the use
of blocks in Scratch
and better understand how they impact computational thinking
as well as carefully consider how a broader understanding of block usage can
inform the redesign of the Dr. Scratch metrics and make them more context and
design sensitive.
To recap, we have used the doctor scratch metrics to assess 391 student-designed
games
For their computational thinking proficiency.
We also use them to monitor the computational thinking development of
325 games.
We identified 10 game genre using the TGD model in combination with the HCI
categorization of game genre proposed by Heinz and Law.
The results show that game genre impacts computational thinking proficiency and
development
and that block usage correlates to computational thinking but needs further
scrutiny in future work.
Finally, we leave you with some points for discussion and reflection.
We've seen how game genre impacts the computational thinking score progression
already in the early phases of game design.
For instance, action games led to the highest computational thinking
development between decile D0 and decile D1
meaning that Action games boosted computational thinking already within
the first day of work.
Essentially, after decile D3
or day for of work is "game design time!"
and this meant that in our curriculum students had 10 days
to focus on their game design, which is plenty of time.
But it also meant that their computational thinking profiles
consolidated pretty early and only marginally changed thereafter. So these
results suggest that educators in game-based constructionist curricula
carefully monitor the computational thinking development of their students
right when they start their game design projects
and that ideally they expose their students early
to different game genres, or ask their students to program different games with
different genre
so that they can learn computational thinking from different game design
perspectives.
Also, we've seen how the familiarity of popular games influenced what students
designed.
In fact, Clicker and Pong games are 47 percent of the data set.
This results are consistent with previous work
who showed how popular games influence game design and young students
and this may even have implications in gender and inclusivity in game-based
curricula.
For instance, Storytelling games are the ones that scored the lowest.
However, the literature also showed that female students prefer storytelling
games the most
As designing Quiz and Storytelling games may have prevented students from
developing in computational thinking dimensions such as logic
our results partially challenge the notion of freeform learning in
constructionist curricula.
And so, we suggest that maybe a mix-and-match of instructionism and
constructionism
may be beneficial to computational thinking learning.
Again, my name is Giovanni Troiano
and I want to thank you all for having watched this presentation
also on behalf of my colleagues.
We hope you enjoyed it, and we hope to see you soon at the next CHI.
Also,
if you want more information about the project Building Systems from Scratch
you can visit buildingsystems.terc.edu
Thank you.