Hi everyone, my name is Mirko Gelsomini and today I'm going to introduce you to "IMAGINE
- embodied learning in immersive smart spaces".
This work has been conducted at Politecnico di Milano by me, Mirko Gelsomini, Giulia Leonardi
and Franca Garzotto.
This work investigates the pedagogical potential of IMAGINE, an immersive interactive environment
for educational purposes where children are exposed to auditory and visual stimuli, activated
by performing mid-air gestures and movements in a digitally enhanced physical space.
The system is remotely controlled by the teacher, who can customize each experience according
to group's or individual's needs, modifying intensity, duration and orchestration of stimuli
and action-stimulus sequences.
Embodied cognition theories emphasize the relationship between physical activity and
cognitive processes and are supported by a growing body of evidence from psychology and
neuro-biology.
According to the hypothesis that embodiment can enhance the learning process, we have
designed and developed an educational didactic experience within IMAGINE aimed at facilitating
learning of "factual knowledge" on a curricular subject.
The experience is modeled following Kolb's experiential learning process according to
the cyclic sequence of "experience-reflection-learning" and is divided into the different phases of
Challenge & Initial Thoughts, Resources and Assessment.
In the very first phase, children are welcomed in IMAGINE and the teacher presents the topic
that will be discussed during the lesson making use of images projected on the wall and on
the floor.
She then collects students' questions and elicits children's discussion from previous
knowledge.
During the Resources phase, a 5-minutes video is projected to the wall: when the projection
ends, children are called in turns, one at a time, to "perform a task", which consists
of answering a proposed question using one of the given interaction types.
Lastly, children are evaluated on their acquired knowledge.
The structure of each single task was designed on the basis of the textbooks currently used
in primary schools.
From the analysis, 5 types of frequently used exercises have emerged, cross-cutting different
subjects, which we have reinterpreted with a view to motion-based interaction.
The first, the "selection" pattern presents the user with an explanatory image of the
question on the front projection and a variable number of response option cards on the floor
projection.
A star projected on the floor, placed immediately in front of the user's feet, follows the student's
movements.
In order to select the chosen answer, the child must place the star, and consequently
herself, over the card corresponding to the option deemed correct.
The child confirms her decision standing over the card for at least 3 seconds
The second pattern, the "classification", presents the user with a set of images and
a series of labelled boxes, both projected on the front projection.
A virtual superimposed hand appears, following the preferred user's hand.
The user is asked to grab, drag each of the images presented at the top of the screen
and release each of them in the correct category, shown in the form of a box at the bottom of
the frontal screen.
The third case, the "reordering" pattern presents the user with a series of images randomly
ordered on the front screen.
A virtual superimposed hand appears, following the preferred user's hand.
The user is asked to reorder the option cards according to a sequence contextualized by
the question.
To reorder the cards, the child is required to grab, drag and release each element until
the desired sequence is obtained.
A pop-up will ask the user to confirm or cancel the response given after 5 seconds from her
last interaction.
The "identification" pattern requires users to recognize a given object based on its characteristics.
Two different types of identification are supported.
The fourth one is based on "Color and Shape".
A variable number of images are randomly distributed on both projections.
The user has to identify the item requested in the question and select it by positioning
her body or by moving and closing her hand.
The fifth one is based on "Position".
A variable number of images are arranged neatly on the front screen.
The user has to identify the item requested by the question by moving and closing her
hand on its top.
The sixth one, the "association" pattern presents users with four frames at the top and four
items at the bottom of the front projection.
Each frame contains two slots, one housing an image and the other empty.
The user is asked to associate (grab, drag and release) each bottom item with the images
presented in the frames, placing the objects in the free frame slots.
Once all objects are placed in the corresponding slots, a pop-up asks to confirm or retry.
In order to investigate whether IMAGINE can actually facilitate children's learning, a
controlled experimental study was conducted.
As you can see from the image, the study is based on the comparison between the learning
process that takes place in a traditional classroom and the one occurring in IMAGINE,
with the aim of comparing impact on children's learning achievements.
70 children from 6 to 8 years old were recruited.
They were pair-wise assigned according to performance on a pretest to an experimental
group (on the left), subjected to the practice within IMAGINE environment and to a control
group (on the right), subjected to the traditional practice based on the frontal book-based approach
within the classroom.
Both groups were divided in classes, accordingly.
This study investigates a main research question: Can learning benefit from an immersive embodied
approach compared to a traditional paper-based classroom approach?
A Baseline Test T0 was administered at the beginning of the first week, in order to establish
homogeneous control and experimental groups.
Two learning sessions of 1 hour each were held in two separate days of the first week
(same amount of time dedicated to similar topics at school).
A Short-Term Test T1 and a Long-Term Test T2 were provided on the last weekday of the
first week and after seven weeks respectively.
Both IMAGINE and classroom activities used the same exercises, proposed in the same sequence
and with the same graphical contents.
The immediate effect of the intervention after two training sessions, which we refer as Short
term Retention, was evaluated based on changes in the values of the score obtained during
T1 with respect to that achieved with the baseline test in T0.
Short term Retention was significant for both groups as they scored an average of 18 notions.
This allows us to state that IMAGINE educational activities are a comparably effective teaching
tool with respect to classroom approaches.
We categorized each of the two groups into tertiles based on Short term Retention scores.
The first tertile can be traced back to the children who have retained fewer concepts
and, therefore, probably those who have encountered more difficulties.
The third tertile is associated to those children who acquired more concepts, which we can assume
have had fewer difficulties.
Consequently, the second tertile represents the group of children whose Short term Retention
is in the middle range.
Children belonging to the first tertile retained an average of 8 concepts in class and an average
of 11 notions in IMAGINE.
Results are reversed in the third tertile, in which children in the control group learned
an average of 27 concepts, compared to 25 acquired by the children in the experimental
group.
In the second tertile, children learned an average of 17 concepts, regardless the learning
context.
Speaking about Long-Term Retention, the values on Memory Loss, that is the number of forgotten
notions between T1 and T2, show that students from first tertile forgot in average 7.7 and
6.6 notions in the control and experimental group respectively.
Almost the same difference occurred in second tertile students, who lost 8 notions in the
control group against 7 notions in the experimental group.
The highest Memory Loss can be noticed in the third tertile, in which students who learnt
in classroom forgot between 8 and 24 notions and those who practised in IMAGINE forgot
between none and 11 notions.
In conclusion, each child memorized an average of 18 concepts in the short term and retained
7 notions when learning in the classroom and more than 11 notions when learning in IMAGINE
in the long term.
Children with more learning difficulties found it easier to memorize within IMAGINE.
The better outcome could be ascribed to the effect of the wide range of different interaction
modes and the various senses involved, which allowed them to more easily find the learning
modality suitable for their natural inclinations.
In addition, the evident greater ease with which the teacher has carried out teaching
sessions within IMAGINE, allows us to deduce that it acted as a catalyst of attention,
thus improving retention.
Finally, we present here the conclusion and future works.
The results of our empirical study are encouraging, as we detected improvements in participants'
long term retention skills that could be ascribed to the experiences in IMAGINE.
From a broader perspective, our research suggests that motion-based immersive technology to
support embodied learning activities in regular school contexts is a promising approach, and
smart spaces characterized by low-medium embodiment like IMAGINE might lead to revisit existing
educational processes as well as to reinvent new ones.
In the future, we plan to leverage the potential of motion sensing technologies by designing
full-body activities that make use of iconic and representational gestures so to take full
advantage of the affordances of IMAGINE.
We will expand the study to a larger number of participants who will be involved for a
much longer time, tracking their performances within each learning session.
Such research would also support the creation of a corpus of data on children's bodily learning,
unique in this field and valuable for educational practice and research.
Eventually, this environment still holds promise in supporting children with special educational
needs in mainstream education, given its plethora of interaction modalities and stimulus that
can be carefully selected by the teacher to give special educational needs children the
best tailored experience.
Concluding, we thank Polisocial Award, the social responsibility program of Politecnico
di Milano and the municipality of Cornaredo for having funded the project.
We thank also LUDOMI team, the teachers and students at Istituto Comprensivo Statale Via
4 Novembre, Cornaredo (Milano) and Istituto Comprensivo Statale Leonardo da Vinci, Cornaredo
(Milano).
Thanks all.