Hi everyone, thanks for coming to my recording! My name is Nigel Bosch and I'll be
speaking about our paper, "Students Verbalized Metacognition dduring Computerized Learning".
This paper describes a study using Betty's Brain, which is a learning-by-teaching environment in
which students learn about natural science topics by explaining them graphically to a virtual agent
called Betty. So, students lay out the various concepts in their natural science topics including
nouns, concepts, and the the causes that link these things together. So, for example,
the student might link together deforestation to vegetation by noting that deforestation reduces
vegetation. Vegetation absorbs carbon dioxide, in turn, so we can make a causal connection there
and perhaps conclude that deforestation reduces vegetation which absorbs carbon dioxide so maybe
deforestation would increase carbon dioxide. And that's exactly what the "Betty" virtual agent
does. Betty uses these causal links to to answer quiz questions that the student administers to the
virtual agent. So, the causal map forms Betty's Brain (what Betty uses to answer the questions)
and the students can then see what questions they got correct or incorrect in the quiz as
well as what links in the causal map led to that particular answer. The student can also see cases
where Betty could not answer a question because the topics were not linked together appropriately
in the causal map at all. So, the students can then use this information to update their causal
map and hopefully learn a bit about these natural science topics in the process. In this study we
had some middle school students who used Betty's Brain as part of their classroom activities,
and while they were using Betty's Brain we applied real-time affect and behavior detectors,
especially focused on confusion detection, to provide prompts to interviewers, who then
interviewed some of these students. During these interviews, students describe their software
usage, their problem-solving strategies, problems they encountered, and other topics. It was fairly
open-ended, but designed to help students resolve issues they were encountering and [for us] to
understand those issues. We then -- after the fact -- measured metacognition from these interview
transcripts. We did so by applying some natural language processing methods that recognize
first-person metacognitive statements, where the students are evaluating
their knowledge. For example, a student might say "I really didn't know what I was doing", where
they talk about what they know or what they did not know and recognize their own knowledge state.
You can read the paper if you're curious about the details of exactly how we did this.
For a little bit more complicated example, a real example from a student, where a student says
"I wouldn't say it surprised me. I already knew..." (that's a metacognitive statement
from the student) "it was bad, but, like it was a little bit like a daunting realization of how,
uh, bad it is." Then the interviewer might jump in and probe a little bit more deeply ask some
follow-up questions, and the student can respond (in this case) with more metacognitive statements.
This is an especially metacognitively-verbal student, and some of them really do not
verbalize whatever metacognition they may or may not be experiencing in the moment. So,
we then explored how students' verbalization of metacognition during this learning experience
with the Betty's Brain software related to various other aspects of the learning experience that we
expected would be related. Somewhat surprisingly, verbalized metacognition was not directly related
to learning. However, the interviews themselves were, so there's a little bit of a complicated
relationship there. We get into it a bit more in the paper if you're curious. However, we also
found that unresolved confusion tended to relate negatively to verbalized metacognition, which is
indicative of how the students were able or not able to resolve their confusing experiences.
We also found that verbalized metacognition was related to some metacognitively-informed software
use -- for example, updating the causal map based on quiz results they had just seen. So, thanks
very much for watching and/or listening! Feel free to reach out to me if you have any questions!