My name is Rebecca Krosnick and I'm excited to present our work "Think-Aloud Computing:
Supporting Rich and Low-Effort Knowledge Capture".
This is work we did while I was an intern at Autodesk Research.
When people do work on the computer, they have many different thoughts going through their head,
for example software know-how, design intents, and future work plans.
People often share their work with collaborators, or they might put it aside and come back to it
later. The problem is that when people review a digital artifact, the original creator's thoughts
are nowhere to be found. And there's little trace of the creator's process and design rationale,
making it harder for people to understand the work. To combat this,
people write documentation, but this it tedious to do, and won't capture everything.
We thought, what if ask people to speak while they work? So we propose,
think-aloud computing, a new application of the think-aloud protocol. In particular
we propose a system to encourage people to think-aloud while they work on the computer,
and automated approaches to prompt people to speak, to capture and process speech in real-time,
and to present captured information for colleagues or your future self to leverage.
We started with a formative study to observe people behaviors while they think-aloud while
they work, and to learn their opinions. We had 12 different participants across
3 different domains and asked them to create any content of their choosing, and we had them speak
concurrently and retrospectively with regard to their work. Participants said that think-aloud
requires some effort, but they thought that the information they captured would be useful.
And we saw a number of different categories of information that people captured,
you can see more of them in the paper. But we boiled it down to 5 main categories.
And we use this to develop our think-aloud computing prototype.
The user keeps the small widget in the corner of their screen while working. And
the widget captures their speech, records their screen and their software commands.
And it subtly prompts them to speak more by illustrating their speech levels overall and
for particular categories like design intent and process. And it categorizes their speech
using keyword matching, and these are some of the phrases. And when it thinks the user
might have something interesting to say, it pulsates to prompt them to speak more.
I'll now show a participant who is doing a modeling task using think-aloud computing.
"One thing I like to do while sketching, a super helpful tool is the extend tool. Extend and trim.
So you can just, you don't necessarily need to line up your pieces, you
just kind of extend those lines to get the shape, the desired shape that you want."
Once the information is captured, it's showed in an archive view for consumers,
with this text transcript on the left as well as categorizations of each utterance,
for example here is a to-do item. And the screen recording is included, as well as the software
commands that the user executed. And the original creator can also go back and add commentary.
We performed an evaluation comparing users using think-aloud computing versus writing
documentation, and we found that users captured unique kinds of information in the
think-aloud condition, for example we got some insight into their problem solving approaches,
we got to see some of their design alternatives, and we also got to learn about their design
constraints which are really important for people who want to use the artifact in the future.
A modeling participant commented on the usefulness of think-aloud computing.
They said with no documentation, it's really difficult to just look at a design
and be like 'oh, this is why they did it'. What you usually do is scroll
through the history of the design, and you step-by-step see how they built it.
But if that could just be spoken to you, like with think-aloud computing, that's super useful.
Regarding effort, a modeling participant said I'm just speaking out loud and I'm already doing that
when I'm modeling in my head, so I don't think it adds additional effort. A coding participant said
I guess because coding is kind of close to just typing a comment it wasn't as hard to switch. So
this points to the fact that think-aloud computing is probably most helpful for spatial/visual tasks,
tasks where switching to writing would be more difficult.
And so we think that think-aloud computing is a promising approach for capturing
people's information and thoughts as they're working, and for providing it
for others to reference in the future. Thank you, happy to take any questions.