Hello, my name is Kimberly Ying, and I'm a PhD student
at the University of Florida.
Today, I'll be presenting my CSCW paper titled,
Understanding Women's Remote Collaborative Programming
Experiences, The Relationship between Dialogue Features
and Reported Perceptions.
There are two main pieces of background info
that motivate this work.
The first is that remote collaboration
is becoming increasingly common, and even more
so now with people working from home
due to the current pandemic.
And the second is that women have been historically
marginalized in computer science in many contexts
throughout the world.
One specific example is shown here.
In the US and Canada, just 21% of bachelor's degrees in CS
were earned by women in 2018.
So in this context of remote collaboration in computer
science, how are people's experiences and perceptions
influenced by structured roles different by gender
and related to dialogue features?
That's what we explore in our study.
58 participants worked remotely in pairs
on a programming problem for an hour.
They used a programming application
that allowed them to collaboratively code
in real time, similar to how Google Docs allows
users to collaboratively write a document.
They communicated using textual chat, which was embedded
in the programming application.
The structured roles condition had 16 pairs,
and the unstructured roles condition had 13 pairs.
In the structured roles condition,
participants followed a collaboration paradigm
known as pair programming, with the collaborators
having the role of driver or navigator
and switching every 15 minutes.
Those in the unstructured roles condition
had no predefined roles to follow,
and there was no announcement every 15 minutes.
We measured learning gains through pre and post-tests
and collected post-survey responses
using the validated intrinsic motivation inventory
and the system usability scale.
We found no significant differences between conditions,
but there were significant gender differences
regardless of condition.
From these results, we can see that women had a very different
experience from men.
They felt less competent, although there
were no significant differences in their pre-test scores
or their reported prior experience.
They were more stressed, and they felt less choice.
For the remainder of the talk, we refer to these survey items
as competence, attention, relaxation, and choice.
To gain a deeper understanding of women's experiences,
we investigated their dialogue features.
After performing principal component analysis,
we finalized six features for modeling.
The first three refer to the women sent messages,
and the last three refer to the messages
she received from her partner.
We used ordinal logistic regression
to model woman's experiences with respect to their dialogue
features for each of the four significant survey responses,
and three of these four models were significant.
And these five dialogue features had significant correlations
with women's experiences.
Specifically, number of messages sent
was positively correlated with relaxation and competence.
The average words per message was positively correlated
with competence and choice.
The number of messages received from the partner
was negatively correlated with competence and choice.
The average words per message from the partner
was positively correlated with relaxation.
And the partner's sentiment was positively
correlated with woman's relaxation, competence,
and choice.
Women reported significantly higher levels of stress,
lower levels of perceived competence,
and less perceived choice compared to men.
We found that dialogue features can provide a window
into women's experiences, potentially
allowing for early detection of stress and other outcomes.
It's important to remember that our work here
shows correlations between dialogue features and women's
experiences, but it does not imply causation.
Additionally, our analysis uses summative dialogue features,
such as total message frequency and average sentiment value
for the entire conversation.
Future work is needed to understand whether there
are causal relationships between these dialogue features
and women's experiences.
We also need to understand how these dialogue features may
change over time during a collaboration.
This work leaves us with questions like,
can we improve women's experiences
by encouraging behaviors associated
with positive outcomes and discouraging
behaviors associated with negative outcomes?
Does fostering the expression of positive sentiment
toward the task have a positive impact on women's experiences
during collaborative programming?
That concludes my talk.
I would like to acknowledge NSF for funding this research
and thank the members of the LearnDialogue Group
for all their help.
Thank you for listening.