(audience clapping)
- Thank you, Matt, for the introduction.
Oh, it does work, okay.
Thank you all for joining our session today,
and welcome to CHI.
Being the first presenter, I think I should say this.
So my name is Maria Tomprou.
I'm postdoctoral research fellow
at Carnegie Mellon University,
and today I'm going to present you our study
on Career Mentoring in Online Communities
and how people seek and receive advice.
And this is a joint work with my co-authors,
Laura Dabbish, Robert Kraut, and Fannie Liu.
So mentoring has been documented
as a valuable career resource, and it typically refers
to the support people get professionally and personally
for their growth throughout their career pathway.
There is enough research suggesting
that there are both benefits and downsides.
On the side of the benefits,
people experience career growth, report career growth,
career satisfaction and learning.
However, when you don't have a good match with your mentor,
then it's more likely to experience sabotage,
feelings of sabotage, depression,
and psychological withdrawal (sighing).
Nonetheless, if you don't have a mentor,
it may have its own consequences.
So research suggests that the people without mentors,
without any kind of mentoring source,
experience isolation, professional stagnation,
and job dissatisfaction,
so you do need some kind of mentoring
throughout your careers, and especially in academia,
we all know that.
So in HCI literature and research has focused
in career research in two main research venues.
The one is job search and how people use technology
to find jobs, especially,
for underserved population and youth.
We also have enough research on how people use technology
and e-mentoring for educational purposes.
What we do not know is
how people use career mentoring relevant to their careers,
and we will be focusing on the online communities
as a source of mentoring.
And the main research question we're asking is
how people seek and receive advice.
So this is a traditional view
that we have about mentorship.
We typically see the mentor as someone looking
like Obi-Wan in Star War movie.
Have you ever seen that movie?
Okay, thank you.
And you always have this Obi-Wan guiding and inspiring Luke.
However, reality is that we do not always have
an Obi-Wan guiding us and inspiring us,
so let's think of this Luke in 2019.
His supervisor's trying to get credit for the work
that he has been doing.
He cannot go and ask for help.
His supervisor, obviously, conflict of interest,
or his peer, his colleagues,
because this may threaten his job.
So what can he do?
He goes online, and interestingly enough, we do not know
what kind of career advice requests people make
even in closed doors,
because we don't have access to this data.
So by going online and posting your question online,
it'd give us a unique approach related to study,
what kind of career advice requests do people make
in an online community?
And that leads us to the first research question.
And then, you have a number of strangers
with similar professional expertise
that they actually give you some suggestions
and solutions and some advice of
how you can deal with a problem.
And that leads us to the next research question
about the dynamics of mentoring online.
So do different kinds
of requests elicit different types of responses?
And not only this, oftentimes,
we have third parties that they visit the community,
the online community, with probably similar problems,
and they try to figure out their own solution.
So by going online and seeing what other people have,
question and answer, you may benefit and learn from that.
And that takes us to the third and final research question.
What kind of responses are most valued?
Both from our original requester
but also from third-party observers.
And this is a research overview of our study.
We had Study 1, where we actually tried
to develop a taxonomy of career advice requests,
and then, in Study 2, we wanted to understand
to what extent these career advice requests elicit
different types of responses
and which of these responses are actually valued
both from the requester but also from other people.
And let's start with Study 1.
Our setting is Workplace on Stack Exchange,
and it's a Q&A forum that anybody can go post a question
and everybody can answer.
The best answers are being upvoted and raised to the top.
The person who is asking the question, the requester,
can only accept one answer,
and this is typically represented with a green check.
So we had, our final data set,
we collected our data set from Workplace.
Our final data set was 470 posts
with the tags "Career Development" or "Careers".
Every few months, Stack Exchange releases
their data dumps on archives,
so people can go, and research can go use it
for their research purposes.
So we used this pool of posts,
and we had one coder,
that went thorough the 470 posts and analyzed them
from bottom to top, resulting in three main types
of career advice requests that I will be presenting you.
And then, in the next phase,
we did taxonomy validation using two independent coders
that they took each type of the posts
and tried to match each post,
with the Cohen's kappa,
to rate the reliability, ranging between .75 to .90.
And this is a taxonomy of career advice requests.
Three types of career advice requests:
Best Practices, Threats to One's Careers,
and Time Sensitive Decision Making.
So let's go through each one of them
and see some examples.
Best practices, basically, seeking solutions or approaches
to career-relevant problems
that are superior than other alternatives
or they are standard solutions in the organizations.
And here is an example from Workplace.
We have this person who just finished his internship,
and he wants to update his resume.
So he's going online, on Workplace,
and he's asking, "I want to add this information,
"how can I do that?"
We had four best practice themes.
Resume Improvement, how can I improve my resume?
Like the example I showed you.
Managing Interviews, Use of Technology and Social Media
for leveraging my career,
and the Utility of Academic credentials and certificates.
The second type of career advice request is threats
to career progress,
seeking advice for problems that may hamper your career.
An example of this is this user who used to love his job,
and at some point, they transferred him to another project,
and he started hating it.
So he's considering leaving the organization.
And he's going online and,
actually, he's posting his problem, seeking for advice.
Six themes relevant to threats to career progress.
Managing Relationships,
how can I manage my relationship within the organization?
How can I manage failures?
How can I resign?
How can I prove myself within my organization?
How can I balance health problems
without affecting my career progress?
And how can I keep myself motivated?
And the last type
of the career advice request was seeking advice
for solutions or input on a timely fashion,
on a time-sensitive fashion.
So this is a very typical example.
You have this user who just accepted an offer,
and then, the employer comes with a counteroffer.
So he's going online and asks some advice
on how to deal with the problem.
Four types here of time-sensitive decision making themes:
how people can manage a negotiation, Job Offers,
how can they can avoid tasks or projects,
and ethical issues.
So, overall, we found three types of career advice requests,
so the question here is,
do these type of requests elicit
different types of responses,
and what of these responses are actually most valued?
And that leads us to Study 2.
So, again, these are the two types of requests.
I'm not gonna go through them.
And here is the method overview of what did in Study 2.
We collected 36 posts for that data set,
with one accepted and one nonaccepted answer,
that leading us to 72 pairs.
And we coded responses using Amazon Mechanical Turk,
so they rated both on the content
and the value of the response.
For the content, we developed 13 items based
on mentoring literature
as well as observations of our corpus,
and with the value, we measured with two items
on a six Likert-type scale,
asking them, "To what extent do you think
"that was a helpful answer?"
and "How much effort does the requester
"actually put to give the answer?"
And this is an example of
how the Mturk response coding look like.
We first gave the question and then the answer,
and then we gave the 13 items plus the two
of the value measure,
and we asked them to rate each response.
We ran a Principal Component Analysis
to identify the main types of responses,
and we ended up with a 4-Factor solution
that explained 68% of the variance.
Two items were dropped, so our final scale was 11,
our final measure was 11 items.
So let's go through the factors of career advice requests.
The first one was General information provision.
The information can be applied to multiple situations
or multiple individuals.
The second one was Encouragement, the extent to which
the career advice giver provides some effort
to help the requester feel better but also look forward.
The third one was Role-modeling,
to what extent the career advice giver
actually offers personal experiences
and to what extent these were positive.
And the fourth one was Guidance,
problem-solving strategies and steps
that they are tailored to the original request.
So the question here is,
do these three request types
elicit different types of responses?
And we used Linear regression for nested data,
to answer our question.
And we did not observe any,
contrary to expectations, actually,
we did not observe any effects for that in our data.
So and then, for our next and final research question is,
what kind of responses are most valued
both from the person who is asking the question
but also from third parties?
So for the Requester acceptance,
you will use Logistic regression
because you have a binary outcome, accepted, nonaccepted.
And for the Third party ratings, we used Linear regression.
So our findings show that Guidance is this factor
that actually can predict the likelihood
of an answer to be accepted,
of a response to be accepted from the original requester.
In terms of with the third parties' evaluations,
we found that not only Guidance actually considered useful
to the valuable response
but also General Information and Encouragement.
And interestingly enough,
when people actually share personal experiences,
this was less appreciated,
so it was negatively related with the value of the response.
So in summary, responses were not request dependent,
posts with Guidance are four times more likely
to be accepted, and third parties value General Information,
Encouragement, and Guidance.
Role-modeling is less appreciated.
So in terms of for our Design Implications,
how can we design better career-related online communities?
So in the paper, we have a few of our Design Implications
that you can go and read further,
but I'm gonna focus on one.
Our main one was promoting career advice response
that users will actually value.
So, for example, having an Intelligent response critic
that help the person who is actually giving the advice
to provide responses that are Guidance-related
so they can increase the likelihood of acceptance
from the original requester.
Also, encouraging language
and general information can benefit third parties,
and this is something that we can also actually implement
to improve career-related online communities.
I don't know if I'm within my time limit, but I am done.
Thank you so much, thank you. (audience clapping)
- Wonderful, thank you, Maria.
And we'd like to invite any questions,
so if you do have a question,
please raise your hand clearly.
And there's a few at the front here.
- You look like my coauthor, Fannie,
so I was like, "Fannie's gonna ask a question?"
But you're not, okay. - No.
But I do have a question really quickly,
obvious one, in and out.
What do you think about this result
of role-modeling actually having a negative influence?
Is there something in the literature suggesting
that that would happen?
Because anecdotally from personal experience,
I think we often find that valuable.
- So actually, in mentoring literature,
role-modeling is considered one
of the functions of mentoring.
However, I don't think,
because it is very contextual to the mentor,
people do not, most likely,
when they see this kind of information,
they just waive it.
So this is why we have no significant effect
with the original requester,
or when third parties read these,
more likely they are thinking, "This is irrelevant,
"this is not something that we would actually appreciate,"
so instead, they would prefer to see actual steps.
That's my feeling.
Yeah.
Is there any other question that I can answer?
- [Announcer] We have time for one other question,
otherwise, I can ask one.
So I'd really like to ask,
in your paper, you talked about the difference
between online support and off-line support,
and
also automated support could be a future direction,
what are the kind of challenges
in providing automated mentoring and other places
where that would be appropriate and inappropriate?
- Okay, that's a very interesting question.
So I think one of the challenges is
how we can integrate the relational component of mentoring,
and I had a little bit of that in my slides,
but because I'm out of time.
So most of the time,
like in the case of health online communities,
you see extended relationships,
wherein career mentoring, people just go,
they're ad hoc interactions.
They go, they post their question,
they get the response, they leave,
so, one, I think,
of the challenges is
how we can have extended relationships online
that they can actually help the people
throughout their careers,
especially, for people who do not have access to mentors,
and believe me, there are many out there.
So I think that can be a way
that automation can actually help people,
as mechanics of the extending part of the relationship.
- [Announcer] Wonderful, please join me in thanking Maria.
- Thank you so much,
thank you. (audience clapping)