My name is Chris Bopp, and along with my co-author,
I would like to present the results
of our Systematic Review of Stakeholder Voice in HCI
Research with Nonprofit Organizations.
The prevalence of HCI research carried out with non-profits
has increased dramatically over the past 35 years.
Since the first mention of a nonprofit and HCI publication
in 1983.
Despite this history and our accumulating understanding,
there have not yet been any systematic reviews
of this body of research.
Before making more holistic claims
about what this body of knowledge
might suggest for a research agenda moving forward,
we carried out a systematic literature review
that included 145 publications to understand the varying
degrees of inclusion or exclusion
of stakeholder voices.
And for research in HCI emphasizes the importance
of ensuring that a breadth of stakeholder voices
are included in system design, especially
in situations with significant power dynamics and conflicting
values.
Other scholarship and community based research
emphasizes that the methods we use foreground certain voices
over others, making it essential that the methods used
across a broad body of research, recognize
the voices of a comprehensive set of stakeholders.
Our first research question is to what
degree have the voices of stakeholders
in the social sector been included or excluded
in the body of research being developed
to understand the role of technology
in nonprofit organizations?
We explored this in a number of ways
as described in the slide, such as groups that were included,
methods, and geographic regions.
As indicated in this graph over time,
researchers have engaged with a greater diversity
of stakeholders in a nonprofit sector.
However, the preliminary focus on employees
has persisted over time.
While researchers engagements with clients and the public
has increased over the past decade,
the frequency has still not approached that of engagement
with employees.
Qualitative methods are the most common mode of increase
for researchers and as corpus.
Overall, interviews are the most frequently cited method.
Secondly, when researchers engage with the public,
they have adopted a more diverse set of methods
than is used with employees.
While interviews are still frequently conducted,
researchers have also used methods
like observations and surveys.
Clients are the stakeholder group that
is least likely to be engaged.
When they are engaged, it is most often
via observational methods.
Research was most commonly conducted in the US and the UK.
After accounting for differences in nonprofit sectors
across the country, we found that the field
has most significantly under engaged
with countries in the European style welfare partnership
cluster, such as Germany, Austria, and France.
The type of organizations with which researchers
most frequently engaged are health and social service
organizations.
While other organizations, such as professional associations,
have been engaged with less frequently.
Additional research that bolsters our understanding
of the perspectives of stakeholders who are currently
underrepresented would help to ensure
that there is more equity in the voices that
are informing system design.
Engaging these stakeholders through more diverse methods
would also enable valuable new ways of knowing.
And interacting with organizations
in more geographic areas and policy fields
allows us to not only expand our knowledge,
but to conduct comparative analysis across contexts.
Our second research question focuses
on the more generalizable lessons,
but the ways that stakeholder voices are rendered more
or less analytically included.
This question unpacks the systemic influences
on decisions around the inclusion of stakeholder
groups, methods, geographies, and organizational types.
Inspired by Volker's application of the concept
of charismatic megafauna, we outline a framework
for engaging in such disciplinary reflexivity
by examining three different analytic charismas.
Rather than stemming from any one individual researcher's
motivations or research questions,
systemic biases across a body of research
result from more systemic cultural forces
or charismas that compels certain types of research
over others.
And compel the inclusion of some stakeholders over others.
Methodological charisma encourages researchers
within discipline to adopt or prefer
certain methods over others, and could
be influenced by the epistemology of researchers
who are drawn to working in a given area.
Or because partner organizations prefer particular methods
in their role as gatekeeper.
Institutional charisma encourages researchers
in a discipline to prefer to work with certain institutions
or configurations of institutions over others.
While domain charisma encourages researchers to
prefer working in certain domains or problem
spaces like health care or crowdfunding over others.
Through our interrogation of the fields engagement
with these nonprofit stakeholders,
this systematic review finds significant opportunities
for increased inclusion of voices.
Ameliorating the gaps where stakeholder voices have
been underrepresented will be essential for preventing
emergent systemic biases in our understanding
of the role of technology in the nonprofit sector.