Hi everyone. My name is Sebastian, and this is joint work with my colleagues from the University
of Basel, the Hamm-Lippstadt University of Applied Sciences and the University of Washington.
CHI often provides results and ideas that influence technology innovations
designed and developed in technology hubs in Western countries, by people who are arguably
Western, Educated, Industrialized, Rich, and Democratic [pause] or short, WEIRD.
Now, many studies have shown that the use and perception of technology
varies across countries and cultures. This means that we, the HCI community,
should not only study people that are WEIRD, but also people from around the world to develop
technology that is globally useful and usable. The CHI community already tries to increase
international diversity by SIGCHI Executive Committee efforts, activities like RepliCHI,
HCI Across Borders, ICTD as well as researchers who studied diverse
groups in Western and Non-Western societies. Our goal is to support these efforts and to shed
light on the current international diversity of CHI by answering the following questions:
To what extent are participant samples in CHI papers recruited in Western, Educated,
Industrialized, Rich, or Democratic societies? and Which countries are over- and understudied?
To answer these questions, we conducted a systematic quantitative content analysis
of over 3000 articles of the last five proceedings of CHI.
Our analysis showed that participant samples of CHI publications
are not representative of the world s population.
More than half of all countries in the world are not covered by CHI research studies.
73% of findings are derived from studies with Western participant samples.
97% of these papers are fully WEIRD. This means that almost three quarter
of the knowledge we produce at CHI is based on roughly 12% of the world s population.
We identified positive trends, too. CHI participant samples
become steadily more geographically diverse. There is a reduction of participant samples
from the US by 13% and almost a doubling of non-Western samples to around 30%.
This could be explained by: different conference locations, general efforts to diversify CHI,
the usage of online and big data studies, and research projects in the area of ICTD.
Our second goal was to identify which countries are over- or understudied. It
turns out that many small European and WEIRD-countries like Finland, Denmark,
Sweden, Luxembourg, Switzerland, and Great Britain
are strongly over-represented at CHI, compared to their country s population size.
So, it seems that our average CHI study participant tends to be more western, educated,
industrialized, rich, and democratic than most of the world s population.
Based on our results we developed several ideas to make CHI less weird.
Keep in mind that all authors of this paper are WEIRD themselves. So,
this background has influenced our view of the following potential solutions.
We could see in our data that most authors recruit local participants. Balancing the number
of authors and co-authors from non-Weird and WEIRD countries would likely lead to
a more balanced composition of study samples. Our study found that online studies usually have
more diverse participant samples. They are limited to participants with access to the Internet.
But we would like to encourage the CHI community to conduct further research
on how to increase data quality and allow quantitative as well as qualitative methods.
The CHI community has a lot of experience in conducting
research with different methodological settings across countries and cultures.
We should start to share best and worst practice examples gained when
designing, recruiting for, and conducting these studies.
In the case of cross-cultural studies, replications can uncover
interesting variations that may help us to better understand whether the innovations we make are
equally suitable for people around the world. Such replications and extensions of studies should
continue and be promoted by the community. 56% percent of the analyzed papers did not
present detailed information on their participants country of origin.
By including this information, we would facilitate replications, extensions,
meta-analyses, and could track the international representativeness of participant samples.
Altogether, our takeaway message for today is that CHI is WEIRD,
but we re hopeful that this is changing! Thank you for listening and I m excited to
discuss our results and ideas for making CHI less WEIRD.