This study found that one typical dating website design encourages users to make biased decisions.
Additionally, we have found one possible way to reduce such bias.
Hello, my name is Zilin.
And I am going to talk about our paper, Not Just a Preference: Reducing Biased Decision-making
on Dating Websites.
Dating websites have become a vital space for people to connect.
More and more evidence shows that dating website designs are not bias-free.
Many features on dating websites have directly contributed to the racial segregation between
minorities and white.
For example, dating websites include filtering features based on people's race.
Scholars have argued that filtering based on race can exploit social stigma and exclude
minorities from access to love and romance.
In this project, we attempt to investigate other dating website design features that
are also biased are also bias prone.
Here are different versions of the same swipe interface.
It is almost the most popular dating website design.
The users will swipe left to dislike and swipe right to like.
A quick interaction that allows users to go through thousands of profiles a day.
A notable attribute of such a system is that it prioritizes profile pictures.
This design encourages quick decision-making based on superficial features.
People form first impressions based on the profiles' race and then give false assessments
on the profiles even when the evidence presented indicates otherwise.
We designed three interventions to reduce such bias.
First, because the current interface emphasizes a lot on the pictures, we might need to decrease
the importance of the profile pictures.
We reversed the sequence of the information presented to the users to do that.
We show the profile descriptions.
Then at the very end of the profile, we show the race-revealing information, such as profile
pictures and the names,
Second, if quick interaction induces bias, we can encourage users to think slowly.
Similar interventions in reducing one's intuitive decision-making have been used in human-AI
interaction settings to promote more analytical decision-making.
These types of interventions to foster analytical thinking is called cognitive forcing functions.
In our version of the cognitive forcing functions, we asked the users to make one decision, then
showed them a match score calculated by how much the profile matches with the users' ideal
traits of a partner.
Then we asked the users to make a decision again.
Finally, we combined the previous two interventions.
We put the race-revealing information at the end of each profile and ask the users to make
a new decision in light of the match score.
To formalize, we provide the following hypothesis:
H1: Users make racially biased decisions even when they do not want to.
H2: Showing race revealing information at the end of a dating profile reduces bias
H3: Asking the users to decide in light of new information reduces bias.
H4: Finally, asking the users to make new decisions based on new information and putting
race-revealing information at the end of the profile can reduce bias.
To test these hypotheses, we designed a dating simulation task that is similar to interacting
with most of the dating websites.
This experiment is a between-subject study.
Each participant interacted with one variation of the interface, and we conducted our study
on lab in the wild, a crowdsourcing platform.
First, when users came into our study, they had to mark the importance of some questions
about their ideal partners' traits.
We picked these questions from the OkCupid dating website.
We use the answers to these questions to construct fictitious user profiles that the users can
interact with.
After answering the questions, the participants interact with the fictitious profiles.
Each profile has a picture, a name, and descriptions.
On the profile page, the participants can swipe left to dislike and right to like
Or they can expand the profile by clicking on a button.
Each participant interacted with profiles of 3 races, Asian, black, and white; each
has 5 profiles.
Each race profile has a different match score of 20%, 40%, and up to 100%.
These percentages are calculated according to the number of questions the participants
considered important earlier.
The profile images were generated from a GAN network.
We handpicked the faces to ensure that the variables such as the direction of the face
and facial expressions were controlled.
These images were further evaluated by MTurkers, and vetted according to the attractiveness
generated by the Turkers.
We vary the presentation of the profiles in the following four ways (as we have discussed
before).
After interacting with the profiles, the participants answered a question assessing whether they
had a racial preference so that we can use this variable to determine if the participants
have explicit bias.
In our analysis, we conducted Mixed-effect Ordinal Regressions.
We controlled for participant gender, profile gender, the score of the profiles.
You can find detailed statistical analysis in the paper.
As a result, People who said that race was not a factor in their dating decision exhibited
anti-black bias.
Reversed sequence intervention reduced such bias.
Finally, Update/ Combined interfaces did not reduce anti-black bias
In summary, we have two hypotheses supported.
People showed bias when they didn't want to, and displaying the race-revealing information
at the very end will result in less biased decisions.
Then let’s consider if we can implement these interventions in real life.
Some argue that removing the saliency of images reduces the fun in dating websites.
Users are less likely to engage with the interface in that case.
Instead of intervening all the time, we can predict when people are biased, and then apply
the intervention accordingly.
Additionally, dating websites do not have to implement the exact interface design.
As long as the racial information is deemphasized, people will show reduced bias.
Some websites have already adopted similar designs.
For example, Taffy only reveals the user profile pictures after several exchanges of messages,
reducing race's saliency.
At the same time, this interaction is fun because the users are curious.
There are limitations to our study.
We ran our study on a dating simulation rather than an actual dating website.
Please let us know if we can contact anyone from the audience who wants to do a user study
on a real dating website.
To conclude the implications, I want to point out that dating websites constantly shape
our agencies in romantic relationships.
As designers, we should consider the consequences of even the most minor decisions.