Hello everyone I'm Maggie Liao from the Media Effects Lab of Penn State I'm really glad
to present the work on behalf of my co-authors Dr. Shyam Sundar and Dr. Joe Walther today.
Recommender system has never been more prevalent in our life than before.
Therefore how to explain the result, and enhance users understanding of, and trust in the system
become extremely crucial as these recommender systems tend to be more user-facing and popular.
Traditionally when we think about trust we usually equate trust with the actual performance of the
system, particularly how to improve the algorithm and develop different advanced computational
models to predict users' needs and preferences more accurately. However recent studies
start to show that ensuring good UX can be far more complicated as users' evaluations are fairly
subjective. So despite the difference in terms of the objective performances, we still do not know
whether users have a general subjective tendency to trust one type of recommender system more than
the others. In this study we primarily investigate three of the most common approaches used by
recommender systems, so users usually become aware of these approaches when the recommender
systems provide explanations. The first one is the demographic-based recommender system which relies
solely upon users' demographic profiles, like age, gender, education, and many more. And users will
usually see explanations like "other users in your age group also enjoy this......". And the second
one is the collaborative filtering which analyzes correlations among similar users based on users'
potential common interests or tastes in products. It can express in terms of their past purchases,
ratings, browsing history, etc. And users will usually see explanations like "other users with
similar interests as you also enjoy this.....". And finally is the content-based filtering
which focuses primarily on one's own indicated preferences and analyzes the correlation between
user's preferred product characteristics with those of the new products. And users will usually
see the explanations like "based on your past selections you might be interested in this......".
Simple as they are these explanations can have significant psychological implications among
users due to the triggering of different cognitive heuristics or mental shortcuts.
So compared to the other two approaches, collaborative filtering might lead to
higher bandwagon perception or trigger the bandwagon heuristics,
namely "if others think something is good then I should think so too" because it recommends based
on a large group of users who share similar tastes and preferences with individual users.
And content-based recommender system, on the other hand, might be able to give individuals a greater
sense of personalization by tailoring products to feed one's specific needs, therefore, it is more
likely to trigger the identity heuristics, namely "if something reflects my unique characteristics,
it is better". And these heuristics could further lead to more positive evaluative outcomes.
However, recommender systems did not always go as we planned, especially in the initial stage
of users' interaction with them, when the system knows relatively little about users,
and the most likely to make mistakes, what we call the "cold start" problem. So how will users react?
We could draw some insights from the responsibility attribution literature,
so in interpersonal communication, according to the Attribution theory, human tends to hold
a self-serving bias, namely, we tend to think our own failures as been due to like aspects
of the situation including other people, while we tend to credit ourselves for the success we have.
And we would like to investigate in our study if the same bias still holds true with
humans' interaction with the recommended system as well. And more importantly,
how users assign responsibility while interacting with different types of recommender systems,
and how the responsibility further influences their trust and UX.
So to answer all these questions, we conducted a two by three between-subject-only experiments
among 226 participants from Cloud Research, and participants were randomly assigned to
one of the six experimental conditions and were instructed to interact with the
corresponding prototype of one of the ostensibly different recommended systems, called Movie Taste
the recommended movies to users, and these are the major dependent variables we investigated.
We mimiced the real life recommender system, so after users landed on a home page,
they also need to build a profile and then they will receive the recommendations. To manipulate
the system's performance, we actually conduct a pilot test first to identify the best or the worst
movie trailers from a large pool of movie trailers and select these movies for our recommendations.
The manipulation of RS type was embedded in the explanation that the system provided to users
on the welcome page, the loading page, and the result page respectively, so for content-based,
they're going to see explanations like "the movies are recommended based on your unique taste", and
for collaborative, "those are recommended based on viewing actions of the like-minded people",
and then for demographic "it's recommended based on some demographic factors". So what we find
is that as we hypothesized, users tend to trust collaborative filtering more than content-based
filtering due to the triggering of bandwagon heuristics. And also as we hypothesized,
content-based filtering is trusted more compared to the demographic filtering due
to higher identity perceptions and this result holds true regardless of how the system performs.
The same result also applies to other dependent variables as we described in our paper. We also
find users tend to over-attribute responsibility to themselves when receiving good recommendations,
while over-attributing responsibility to the system when they receive bad recommendations.
In other words, the same self-serving bias still holds true between users' interactions with the
recommender system as well. More interestingly, we find this bias is less salient when they
interact with the demographic recommender system because they would assign more responsibility to
the system regardless of how the system performs. And responsibility attribution is also found to
be a significant mediator such that users will trust the collaborative filtering more than the
content-based and demographics ones especially when the system performs well due to the lower
attributed responsibility users assigned to the system. In general, our study corroborates
previous findings that users rely quite heavily on social endorsement cues when making decisions
online. However not all social information is trusted, the superficial ones like demographics,
age and gender, is not sufficient for building users' trust. Instead users might need social
information that actually reflects other's collective tastes and preferences to a specific
product. And we also find users higher trust towards collaborative filtering is
independent of the system performances. So in the past, researchers usually propose that you know,
we could leverage the demographic filtering to solve the cold start problems. However, what we
find in our study is that although collaborative filtering might face a bigger cold start problem
empirically or computationally, it can promote user subjective trust in the system, even when the
performance is not good. For content-based system, the key to gain users trust with the system is the
identity heuristics, so content-based system can leverage the positive effects of customization and
individuation by including design elements that allow users to consciously engage in identity
related preference settings, like "are your sports fan", and accompany system's recommendations
with explicit appeal to their identity in the explanations like "as a sports fan, you will
love this". Finally, responsibility attribution is found to be a significant mediator which tells
us that recommender system can be more upfront and transparent with users about factors that
might influence performance quality, especially in their initial interaction with the system.
But more importantly, we can also build in cues on the interface that visualize user's contribution
to the outcome of the system at every stage of the interaction, and emphasize the collaboration
between users and the system to increase users perceived credibility of the system.
So we invite you to read further into our paper, if you have further questions please
feel free to email us, and thank you very much for listening to our presentation!