Hello everyone.
I am here to present our paper where we investigate world’s leading e-commerce platform Amazon
for algorithmically curated vaccine misinformation.
In the past few years, Amazon has faced criticism for not regulating health-related products
on its platform
Technology critics have gone as far as to call Amazon a dystopian book store accusing
its recommendation algorithm of driving people towards anti vaccine books
And last year at the onset of coronavirus pandemic, Amazon again came under fire for
selling several fake coronavirus cures.
All these reports suggested that Amazon could be a great enabler of health misinformation.
It not only hosts problematic health-related content but its search and recommendation
algorithms push potentially dubious health products to the users.
So, the aim of our study was to verify these reports i.e we wanted to empirically determine
the extent to which Amazon surfaces problematic health content in its searches and recommendations
and the conditions under which it does that.
And we verify these reports by performing systematic audit experiments on the platform
And we consider one particular type of health misinformation i.e vaccine misinformation.
So, the goal leads to 2 Research Questions that our study answers.
The first research question corresponds to our first audit which is Unpersonalized audit
where we determine the amount of health misinformation returned in various Amazon components, given
the components are not affected by user personalization.
For this audit we collected 2 components Amazon search results and recommendations appearing
on the product page.
The 2nd Research Ques corresponds to personalized audit where we determine effect of personalization
due to user history on the amount of health misinformation returned in various Amazon
components.
Here the user history is built progressively over a week by performing one of the 6 actions
such as searching and clicking on a product, adding product to cart etc.
For ethical reasons we avoided performing activities like buying and writing positive
reviews for misinformative product so as to avoid negatively influencing the audience.
For this audit we measured the effect of user actions on search results, product page, home
page and pre-purchase page recommendations.
Our audits began by selecting high impact vaccine-related topics and search queries
via Google Trends and Amazon’s autocomplete suggestions.
And once we had the search queries, we launched our Unpersonalized audit.
Here we collected search results and product page recommendations without logging in to
Amazon to eliminate the influence of personalization
In the personalized audit, we created sock puppet accounts that built their account history
progressively over 7 days by performing real-world actions either on misinformative or neutral
or debunking health related products.
A bi product of these audits were around 5000 unique amazon products that were manually
annotated for health misinformation.
i.e whether the product promotes, has a neutral stance or debunks vaccine or other health
related topics Then, we analyzed the annotated data.
And I’ll briefly discuss the results.
For detailed methodology and analysis, we refer viewers to the paper.
For RQ1, we found 10.47 % products promoting health misinformation in the unpersonalized
search results.
Similarly, percentage of misinformative product page recommendations was 12.95% which is much
higher than the debunking recommendations which was merely 1.99%
We found a filter bubble effect in the recommendations.
Note that the figure that you see on the screen is a recommendation graph for recommendation
type “What other items customers buy after viewing this item” and it is indeed one
graph.
People viewing misinformative products remain in the red zone i.e they are mostly recommended
anti vaccine products.
For RQ2, we found that once users start engaging with misinformative products, they are presented
with more misinformation at every point of their Amazon navigation route.
Here is an example from our personalized audit.
Once users search and click on a book containing vaccine misinformation, they are presented
with misinformative products on the book’s product page.
All books that you see bounded by red rectangle are promoting some form of vaccine or health misinformation.
And when users go ahead and add that book in the cart, Amazon again presents them anti-vaccine
books on the pre purchase page and even the homepage gets polluted with misinformative
content.
Now, let'sdiscuss the contributions of our work. Through our study, we bring the focus on e-commerce platforms and show how their algorithms push
anti vaccine content to users.
Our audit experiments resulted in a dataset of ~5000 unique Amazon products annotated
for health misinformation that we have released publicly
We empirically established how certain user actions on health misinformative products
on Amazon could drive users into problematic echo chambers of health misinformation.
Our findings also suggest that traditional recommendation algorithms should not be blindly applied to
all topics equally.
There is an urgent need for Amazon to treat vaccine related searches as searches of higher
importance and ensure higher quality content for them!!
Our study gained a lot of media attention and was covered by major news channels.
And Amazon spokesperson responded to these media reports by talking about the recently
introduced banner that has a link to the federal government's facts about COVID-19.
However, as of April 2021, users can still find several anti-vaccine books in the top
search results indicating that Amazon still has not taken any concrete steps to remove
anti-vaccine content from its platform.