In our paper the placebo effect of artificial intelligence and human computer interaction.
We contribute a study that investigates whether AI technologies can serve as a placebo.
This paper is joint work between the HU Berlin, Aalto University, Technical University of Chemnitz, and LMU Munich.
My name is Thomas Kosch and it is my pleasure to have conducted this work with Robin Welsch, who is one of the main contributors of this paper, Lewis Chuang, and Albrecht Schmidt.
A goal in human computer interaction is to build technologies for users.
We often follow a human center design process that makes design decisions based on data obtained by users.
We know that users have high expectations with regards to AI that might facilitate our lives, improve how we interact with technological systems and enhance our performance.
However, we know from psychology and medical research that high expectations can produce placebo effects that undermine evaluation efforts.
Nonfunctional sham treatments such as a sugar pill can improve people's conditions.
In the context of the prison study, we investigate how the narrative of an AI assistant acts as a placebo that changes the results of a user study.
Considering placebo effects.
We ask ourselves do placebo effects, also occur when evaluating Human-AI Interaction.
In our systematic literature review, we found that the placebo effect has not been extensively covered in Human-AI Interaction. Afterward, we decided to investigate it.
In two studies we approached this with simple experiments in an online study, where we presented users with a non-functional AI system that was supposed to improve the performance in a task using a control condition.
We investigated how participants evaluated and interacted with the system with and without AI assistance
The participants task was to solve word puzzles in two conditions.
Participants were told that an AI was evaluating the camera feed and optimizing the word puzzles difficulty to increase performance.
In the other condition, participants were told that they would not receive any support in the task by an AI.
However, participants did not know that none of the word puzzles were adapted in any condition and word puzzles were always presented at random.
We made three measurements in the experiment.
First we measure the expectations for performance before the interaction.
Then we measured the error rates while solving the puzzles and finally the expectations for performance with the system.
After the interaction, our results show that the manipulation worked.
Participants had high expectation with regard to their performance with the eye system Furthermore, to set no direct effect on the performance in solving the world puzzles.
After about 40 word puzzles, they still assumed that the performance had improved.
This shows the placebo effect with regard to subjective improvement.
Looking more closely at the data we find that people with high expectations performs significantly better in the task.
We found an indirect placebo effect when considering expectations before interaction.
We have therefore derived several recommendations that help in diminishing or controlling the placebo effect ranging from polling expectations of users to adding placebo controlled conditions with sham systems. 00:04:05,000 We conclude that placebo effects replicating human ai interaction that they can undermine evaluations, and that H ci needs to start considering placebo controls to establish a reliable and trustworthy research process.