Hello I'm Tim Adamson. In this presentation I will describe our efforts
designing social interactions with a humorous robot photographer. Comparing
the two photos in this slide it is clear that though they portray the same person
one is significantly better than the other.
What can robot photographers do to capture quality portrait pictures? We
study this question for a robot that operates in a university context
One idea is to advance technical aspects of robot photography and enable
these machines with a notion of image quality. This has been the focus of most
prior work in robot photography. For example prior work has explored
questions such as how can robots take well composed pictures of people
considering things such as the no middle rule the rule of thirds and the golden
rule? Our research however explores the design space of the social interactions
that a robot photographer has with individual photography subjects. We
recognize the significant opportunity that this design space provides to
complement the technical focus of previous work. Designing the social
interactions of a robot photographer also comes with its own set of
challenges. For one you have the people involved coming from a diverse set of
personalities backgrounds and emotional states.
Some people may love having their picture taken and being in front of a
camera while others may be shyer. People also come into a robot photography event
with different goals in mind some may be intrigued by the robot and want to spend
as much time with it as they can while others may focus on getting a high
quality photo or on taking a decent picture as quickly as possible. We used an
iterative and human-centered design process to explore this design space
This process included informal interviews with professional
photographers building an initial prototype to engage
users and social interactions and an informative study to learn more about
photography from people and from our robot. We then improved our robot prototype
and the way it engaged users and conducted user tests. I will now briefly
describe each of these steps and end with a summary of key design
recommendations based on our design process
At the beginning of our research we conducted informal interviews with three
professional photographers. We interviewed them about their work and focused the
conversation on the social aspects of portrait photography. During the
interviews the photographer's mentioned two social aspects especially salient to
a photography event the first was what the photographer says to the subjects.
This includes learning their name getting to know a little bit about them
and saying encouraging things about the subject. The second salient aspect was
how the photographer strives to make the subjects feel comfortable and elicit
genuine smiles. This idea came up when one of the photographers shared that
one of the most important parts of his job is making his clients feel
comfortable and at ease during the photography event. We also asked the
photographers what the best way to learn more about the social aspects of
photography would be going forward and they all suggested shadowing a
professional photographer in action. Before shadowing human photographers we
decided to first build an initial prototype of the robot which we named
Shutter. Now when we add human photographers come into our lab to take
pictures they could also evaluate the prototype that we had made so far and
provide us with feedback and suggestions. We made the prototype focusing on social
interactions. All the hardware components of the robot were chosen with this goal
in mind. We added a real sense camera to our robot to identify faces and allow it
to take the pictures. The platform had a screen face which displayed eyes. The
eyes would blink and help give the robot greater animacy. The face was mounted
on a four degree of freedom arm which was used to enable moving the camera and
keeping the photography subject's face in the image. After building the robot we
brought in professional and amateur photographers to have them take pictures
of each other. We shadowed the human photographers and asked questions along
the way. We then had the robot take pictures of the photographers. At the
end of the study we had a group discussion to gather additional feedback
We came away from the informative study with three main takeaways. Though Shutter
provided no direction we saw that much of the human photography experience was
driven by how the photographers directed the photography subjects in
where to stand and how to pose. We also saw humor come up again with both
the professional and amateur photographers who used it to make the subjects feel
comfortable and enhance the interaction as a whole. Shutter provided no feedback on
the photos and the professional photographer provided minimal feedback
which led the photography subjects to request that more feedback be provided
in the future
We took these three points of feedback and decided to implement all of them on
our robot. To allow our robot to direct the subjects we added five numbers to
the floor as can be seen in the sketch on the right
We had the robot direct the subjects in where to stand by telling them which
number to stand on top of. We also added humor to the robot. The humor was
presented through one of four media formats which included Memes, GIFs
spoken phrases
[Shutter] "two four three number number number"
and sounds.
[kid laughing]
In total we enabled the robot to deliver 13 different types of humor each belonging
to one of these four mediums. The humor was delivered right before the robot
would take a picture in order to elicit smiles. To add feedback after taking a
picture we had shutter to look over to the
monitor which had the image taken and then say something positive about the
image. After making these modifications we connected user tests in which we had
25 participants come into the lab to have their picture taken by Shutter
Shutter began the interaction with an introduction asking a user if he or
she would like to be photographed. It then proceeded to take five pictures
with humorous content being delivered in all the pictures but the first. In each
picture event was humor Shutter provided additional content during the
countdown right after the numbers 5 3 & 1. This content was either one of the
four humor types described before or a "no humor" option. Following the countdown
Shutter took the picture complemented the photo taken and then directed the
participant on where to stand next. From our design process and the results from
the user tests we learned important lessons that can benefit future robot
photographers. I'll go briefly over a few of them now
please see our paper for more details. The first lesson that we learned is that
when adding humor it is important to include a wide
of content. When looking at users self-reported smile response to the
content delivered during the interaction we found that humor content was
generally more successful at eliciting spontaneous miles than when no humor was
provided. As you can see from the graph humor content was two to three
times more successful at eliciting spontaneous smiles. Worth noting no one
type of media was drastically better than the others. Because
participants preferences were diverse it is important to include a variety of
humor content in these interactions when attempting to elicit spontaneous smiles
We also discovered that when adding humor content to a human-robot
interaction the time required to deliver the content should be seen as a cost
borne by those who don't find the content to be funny. Some participants
smiled or laughed at all the humorous content as can be seen in this short
clip. While others never smiled even once
In order to automate spontaneous smile detection for robot photography it is
essential to consider human robot interaction dynamics. As you can see here
people respond in the shutter with both posed and spontaneous smiles. Sometimes
these smiles were easy to distinguish and other times they were not.
Traditionally spontaneous smile detection algorithms are trained with
videos that show people transitioning between neutral expressions and
spontaneous smiles but when it comes to detecting spontaneous smiles at
photography events it is important to consider posed smiles as well. Our work
also reinforces a large body of research in HRI which has identified the
importance of robots acknowledging mistakes. Our robot always complimented
photos but when a picture did not turn out well like the one seen here the
compliment was seen as disingenuous and forced. Thus it is important for robot
photographers to detect and acknowledge mistakes
First we studied social interactions between individuals and our robot but
moving forward we will consider groups of users who may want to be photographed
simultaneously. Second we conducted our user tests in a laboratory setting. We
are now starting to study interactions in the wild allowing for a
more diverse set of users and more complex environments. Third the quality
of the pictures taken by a robot was sometimes not as good as it could have
been. Though our robot did use a centering composition when taking
pictures we plan to use more complex composition rules in the future to
improve the quality of the photos taken. To summarize our work showed the importance
of studying social aspects of robot photography. In this context humor is a
key element which robots can employ to improve the experience. We hope others
will be inspired by our work to continue research in this row