Hi, I'm Sam Reig from Carnegie Mellon University.
This talk is Social Robots in Service Contexts: Exploring the Rewards and Risks of Personalization
and Re-embodiment.
This is work by myself and my colleagues, Michal Luria, Elsa Forberger and Isabel Won,
Aaron Steinfeld, and Jodi Forlizzi, and John Zimmerman.
Now the leftmost photo in this slide shows a social robot giving museum information to
visitors.
Robots like this one are making their way into front-line positions in brick-and-mortar
services, where they work alongside human service workers.
While some of the robots out in the world today don't actually interact with customers,
like the Stop & Shop robot in the middle,
which roves around alerting store workers to spills,
the future of service robots is looking more and more like the photo on the far right,
where a robot receptionist with multimodal interaction capabilities and characteristics
of an identity directs users around the building.
With this increasing presence of robots in services, and their increasingly social design,
some questions are provoked about the novel behaviors that they might perform.
How should they provide service over repeated interactions with regular customers?
When should they act more like humans, and when should they act in nonhuman ways?
And in this work we focus on two specific questions about robot behavior.
So, first: do people want robots to identify customers?
And if so, when?
And the second question is about robot re-embodiment,
or agents that move their social presence
from embodiment to embodiment.
And for an example of this, think of the agent in a personal smart speaker or phone
appearing in, say, the grocery store robot
from the previous slide, and giving you some information
about where products are that you're missing from your fridge.
So the question there is: do people want agents to re-embody within a service,
and take on multiple service roles?
Research shows that service customers increasingly wish to interact with a single front-line
worker,
and agents that re-embody can make for a seamless service experience, providing comfort and
improving trustworthiness,
but re-embodiment also raises concerns about robots' competence with tasks requiring different
expertise.
Personalization by robots is something that necessarily entails the collection of personal
data,
and can often be seen, unsurprisingly, as creepy or unpleasant, but can also be seen
as practical, or
a combination.
The previous work on service robot re-embodiment targeted early stages of the design process,
using the method of user enactments to help determine which design concepts to advance,
and which to leave behind.
Our current work targets an intermediate stage of the design process,
where we have the beginnings of an understanding of the
contextual nuances of personalization and re-embodiment, but only the beginnings.
And so we dug into claims derived from this previous work.
Two of the claims regard robots identifying people in services.
First, people will be bothered when a robot identifies them in a service where they don't
expect to be identified, but when they do expect to be identified -- so, in situations where a
human service worker would maybe say hi to you by name -- they're okay with it for robots too.
The second claim is that people are less bothered by being identified
when a robot uses customer profile data to deliver something of value.
And the third claim has to do with re-embodiment and perceived expertise,
suggesting that people don't want robots to re-embody when to do so
would involve a large change in social role or expertise.
We used the technique of an online speed dating with storyboards study.
Speed dating with storyboards is something that's typically done in-person,
where a researcher sits down with participants,
shows them sketches of possible scenarios involving design concepts, and interviews them.
But by instead loading several storyboards into a Qualtrics survey, we allowed participants
to experience small sips of many different possible futures, depicted in medium fidelity.
And we also were able to get data from a relatively large number of people.
The storyboards were developed in pairs, such that one from each pair captured the assumed
preference according to the claim, and the other captured the opposite.
And we piloted many different stories and a few subtly different styles,
and eventually selected down to eleven pairs.
For the first claim — which is about being identified by a robot when there is
or isn't an expectation to be identified — we varied context across four pairs.
We compared a carnival to an auto shop, an office supplies store to a hair salon,
a department store to a gym, and a grocery store to a doctor’s office.
And this example, a carnival is contrasted with an auto shop.
So, in the storyboard on the left, Pat goes to a carnival, and a robot says, “Hey Pat! Welcome
back. It’s been a year since your last visit. Your favorite ride has a wait time of 15 minutes.”
In the storyboard on the right, Pat goes to get their car oil changed, and a robot says,
“Hey, Pat! It’s been 6 months since your last visit. We’ll get started
on your Subaru with your preferred oil type.”
For the second claim — which was about being identified when there is or isn't some value
delivered — we used the contexts of a movie theater, a fast food restaurant, and a superstore.
And in each setting, the robot commented on its personal knowledge about the customer, but in
the value version, it also delivered free tickets, or a coupon, or useful information about products.
So in this example, in the storyboard on the left, Macy goes to a movie theater and a robot
comments on Macy’s visit frequency. And in the storyboard on the right,
the same interaction takes place, but once that's done, the robot gives Macy free movie tickets.
For the third claim — about robots re-embodying into roles that are
more similar or more different from each other — we used the contexts of air travel,
personal care, a dentist’s office, and a hotel.
So in the left column here, an agent goes from an airport check-in robot to a seat-back
entertainment system, a hair dresser to a masseuse, a parking garage assistant
to a receptionist, and a maintenance worker to a room service robot.
And in the right column, an agent goes from a check-in robot to a co-pilot,
a hair dresser to a physical therapist, a parking garage assistant to a receptionist
and then to a dental nurse, and from a maintenance worker to a concierge.
So this example shows the airport scenario, where on the left,
Bob the agent checks Jerry in for his flight, then transfers his presence to Jerry’s seat-back
entertainment system while a different agent announces itself as the co-pilot.
And on the right, Bob checks Jerry in, and then, once Jerry’s on the plane, Bob the agent re-embodies into a
robot that's near the cockpit of the plane, goes inside the cockpit, and announces himself as the co-pilot.
We showed these storyboards to 197 participants using Qualtrics and Prolific.
For the personalization storyboards, we used Likert-type scales to assess perceptions of
creepiness and friendliness. And for the re-embodiment storyboards,
we looked at perceived competence and the agent's capability of taking on multiple roles.
Our primary analysis was on the qualitative data, but we used the Likert ratings to guide it. As a
first step, we organized the responses according to the scale ratings. Then, we annotated the
responses and wrote notes and memos about patterns and surprises. And then, our full research team
discussed those notes extensively until we reached consensus about how to interpret them as findings.
Through our findings, we generated four design implications.
First, if personalization is necessary to adequately provide a service,
then a robot should identify the user. Otherwise, service robots should avoid identifying people.
Personal storyboards were perceived more positively than impersonal ones,
but there was nuance across the different contexts. So recognition seemed much more
appropriate in the doctor’s office scenario, where proper identification is of course really
important, than in the other “expectation to be identified”-classified settings. And also,
people were concerned that other customers might overhear what robots were saying.
Second, service robots that identify customers should not do so using facial recognition.
There is a really critical ethical conversation to be had about the many harms of facial recognition,
but the part of that that’s really well within scope of this work is that we see a pattern
wherein when people actually envision themselves being recognized by a robot
and assume it to be tracking them biologically, they really do not like it.
We didn’t even mention facial recognition in the storyboards, but twenty participants
assumed it and reacted strongly to it in their free responses.
Third, in service contexts that are perceived to be low-risk,
service robots can re-embody to provide a better service experience.
And this comes from the fact that responses to the different versions of the air travel
and dentist scenarios contrasted more starkly with each other than responses to the different
versions of the hotel and personal care scenarios. And responses reflected concerns
about the possible risk of flying in an airplane and getting teeth cleaned. There was a sense that
robots taking on tasks with higher perceived risks would need to prove their competence,
whereas robots in tasks with lower perceived risk are more easily presumed to have that competence.
Finally, if robots fulfill several tasks in one larger domain, re-embodiment can improve
the service experience. But a robot should not re-embody to do tasks in different domains.
This implication came from a finding that belief that a robot could re-embody to do
multiple tasks typically done by different people was higher in the hotel scenario than
in the personal care scenario. People seemed to more readily buy into, or even expect,
a robot to do many different tasks that are part of a hotel service—whereas in the personal
care setting, hair staying and massage were two vastly different tasks that were not even
really part of the same service, and so they shouldn’t be done by the same “person” or robot.
To close, I'll reiterate some takeaways here: People do not want robots to identify
them for the sake of friendliness or customized recommendations—only
when they need to do so to provide the service.
Whatever is used for that identification should be something that’s really within the user’s control.
And this is something that people value when they think about themselves as these users.
Perceptions of risk and the need for demonstrated, specialized training
shape people’s comfort with robots re-embodying to take on different roles.
And, finally, as a last closing thought:
design research has methods for when we don’t know anything about a novel design concept,
and methods for when we already know quite a lot about what it is we're trying to achieve.
Likewise, it has methods that are experiential, data rich, and involve a smaller n,
like user enactments and many different user studies; and it has methods that focus on a
narrowly-scoped interaction, and deliberately lack context, and use a larger n, like online
studies using high-fidelity videos that test a specific interaction with a specific robot.
But online speed dating with storyboards is one way of serving a
methodological need not fulfilled by a lot of these other methods
in that it generates intermediate-level knowledge on evolving, mid-stage design concepts.
That’s all. Thank you to our summer REU students, our lab mates, our research
participants, and to the US National Science Foundation and NASA for funding this research.