I’m Sam Reig, and this talk is on the paper “Not Some Random Agent: Multi-person Interaction with a Personalizing Service Robot”.
This paper is co-authored with Michal Luria, Janet Wang, Danielle Oltman, Liz Carter, Aaron Steinfeld, Jodi Forlizzi, and John Zimmerman.
As different kinds of robots and other agents proliferate in our daily lives, a single user can interact with numerous systems in their life—and even in a day.
Today, I might interact with Siri in my pocket, Alexa on my kitchen counter, Alexa at my friend’s house, and a robot assistant at the grocery store. These agents all know different pieces of information about me.
Some of them also have information about the people around me. When I go to my friend's house and talk to their Echo, Alexa doesn’t know my music preferences—I’m a secondary user, not a primary one—
but it can respond to my request to hear a song nonetheless. The future of agent- and robot-related service interactions in a single user’s life will be complex. So what is the best design for these future interactions?
One idea is flexible agent embodiment, or the idea that a single agent can migrate across different physical devices.
In re-embodiment, an agent that interacts through one robot can transfer its social presence to another robot and resume an interaction there.
In co-embodiment, multiple agents can interact through a single robot at the same time. Prior work on these concepts found that people are accepting of agents that re-embody robots
and IoT devices, and that this design is especially useful for multi-step sequential interactions.
But interactions with agents won’t be limited to just one person and one agent, and not all service transactions are entirely sequential in nature.
In this study, we explored four research questions relating to re-embodying and co-embodying agents in service contexts with multiple humans.
First: How should a robot personalize its performance of service for multiple users? How does context influence this?
Second: How does co-embodiment influence people’s perceptions of the service robot experience?
Third: How does a sense of personal connection to a robot’s intelligence influence trust in that robot and feelings about the services it helps to provide? What is the social role of a universal personal agent?
Finally: How, if ever, should re-embodying agents cross contextual boundaries?
Since re-embodiment in HRI is largely a thing of the future and not the present, we approached our research questions with User Enactments—an exploratory method that’s been shown to be effective for
studies of technologies that are new and do not yet have social norms surrounding them.
In user enactments, participants are exposed to low-fidelity prototypes of possible technologies and the broader interactions that may involve them. In this way, they can reflect freely on which experiences and
design concepts worked well and what didn’t work well for them.
We envisioned how people might interact with three different types of agents.
The first was a singular agent: one agent that embodies one robot, is constrained to its physical service environment, and interacts with everyone who interacts with the service.
The second was a pair of personal service agents: two agents that co-embodied a robot together, each dedicated to serving one user.
The third was a pair of life agents: each user’s own personal agent would follow them from their home to the service setting and re-embody the service robot there.
We also iterated on our model service robot. In the end, we built up from an iRobot Create base: we used a cardboard skeleton for the body of the robot, and an iPad for the head.
The robot, and the voices of the agents that embodied it, were remote controlled by an experimenter.
We developed three service environments: a health clinic, a department store, and a hotel concierge. Our enactment scripts were written such that participants had a distinct goal in each of the three environments.
This was helpful for scaffolding our exploration of topics of interest:
sensitive information and trust in the clinic, specific knowledge of products in the department store, and decision-making and personal preferences in the hotel.
We encouraged participants to react and respond to agents freely, and we allowed dialogue to diverge from scripts when appropriate.
Participants signed up and came into the lab in pairs. Their real-life relationships varied, but we asked them all to imagine that they were “friends from work” for the study.
We ran 24 pairs of participants. They ranged in age from 20 to 76, and had a variety of personal and professional backgrounds.
Each pair of participants experienced one agent configuration in each of the service environments for a total of three out of the nine possible scenarios.
For example: one pair of participants might experience a singular agent in the health clinic, then personal service agents in the department store, and then life agents at the concierge in the hotel.
Another pair might see: life agents in the Department store first, then a singular agent in the hotel, then personal service agents at the clinic.
In the hotel environment, participants attempted to find a restaurant together. Here, we were interested in decision-making with assistance from agents.
We also had the Life Agents pull up the participants’ general food preferences —things they had learned back at home—
and we had the singular and personal service agents talk about the participants’ history with the hotel chain.
In the clinic, the robot helped participants check in for, and then attend, a flu shot appointment.
Here, we studied how a universal personal AI assistant could cross contexts — or refer to information and tasks that were time-sensitive, but irrelevant to what was happening in the present moment.
We had the agents give updates about an upcoming flight and a FedEx delivery.
In the department store, the robot guided participants around the store and helped them search for products they were looking for.
We also had the agents automatically check out using the participants’ stored payment information.
We were also interested to see how people would react to the agents drawing on their purchase history and pulling up their shopping lists unprompted.
You can see videos of some of these interactions in the video figure that accompanies our paper.
We conducted post-scenario semi-structured interviews after each enactment and a final interview at the end of all three enactments.
We transcribed our interviews and isolated over 1000 quotes to analyze. Through multiple rounds of affinity diagramming, we identified patterns and themes.
We also quantified responses to questions about the most comfortable agent configuration, comfort with a “chorus of agents” (two agents speaking at the same time), and responses to facial recognition.
We organize our findings by general themes and the relevant research questions from the set that I mentioned earlier.
Participants found two personal service agents to be redundant without adding any value to the service experience.
Life Agents were seen as the most comfortable configuration. Personal conversational agents that embody public robots can afford human-robot interactions that align with users’ values and mesh with their preferences for
social interaction—for example, one participant said, “I want it to be sarcastic because that’s how I am”, and another said, “I want it to embody a personality of my friends”.
Beyond conversational preferences, familiarity can provide emotional support.
Participants commented that in stressful or emotional situations, having a personal agent can be comforting, even if its behavior is not perceived as human-like empathy.
One participant said, “If you’re feeling anxious, it’s nice to have old friendly Basil along who knows everything about you” (Basil was the name we gave to one of the Life Agents).
When interacting with the Life Agent in the clinic, participants received notifications about things going on in their lives that had nothing to do with getting a flu shot—for example, an alert that a flight was delayed.
They had mixed feelings about these out-of-context notifications.
On the one hand, the ability to get live updates about things that require immediate attention or action provides a sense of convenience and immediacy that participants generally liked.
One said, “It seemed pretty natural that it would remind me at the appropriate time that I should check in for a flight”.
On the other hand, in certain situations, this context-crossing could lead to bleed-through that’s awkward, uncomfortable, or just unnecessary. As one participant put it, “Work should be work and home should be home”.
Relatedly, there was some tension between the desire for an agent that’s versatile in its role, and the desire for agents that have expertise in subject areas.
Many participants expressed concern about the ability of a universal Life Agent to be skilled enough to handle all interactions in all domains—particularly high-risk ones.
In a healthcare setting, for example, your personal agent might be able to assist you with getting a flu shot,
but for something more serious, you might want to call in an expert agent whose job is explicitly to assist with medical matters.
Participants also reflected on how co-embodying agents should handle multiparty interactions.
They wanted robots that use co-embodiment to recognize the existing relationships among human members of a group.
Some people thought they’d be comfortable having an agent co-embody a robot with another agent that belonged to their friend, but not with an agent that belonged to a stranger.
Participants also commented that agents can be mediators, providing people with information about each other.
When we asked participants if the co-embodying agents had a relationship with each other, most of them said no: they were just two independent entities that shared some hardware.
Co-embodying Personal Service Agents also spoke at the same time—which sounded like this
(sound clip: Agent 1: “Hi Alex”; Agent 2: “Hi Taylor”; Agents 1 & 2 together: “Welcome to the Homestead Inn in Boise”)
This garnered an equal number of neutral reactions, such as “that was nice”, and strong negative reactions, such as “completely unnerving”.
In summary, one design recommendation that we take from this is to design with “opt-in co-embodiment” in mind:
service robots should be able to be embodied by private agents, but they should not do this by default.
Another is clear signaling: Service robots using personalization to assist multiple users should clearly and repetitively indicate whom they are serving, which agents are active and which agents are idle,
and what will happen to data collected during the interaction.
We also noted a role-expertise tradeoff. People may want to interact with one “life agent” throughout various interactions, but also not trust a single agent that claims to be equally competent with all skills.
“Jack-of-all-trade” agents are likely better used for low risk tasks.
Finally, a methodological insight: the open-ended design research method of User Enactments can be combined with a study structure that mimics a behavioral experiment to uncover qualitative results
to better understand a potential future technology during the exploratory, but not absolutely initial, stages of its development.
Thank you to our undergraduate research assistants, Lynn Kirabo, Zhi Tan, Stephanie Valencia, our research participants, NSF, and NASA. Thank you.