Hi, my name is Matthew Hong
and I am a postdoctoral researcher at the University of Washington.
In this video presentation,
I’ll talk about how we can help product teams
plan for natural language failures with a tool called the AI Playbook.
Before I begin, I’d like to quickly acknowledge my collaborators
Adam Fourney, Derek DeBellis, and Saleema Amershi at Microsoft Research.
Our work is motivated by the recognition
that there are too many inefficiencies in the way we prototype AI experiences.
Specifically, failure behaviors of natural language AI products
are often found and addressed after they have been deployed to users.
And there are three main reasons why this could be happening.
First, AI-driven systems are unpredictable
because of the probabilistic nature of machine learning models.
So it’s often difficult for product teams to know what errors are possible,
and how users will react to those errors.
Without having a plan to counter these errors,
we will continue to build AI products that can be harmful to users and costly to fix.
So we set out to understand
how we can help product teams working on natural language products
address AI failures early enough in the prototyping process
With this goal in mind, our research progressed in the following three phases
First, to get a sense of the status quo,
we interviewed 12 practitioners in a single large software company
who worked in areas of search, writing assistance, conversational AI, and recommendation.
Then we developed a taxonomy of natural language failures
in common AI products, such as chatbots and writing assistants.
This presentation will not cover any details of the taxonomy,
so I would highly encourage you to read our paper to learn more about this.
Finally, we developed an interactive tool called the AI Playbook
and conducted an evaluative study.
One of our surprising findings from the first interview study
is that practitioners rarely engaged in any prototyping
because it slows down their pace of development.
When people were prototyping,
we found that product teams were so focused on the “hero or golden path scenarios”
that would only happen in ideal circumstances,
which made it easier for people to overlook failure scenarios
Finally, different product teams often experienced breakdowns in communication
due to having varying levels of AI expertise.
Using our taxonomy as a foundation, we designed the AI Playbook,
which is an interactive survey
that allows practitioners to walk-through a product or feature
that they are designing.
Imagine I were a designer working on a voice-based personal assistant.
I would start with Conversational AI,
then move on to select speech as the primary input.
My system will have a clear way of knowing when to trigger
because it listens for a wake-word.
As I continue to answer the questions,
the Scenario section interactively builds up the related failure scenarios,
which allows me to explore the consequences of the different design choices.
In the end, I’ll say my system will output a ranked list
which is pulled from an existing set.
The Playbook then outputs a full report of test scenarios
and actionable guidance on how to simulate them.
For example, we can simulate a list with reduced diversity
by inserting a duplicate item to the list or adding an article to a list item.
We evaluated the AI Playbook in a think-aloud session
with 9 of our previous participants.
In this presentation, I’ll highlight three key insights that emerged from the evaluation.
First, many people saw how the AI Playbook
could help them step off the golden path scenario
by systematically navigating the error space
and standardizing the types of errors that can be included in testing their prototype.
Participants saw another benefit of the tool
came from its ability to show the full range of options
and consequences of their design decisions
within a short 3-4 minutes of their time.
Finally, we found the potential for the tool
to help different teams overcome breakdowns in communication
by directing their attention to the same terminology and scenarios
in ways that both designers and engineers can relate to.
I would like to end this presentation by discussing opportunities to extend the AI Playbook to other scenarios.
First, because of our focus on early prototyping,
we limited our scope to failures happening within a single interaction session.
Future research can instead cover failure scenarios that occur in multiple sessions over time.
We also see opportunities to include scenarios
that impact end-users as well as other stakeholders.
Finally, the AI Playbook only covered a limited set of AI failure scenarios,
and we think there are opportunities to explore other scenarios
that are tied to the user’s personal, cultural, and societal context.
These failures could potentially lead to unethical or unfair outcomes
that can cause harm to end-users.
In light of these considerations,
we’re happy to announce that the AI Playbook is being released as an open source software
and that it is fully extensible to accommodate these alternative scenarios.
Thank you for listening and I look forward to hearing your thoughts
in the live Q&A session.