This video presents our CHI2023 paper, titled Designerly Understanding:
Information Needs for Model Transparency to Support AI Design Ideation
This work is situated in growing HCI research that studies challenges that UX design practitioners
face when working with AI technology as a design material, and how to better support their work.
Among other challenges, designers need to understand this complex technology. A
designerly understanding is not necessarily a deep technical understanding, but being able to link a
technology’s capabilities and limitations to what values it can deliver to users.
This understanding of limitations is especially important for responsible AI practice, that is,
developing AI-powered application that is ethical and minimizing potential harms.
Meanwhile, we recognize that supporting understanding of
AI has long been the goal of the resarch area of AI transparency.
Among other things, this area has produced various transparency frameworks for model,
data and AI services, that is, what information about them should be communicated to stakeholders.
And these frameworks are widely adopted in the industry through
documentation features such as model card, transparency note, and datasheets
In this study, we aim to bridge these two areas of work, and explore opportunities and gaps for
transparency model reporting frameworks to support designers to understand models
We explore this in the context of pre-trained models
Which may broadly encompass AI services, model market places,
as well as recently developed large pre-trained models.
As they significantly reduce the barriers to use AI capabilities,
understanding them could serve as a first step toward enabling
desigenrs to use pre-trained models as a toolbox of design materials.
We note that these services are currently not targeting designers,
and often assume direct users are ML or software engineers.
But this is a problematic assumption, as engineers may not be best positioned to ideating on how to
use a pre-trained model for a given application. And we emphasize responsible ideation, that one
must responsibly decide whether, where and how to use a model properly for real user needs,
with minimized harms to users. So this study also serves as a need-finding study toward enabling
designers to perform responsible ideation, and play a more central role in RAI practice
More specifically, we ask the following research questions.
First, What are UX practitioners’ needs and challenges in working with pre-trained models?
Second, to what extent are current model transparency frameworks useful for design ideation
And lastly, What are the goals that designers seek model understanding for and how to support them?
we conduct an interview study with 23 UX practitioners. After discussing
heir own experiences with AI design, we introduced a scenario-based design task.
We ask them to put on the hat of a designer working for a microblogging plaftform,
and they are aware of this user problem of people sharing articles without reading them
Their company purchased an AI service including a pre-trained summarization
model. Their task is to come up with a new feature of this platform that can
utilize this model to tackle this “sharing without reading” problem
participants were first given a model documentation created based
on transparency framework, and then they could browse input-output examples
from the playground UI, which is a very common feature of services providing pre-trained models
Then they were asked to ideate on this new feature
The study also include two iterations of their design ideas including reading a
list of potential harms that can result from summarization models
Here I focus on presenting results from the design task, while more discussions
around participants’ experience working with pre-trained models can be found in the paper.
We ask what their transparency goals are, for each goal, what provided information
in the documentation they used, what additional information they requested. Based on that,
we derive design implications to support these goals.
We have this big table in the paper that I hope you will read in detail. In the next few slides,
I will walk through them at a relatively high level.
The first common goal is to understand the model well enough to be able to perform
divergent-convergent design thinking. To be able to generate many potential ideas,
then eliminate ones that are risky to users or delivering less benefits.
This convergent stage requires designers to have a comprehensive understanding of
model limitations and then translate them to potential harms specific to their designs
and users. For example, participants found information about the intended
and unintended use in the documentation useful. They also requested additional
information about general characteristics of model outputs, such as distribution of lengths
and linguistic properties, to anticipate how users would perceive and react to the outputs.
A second goal is to create conditional designs, that is, different designs for different inputs,
outputs, or different user scenarios, to be able to mitigate AI’s varying impact. For example, in this design,
P3 considered only applying the model to long inputs, blocking outputs that contain sensitive
content, and creating a different design for users who share many articles in a row.
This goal gave rise to participants’ prominent need to understand what factor can vary model
behaviors. Importantly, they care not only what varies model performance, but also many design
specific output characteristics, such as output length and coherence, and also API performance.
Participants wanted to discover many factors and test factors based on user scenarios they
could think of. To do so they requested additional information about the training data,
disaggregated evaluation, and explanations.
A third goal is to provide transparency to end-users, especially to expose limitations and
potential harms to them. This often requires participants to translate information in the
documentation to effective UI features. This goal also highlights that an AI product cannot
be truly transparent if the designers themselves do not have a good understanding of the model.
Designers’ last goal is to be able to negotiate and collaborate with the team
to advocate for users. A good understanding of model limitation and how it can be improved,
such as customizability, can empower designers to better engage with the engineers.
Participants’ comments also revealed two additional roles that model documentation
can serve for product team collaboration: one is as common reference and boundary object.
The other is to help designer gain AI literacy, for which some participants expressed desires
for some level of technical information such as the algorithm and training data
Overall, our study demonstrates the utility of documentation for supporting design ideation, but
also points out many gaps, calling for supporting understanding model limitations specific to
designers’ use case, input data, questions and hypotheses through model interrogation.
We suggest a few directions for future work to develop
tools to support designers’ model interrogation.
One is to support example-based interrogation,
but in a more principled fashion. This could mean to allow designers to create or customize
input dataset, guide them to explore the input and output spaces, engage in failure-oriented
interrogation such as discovering what kinds of input can lead to risky outputs.
A second direction is to support disaggregated evaluation of models
to inform conditional designs. Different from conventional disaggregated evaluation,
designers can benefit from defining evaluation metrics specific to UX impact,
and exploring factors based on different user scenarios
Lastly, we hope our study provides some inspiration for tools that integrate
model transparency support and design prototyping.
I want to end this presentation by highlighting the point that designers can and should play a more central
role in RAI practices, especially in the age of pre-trained models including GPT. They are best
positioned to ideating on whether, where and how to use these models because they are trained to
bridge user needs and technical affordance. With this kind of training, they can also translate
technical limitations to risks and potential harms to users to be able to avoid or mitigate them.
Participants’ designs also illustrate a broad range of UX tools that designers
can bring for RAI by creating interaction level interventions such as putting guardrails on model
input or output, providing transparency to end users, and supporting user agency
with that, id like to conclude the talk and thank you for watching the video