Welcome to the video! My name is Upol,
and on behalf of my co-authors--
Vera, Michael, Mark, and Justin, I’m excited to present key highlights of
this paper in this short spotlight video.
By the end of this video, I hope it will add a new dimension to how you think about explainability of AI systems.
So let’s get started.
Imagine this. We have Aziz, a software seller trying to use a powerful AI-based pricing tool
to do something very consequential – offer the right price to the client company "Scout".
So the AI suggest a price. Not just that, it explains
itself using top features it considered. Confident with the AI’s recommendation,
Aziz makes a bid.
But sadly, the client finds the price too high and walks out.
So what happened here?
Despite an accurate model with good technical transparency,
why did the bid fail?
Well, we might not find the answer by further digging inside the opaque box of AI.
There might be contextual factors outside it that can help explain--
for example, historical data of how others handled the recommendations.
But when it comes to AI explainability, why is there so much attention to look inside the box when much of the action is outside it?
Let’s look at the issue differently and think through the problem in a new light.
You see, when we talk about AI systems, what we are really talking about is human-AI assemblages.
The human part is often implicit
Now, we cannot tackle the explainability of the assemblage by just focusing on the technical side.
We need something for the human side.
So, if Human-AI assemblages are socio-organizationally situated, why aren't we incorporating this context into explainability?
In fact, this very lack of incorporation is an intellectual blind spot of XAI,
which is a centralizing focal point of the paper.
Not only do we highlight the blind spot, we also share how we can address it by expanding the conception of explainability beyond algorithmic transparency.
Conceptually, grounded in the design philosophy of Critical Technical Practice and related CSCW work in mediating human-human interaction,
we introduce and explore the notion of what we call of Social Transparency (ST),
which is the incorporation of socio-organizational context through past users' interactions.
But how do we encode this context? What are the design elements?
Let’s return to our example with Aziz.
Now imagine, what if Aziz had the following:
beyond what was there before, what if he could see that 65% of his peers rejected the AI’s pricing recommendation?
What if he knew that Jess, a director in the company, sold the product at
a loss due to Covid budgetary reasons?
You see, this peripheral vision of
who did what, when, and why –
what we call the 4W
can be the constitutive design elements that carry the socio-organizational context.
We derived these through a series of stakeholder workshops.
In fact, this very scenario that I just shared with you, we used it in a scenario-based design activity
with 29 real-world AI users from multiple sectors
and empirically derived a conceptual framework.
Which could have helped Aziz at three different levels.
At a technological level, it could have calibrated his trust in AI–
knowing what the AI knows, and more importantly, what it doesn’t know
At a decision-making level, the historical context could have improved the actionability of the information
At an organizational level, it could have helped him make more consistent actions with others.
So what does all of this mean?
We share many implications in the paper but here’s a major one.
By embedding the socio-organizational context into the conception of explainability
Social Transparency makes the human-AI assemblage more concrete.
This, in turn, makes it harder to take the humans, who are the producers of this context, out of the loop
and indiscriminately engage in automation.
To our knowledge, this is the first paper that explicitly argues
how we can the expand the conception of explainability beyond the bounds of the algorithm.
Thus, it takes a developmental step towards socially-situated XAI.
So where do we go from here?
"Well, at least for the foreseeable future, we will require a split identity –
one foot planted in the craft work of design
and the other foot planted in the reflexive work of critique."
So we invited you to use the framework and extend it and reflexively refine the concept.
And join us in expanding the very understanding of explainability in AI systems.
I hope this video has motivated you to read the full paper.
Thank you for watching!