Hi everyone! I'm Ariful Islam Anik. I'm a first-year PhD student at the University of Manitoba,
and I'm happy to present our paper, Data-centric explanations:
Explaining Training Data of Machine Learning Systems to Promote Transparency.
Machine learning algorithms are used in a range of automated systems we use in our life.
Most of the systems are data-driven, which means that they work based on the data they have,
which is necessarily the training data of the system.
However, the challenge with this approach is that someone has to pick these training data,
and owing to the limitations of resources and availability, sometimes these data are
not accurate and they are not representative of the the population who might be using the systems.
This lead to several performance issues and biases that we see with existing machine learning systems.
For example, most face recognition softwares seem to perform poorly for people of color.
Now one explanation to this is that these systems has been trained with
images of mostly white faces which are not representative of the population.
Now, despite the importance of training data, for a number of stakeholders,
the training data and the training process is a complete black-box.
They don't have any information on it. To provide transparency into this training process, in our
work, we present training dataset information to potential end-users of the system, and then
we study how this information might impact their perceptions of machine learning systems.
Now we presented this information through an explanation prototype where the goal of
the prototype was to communicate important information on training data including how
the data was collected and what data was used in the system. We grouped information about training data
into five categories, where for every category we provided some overview information.
Within each category, we presented the information in a q&a based approach
so that the reader can easily focus on the information that they're particularly interested.
On the right, you can see the questions and answers for the demographics category,
where the prototype, in particular, is displaying the gender distribution of the training instances.
We evaluated our prototype in an user study where the goal of the study was to explore
how these explanations impact users' trust and fairness perceptions of machine learning systems.
We conducted the study with 27 participants where we explicitly recruited participants with a range of expertise.
Participants interacted with a number of explanations from example systems
where we varied the stakes of the systems, ranging from low to high stakes,
and also the quality and the balance of the training data presented into these explanations.
We used questionnaires and semi-structured interviews to collect data from our participants.
Our study data shows that these explanations are positively received by participants
and they have a positive impact on end-users 'trust and fairness perception of the systems.
Participants talked about how they trust the system more because they
have seen this explanation and they feel that they know more about the system.
Among all the information presented, participants found the demographic information to be the most
useful and they talked about how it helped them to identify any potential biases in the system.
For example, this participant mentioned that since 90% of the data were from males,
they can easily conclude that the the system would perform better for men.
Additionally, we saw some interesting insights that machine learning experts
might be underestimating the capabilities of general users of machine learning systems. Some
expert participants in our study seemed to think that these explanations would be too difficult
for people without any computer science or machine learning background. However, we did not
see any evidence of our non-expert participants finding this information difficult to understand.
To summarize our contributions, we designed data-centric explanations that communicate
training dataset information to end-users of the systems, with the goal of increasing system transparency.
We also presented study findings that show that these explanations have the
potential to inform participants' judgment of trust and fairness in machine learning systems.
Going forward, it would be important to explore how we can evaluate these explanations in the
context of real-world systems. And in doing so, we can better understand how
this type of transparency might impact users' interactions with machine learning systems.
We hope that our work would motivate other researchers to work with
different aspects of data-centric explanations. Thank you everyone for your attention.