Hi, I am Wencan Zhang, a Ph.D. student from the Ubicomp Lab
at the National University of Singapore.
Here I am glad to share our paper -
Debiased-CAM to mitigate image perturbations
with faithful visual explanations of machine learning.
AI can recognize an object from an image
and generate a saliency map as an explanation.
Look at this example using Grad-CAM,
a common saliency map approach to explain
where the model looks at when labeling the photo as fish.
When working on clean images, we consider the saliency map as Unbiased,
since it can highlight correct pixels,
which enhances the model’s transparency and improves human trust.
So far so good.
However, when images are distorted, such as under image blur,
although the model can still recognize fish,
the saliency map becomes deviated and misleading
by highlighting the man’s face area,
and we consider it as biased-CAM.
We solve this problem with Debiased-CAM,
a self-supervised training approach that helps the CNN model
to generate less deviated explanations on distorted images.
The world we live in is not perfect where distortion, corruption or perturbation happens in data,
such as an image blurred with a privacy protection scheme,
with a wrong color temperature due to camera mis-settings.
or with varied daylight conditions.
Here to discuss them in a unified way,
we consider all these non-perfect images as biased.
Now let me show how our technical approach works.
A CNN model trained with clean images can produce more meaningful saliency maps,
or so-called Unbiased-CAMs.
However, when querying with blurred images,
this model not only has the chance to predict wrongly
but also generates deviated saliency maps,
or Biased-CAMs, that highlight irrelevant pixels.
One naive solution is by finetuning the model on blurred images as augmented data.
Although calibrating the prediction,
this still suffers from producing deviated saliency maps,
since the training process is not aware of the explanation.
To fix the issue, we explicitly formulate the saliency map generation
as a secondary prediction task.
We leverage the information from unbiased-CAMs
and construct a self-supervised learning task
to help the model generate Debiased-CAMs on blurred images,
which are similar to unbiased saliency maps.
Our approach also supports a more realistic multi-bias setting,
where training images are augmented to different blur bias levels
and the model needs to predict that information as the 3rd task.
Bringing all these features together,
we designed a multi-task, multi-bias architecture.
Let’s see how much CAMs are distorted
and how well Debiased-CAM can help.
We evaluated for different predictions tasks,
image classification and image captioning,
and different bias types, including synthesized biases
such as image blur, color temperature shift,
and natural bias such as day-to-night light change.
Across these, image bias leads to CAM distortions.
Relative to Unbiased-CAM,
we see that the Biased-CAM becomes very misleading.
This is still problematic if we only finetune the CNN model on biased images with their labels.
Fortunately, Debiased-CAM can recover the explanation faithfully,
which means even for strongly biased images,
Debiased-CAMs are similar to Unbiased-CAMs.
As can be seen here, when testing with the base model,
both the prediction performance
and the explanation faithfulness degrade with the increasing bias level.
Finetuning the model with simple data augmentation can hardly handle this issue,
especially for saliency map faithfulness.
In contrast, our Debiased training approach can improve
both the prediction performance and the explanation faithfulness
no matter under single-bias or multi-bias settings.
And our approach can achieve more faithful explanations
for different prediction tasks and bias types.
Next, we explored “How sensitive are people to biased CAMs?”
and “Are Debiased-CAMs good enough?”
in user studies with an image label verification task.
We objectively measured saliency map truthfulness
with a grid-selection question.
Here we considered a saliency map to be truthful
if the highlighted areas were consistent
with human’s grid-selection opinion.
Then we showed participants a blurred image
with different machine-generated saliency maps,
and asked them to give subjective ratings.
We launched the study with 32 participants
recruited from the Amazon Mechanical Turk platform.
We found that although saliency map truthfulness
degraded with the increasing bias level,
Debiased-CAM could always improve truthfulness.
We also measured whether participants consider the saliency map helpful in image label verification.
Instead of a common multiple-choice question,
we used the “balls and bins” question
to elicit the answer as a probability distribution.
This allows us to measure the participant’s confidence of the correct label
even if her most likely choice is wrong.
We collected the subjective helpfulness rating on the saliency map in a study with 162 participants.
The result indicates that
although all CAM variants do not help people label heavily blurred images,
and Biased-CAMs tend to degrade helpfulness,
Debiased-CAM can significantly improve users’ decision performance
on moderately blurred images.
And users generally consider the Debiased-CAM more helpful in labeling
regardless of the blur intensity.
By analyzing participants’ rationales, we found that,
Participants considered saliency maps more truthful or representative,
because the highlighted regions have the accurate size and shape
or captured more semantically relevant cues.
They would consider the saliency map helpful
for recognizing a blurry image
since it helped focus human’s attention or provided hints,
benefiting the process to form hypotheses and confirm suspected objects.
To sum up, we identified that the contextual bias in image may cause misleading explanations.
We provided a self-supervised approach,
to achieve debiased explanations,
and we designed sensitive measures for saliency map explanations
and found various advantages.
Please check our paper for more details.
Thanks for listening and I am glad to take questions!