Hi.
Today I'm going to present the work,
how child welfare workers reduce
racial disparities in algorithmic decisions.
I'm Hao-Fei and joining me is my coauthor Logan.
Let's start by going over the context of our study,
which is child welfare and
child maltreatment call screening.
In the US, there are millions of
child maltreatment referral cases each year.
Call screeners have to make
the decisions between screen in for
investigation or screen out if the case is of low risk.
A machine learning based risk assessment tool called
AFST was recently deployed in Allegheny County,
Pennsylvania to assist call screeners
with the decision-making process.
However, racial disparities are
deeply rooted in these decisions.
Prior work has pointed out that children of colors
are over-represented in the child welfare system.
And subsequently, AI systems also
make biased decisions because
they are trained on these data.
One question remains unanswered.
When people work with algorithms
in the child welfare context,
that is known to have racial disparities.
Will they serve to mitigate
or exacerbate these disparities?
Therefore, we're trying to answer
two main research questions today.
How human decisions affect
racial disparities in child maltreatment call screening,
and whether any changes in
disparities affect the decision accuracy.
To answer our research questions,
we compare between two decision-making paradigms:
an hypothetical AI decisions that
automatically screen in high-risk cases.
And then human AI decisions made by call screeners.
After reviewing the algorithm recommendations,
we evaluate the decision-making against two metrics.
Racial disparity, which is a differences
in screen-in rate between black and white children.
And accuracy, which is a proxy measure
of how good the decision makings are at assessing risk.
We conduct a mixed method study.
For the quantitative analysis we collected data
of call screening decisions from 2016 to 2018.
For the qualitative part,
We conducted contextual inquiry
by observing nine call screeners
and four supervisors,
we followed it up with semi-structured interviews,
aiming to understand how call screeners
incorporated AFST in their decision-making process.
Next, I'll hand over to
Logan to talk about our findings and results.
Thanks Hao-Fei.
I'm Login, I'm going to state our main findings.
So first, we looked at
racial disparity in the screen-in rate,
which we measure as the proportion of children who were
investigated after being reported to CYF, right?
So when we looked at the hypothetical AFST-only policy,
we found that the screen-in rate for black children was
20 percent higher than that of white children.
And this is over 2016 to 2018.
Right? So if we look at that same time period and then we
look at the actual screening decisions that worker,
that workers made. Right.
Worker-AFST only policy as we call it in the paper.
The racial disparities there decreased to about 9% rate.
So the AFST is getting
more racially disparate recommendations.
But then workers who used it ended up
making significantly less disparate decisions.
And we think the reason why this happens is because
workers agreed and disagreed with the AFST in certain ways.
So this figure shows the percentage of
children by race who were investigated or not.
And then broken down by the four possible scenarios for
agreement and disagreement between workers and the AFST.
And I know this is kind of a lot.
You focus on the two middle columns.
This is where workers and the FST disagreed.
So here you can see that
about 22 percent of black children were screened
out by workers when
the AFST would've said "screen them in", right?
And then that's versus that 13 percent of
black children who workers screened in,
When the AFST said screen them out.
This worker-AFST disagreement
reduce the overall screen-in rate for black children.
For white children, the analogous numbers were
17.6% and then 19.5%.
So that led to an increase
in the overall screening rate for white children.
So the screening rate increase
for white children and then a decrease for
black children reduce the overall disparity
within the algorithm's decisions.
Next, we wanted to understand
these patterns of agreement and
disagreement and how workers were reducing disparities.
So we actually went and talk to
the workers who use the AFST.
So through observations and conversations with them,
we hypothesized that workers did
three things to achieve a lower screen-in rate disparity.
First, they made holistic assessments
of risk and safety that were based on
contextual factors and data
that the AFST couldn't properly
take into account. Second workers
recognized and corrected for
limitations in the FST, right?
The main one is that the AFST over-
or under-scored based on prior involvement in
governmental systems like the criminal system,
the public medical system or welfare systems, right?
And third, workers made some decisions collaboratively.
This wasn't all decisions,
but for certain decisions,
maybe ones that they were
uncertain about or something like that.
They made them collaboratively.
So this we hypothesize,
reduce the possibility that
there were individual biases or
inconsistency for individual workers in overall.
Okay, So these three reasons
we hypothesize lead workers to
make less racially disparate decisions
than the AFST on its own.
Next, we also looked at disparities and accuracy.
And we found a similar decrease in
disparity in terms of accuracy between
the AFST-only and the worker-AFST policies, right?
And what's interesting here is if you look at
the AFST and then versus the workers though,
AFST actually made more accurate decisions.
And this is consistent the prior work.
But these results are a little bit complicated.
Because when we talk to workers,
we found that they didn't really agree
with the way that accuracy was measured.
Right in our work and in prior work,
accuracy has been measured against
the outcomes that the AFST predicts,
namely, we referral and foster care placement,
which are ostensibly proxies for child abuse or neglect.
However, workers thought that these proxies,
were oftentimes not really
indicative of abuse or neglect.
And instead, they looked at they being the workers,
looked at other outcomes when making screening decisions,
like whether a family had food in their fridge or
immediate problems involving drugs or domestic violence.
So it might be more precise to say
that AFST was more accurate than workers
when you measure accuracy based on
outcomes that the AFST is
trained to predict and that workers
are clearly not trying to predict, right?
It's unknown whether the AFST is accurate when
measured against outcomes that
workers are actually trying to predict.
So overall, our research shows
that workers use the AFST in
such a way that reduced
racial disparities present in the algorithm's
recommendations.
Workers likely did this through a mixture of
intentional and unintentional efforts
to correct for algorithmic biases.
In the paper, we also give recommendations on ways to
empower workers better to use their discretion
and possibly help reduce disparities
present in algorithmic systems.
And finally, we caution that
current measurements of accuracy may
be flawed because they assess
workers on outcomes that they're not trying to predict.
Please check out the full paper.
If you have any questions you can email me and
Hao-Fei, our emails are listed there.
Thank you to the rest of our coauthors: Anna, Venkat, Yang,
Diana, Adam, Ken, Steven and Haiyi.
And thank you for listening.