This is Tammy Dobbs. She lives in Arkansas and has cerebral palsy.
She's able to live alone with
help from a Medicaid program that provides at-home care.
I just, I have to have help
getting out of bed and getting dressed and getting in my chair and going to the bathroom.
For years, the state of Arkansas gave her 56 hours a week of care.
In 2016, they authorised just 32 hours that
meant that even the simplest, most necessary tasks took too long to receive help.
You know, nothing else got done, I had to put off taking a bath every day because I didn't have time.
So, who made the decision to cut her care?
It turns out that the Arkansas department of human services had started
to use a computer algorithm to decide how much care it would give to people with disabilities.
The algorithm could help make sure everyone was getting their fair share of time with a caregiver,
but for people like Tammy, this meant they didn't get the care they needed.
When someone applies for public benefits, there are usually three parts to deciding
how much help they'll get.
The state will gather information about the person,
figure out if they're eligible for benefits,
and decide how much care they should receive.
In Arkansas, the state wanted to modernise that last
step – allocation – by using an algorithm.
Before the algorithm, a nurse would ask you lots of questions
and then decide how much care you received.
In the new system, a nurse would still ask you lots of
questions, but instead of deciding your care plan based on their judgment, they would enter that data
into a computer form.
Based on those inputs, the algorithm would then group you with other patients.
The algorithm assigns everyone in this group the same number of hours.
Previously a nurse might have been able
to look at the beneficiaries and differentiate between the people who needed
less time and the people who needed more, but the algorithm did not see this distinction.
And it's not just Arkansas: automated systems have hurt vulnerable populations in Pennsylvania,
Iowa, New York and several other states.
Over time, the legal aid lawyers in Arkansas began to see hundreds of these sorts of cases.
In one of those cases, the designer of the algorithm was asked to calculate the outcomes by hand.
In the middle of that trial, he found that the code had not been implemented correctly.
The programmers working on the project had made mistakes when translating the description of the algorithm
into actual code.
Not only had the algorithm cut hours for some people, it had not even done so
correctly and no one in the department of human services had caught it or corrected it.
If the people running government agencies don't have technical knowledge, do they even understand the
algorithms that work in their state?
And if not, how can we keep these algorithmic systems accountable?
These are the very questions the state of Missouri is currently grappling with.
Since 2017, their department of health and senior services has been working on an algorithm
to figure out who qualifies for assisted care at home or at a nursing facility.
Missouri has been sharing versions of their design with the public for feedback.
In the first published version, advocates were able to find mistakes.
This prompted the state to revise the algorithm.
It wasn't until several iterations later that the state finally published how the algorithm
would affect the people who are currently eligible for benefits.
They found that it would take away
benefits for almost one in five people.
The state released another version of the algorithm and
though the latest analysis has not been published, they told the Guardian that the final algorithm
would take away benefits for about the same number of people.
For now, they're going to direct those applicants to the old system.
That way anyone who would have qualified before still qualifies.
Eventually, the state will drop people from the benefit roles and according to the state,
they plan to redirect those people to other resources.
But the details of that alternate plan are still vague.
Regardless of whether a human or an algorithm is doing this work, virtually everyone who works
in the space agrees on a key part of the problem.
Many state legislatures are simply not providing
enough money to give people the care they need to get by.
And that is a very human problem.