Transcriber: Paulina Kaniewska Reviewer: Rhonda Jacobs
When I was pregnant with my son, I was diagnosed with gestational diabetes,
and that meant a lot of extra tests and scans,
and checking my blood sugars four times a day.
But the funny thing was
every time I got a new test result, or a data point, if you like,
it seemed to show that I wasn’t a diabetic.
But when I raised this with the doctor at the hospital,
he was adamant that I was.
A few months went by, and I asked for another diagnostic test,
and it was negative.
So I went to the hospital and asked to speak with the doctor.
And I’m sitting in the doctor’s office, waiting for him to come in,
bracing myself.
If you’ve ever watched a modern day murder mystery,
you’ll know that forensic scientists go to great lengths
to prevent their evidence becoming contaminated.
You’ll have seen them wearing the suits, and the booties, and the gloves.
And we wear those things to stop our physical evidence,
things like DNA swabs, fingerprints, shoe marks,
coming into contact with people or things that aren’t relevant to the case.
Now, I’m a forensic scientist,
but today I want to talk about a totally different type of contamination,
and that's cognitive contamination.
And what that means is how our scientific thinking
can become contaminated by details about the case that we learn.
Of course, we all love to think that we’re purely objective,
and we always make a 100 percent scientific decision.
But sometimes things that have nothing to do do with science
can creep in to our decision-making,
and we’re not even aware that it’s happened.
My area of expertise is forensic toxicology,
and that means I’m interested in drugs and alcohol,
well, professionally interested in drugs and alcohol,
and the role they play in deaths and crimes.
And a really common type of case a forensic toxicologist might deal with
is driving under the influence of drugs.
In those cases, typically, we’re sent a single blood sample from a driver
and asked to test it.
So one day one of these samples arrives in my lab,
and the police officer in charge of the case phones me up and says:
“We’ve already caught this 21-year-old male driver nine times
for driving around on drugs,
so we’re pretty sure he’s guilty,
and the previous nine times he’d taken methamphetamine,
which you may know as crystal meth.
So the first step in any tox case is to decide which test to do.
Despite what you might have seen on CSI,
there is no one amazing “magic machine”
that can test for every drug and poison ever.
There is usually some element of picking and choosing a test,
and we want to choose the test that will be most useful for the case.
So, which test do you think would be most useful on this case?
Well, some of you might be sitting there thinking:
“Your job sounds pretty easy. Just test the blood for methamphetamine.”
And that is the most obvious decision.
But it's also a biased decision
because it’s been influenced by what I’ve been told
about the previous behavior of the driver.
And we call this “expected frequency bias.”
And it’s the reason jurors are not told about previous convictions.
But even if I know nothing
about the previous behavior of the driver,
just knowing a few personal details
can lead to the same kind of biased thinking.
Toxicologists will sometimes choose tests based on assumptions
about the sorts of people who use drugs.
Assumptions based on age, gender, ethnicity, even sexuality.
And the problem with these assumptions is that they are just that,
they are just guesses,
and they’re very often not based on anything particularly scientific.
And the danger in using them
is that it’s really easy to miss something.
I could just look for methamphetamine in this case.
But what if that result is negative?
What if the driver’s taken something else, and I miss it
because I don’t look properly,
because I’ve been swayed by what the police have told me.
So I need to be more open-minded, and I decide to do a screening test
that, yes, can see methamphetamine,
but can also see other drugs, like cannabis and cocaine.
A few days later, I get my test results back,
and they’re positive for methamphetamine.
“Surprise, surprise,” you might be thinking,
but remember I said it was a screening test.
What that means is that all I have is a simple “yes” or “no,”
and the result is preliminary.
It’s very like a lateral flow test for COVID-19.
The negatives are OK, but the positives need to be confirmed by another method.
In forensic toxicology, we call that method “mass spectrometry.”
And just like your PCR test for COVID-19,
it takes longer to do and it’s more expensive.
And it also gives us a more complex answer.
So instead of that simple “yes” or “no,”
we get a picture, a picture on a computer screen,
and it’s my job as a toxicologist to compare that picture to another one
and do a sort of a “spot the difference.”
So let’s look at our first picture, and we call this “a mass spectrum.”
Now, this is a picture of methamphetamine powder,
something I know is definitely methamphetamine.
You can see in the picture we have three vertical lines,
and each of those lines represents a number.
On the left - 91, in the middle - 119, on the right - 150.
Those three numbers together tell me that this is methamphetamine.
There is another important detail,
and that is how tall each of the lines is compared to each other.
You can see that the tallest is actually the one in the middle, the 119.
And the next tallest is the one on the right, 150.
Shortest is the 91.
Now, let’s look at our case.
Hopefully, you can see straight away that there are some extra lines in here
that are not in the methamphetamine.
So we’ve got a really big one there at 88, another one at 60, and so on.
What do those lines mean? Are they important?
Are they just some rubbish that I can ignore?
If you’re eagle-eyed, you will have also seen
the ratio between the lines isn’t right,
but that 150 out there on the right - that is shorter than it should be.
But yet, I do have my three numbers.
And this is the problem with real case samples.
They’re messy and hard to interpret.
And this is what we call in technical terms “an iffy match.”
And when results are iffy,
that's when that case information can start to creep in.
I think that this sample comes from
a 21-year-old male driver who’s already been caught
using methamphetamine nine times.
And knowing that I might subconsciously upgrade this
from “iffy” to “definitely positive.”
“Yes, OK, there are some extra lines in there,
but they’re probably just some rubbish that I can ignore,
and I have all three numbers.
I’m going to say it’s positive.”
Now, what if I think this is being taken from a 92-year-old grandma in a rest home?
I might subconsciously downgrade this from “iffy” to “definitely negative.”
“What about extra lines at the start? I really don’t think I can ignore those.
And the ratio between the lines is definitely off.
I’m going to say it’s negative.”
And if I did either of those two things, it would be called “contextual bias”.
The fact is that when the exact same data is interpreted as positive
for a 21-year-old male but negative for a 92-year-old female,
that’s a real problem for the criminal justice system.
It’s also important to know
that not all of the information I get is correct.
People don’t always tell the police the truth in the aftermath of a crime.
Some of the stuff I’m told is basically just hearsay
that would never be allowed in court,
but yet it’s allowed to influence my scientific decision-making.
So I phone the police officer, and I give him the news.
“I can’t be sure
there is methamphetamine in this sample.”
Now, that doesn’t mean it isn’t there. It doesn’t mean the driver didn’t do it.
What it means is there isn’t enough scientific proof to say
there is methamphetaminein the sample.
That might not seem to you like the best outcome
for the police or the prosecution,
but it’s the one that’s based on science,
not the one that’s based on what I’ve been told about the driver.
In the title of my talk, I mention a battle,
and there are many battles going on in forensic science right now
over these very issues.
There’s a particularly fierce one raging in forensic pathology right now.
And the pathologist works very closely with the toxicologist
because it’s their job
to work out the cause and sometimes the manner of death.
But in my field, colleagues will say to me:
“We use machines to get our data. And machines are objective, right?
So how could our results be biased?”
But as you’ve seen today, the problem exists between the keyboard and the chair.
Yes, machines give us objective data,
but it’s we humans with all our biases who decide what that data means.
My colleagues will also say to me:
“I’ll just forget what I’ve been told.
I just won’t use that when I come to make a scientific decision.”
Sadly, we just don’t have that kind of level of control over our subconscious.
So let’s go back to that doctor’s office
where I’m sitting, waiting, heavily pregnant.
On the one hand,
I have a single positive result for diabetes;
on the other, I have a mountain of negative data.
Then something I was not expecting happened,
and a new doctor walked into the room.
And she looked at the data, and then she looked at me and said:
“This is a misdiagnosis. You don’t have gestational diabetes.”
And I was so relieved, but it got me thinking.
“What was happening here?
Why were these two medical professionals looking at the same data
and coming to opposite conclusions?”
Or, to put it another way:
“What was stopping the first doctor seeing the mountain of negative data?”
And what was stopping him was that he knew something about me
that the second doctor didn’t.
He knew that diabetes runs in my family.
And because he knew that, he couldn’t forget it.
And he was convinced that because others in my family have had diabetes,
I must have it too.
And this is called “confirmation bias,”
and it’s the last piece of the cognitive bias puzzle.
Because once we learn something about a patient or a case,
we can’t unlearn it,
and we start to look for data or evidence
that is consistent with the thing that we’ve learned.
And we start to ignore evidence that is not consistent
with the thing that we’ve learned.
So I want to leave you with a question now.
Given what we know
about how cognitive contamination can affect forensic toxicology,
isn’t it about time we started protecting our evidence from it?
Thank you.
(Applause)