I worked as a mathematician and then as a quant in finance - I saw the worst of finance.
I went into data science and I was struck by what I thought was essentially a lie – namely,
that algorithms were being presented and marketed as objective fact.
A much more accurate description of an algorithm is that it’s an opinion embedded in math.
An algorithm is a very general concept - it’s something that we do actually in our heads
every day.
To build an algorithm we need only two things, essentially: a historical data-set and a definition
of success.
So I build an algorithm to cook dinner for my family.
The data that I use on a daily basis is the ingredients in my kitchen – sometimes the
time I have, the ambition I have for that dinner.
And then I assess the dinner after the fact – was it a success?
I define that because I’m the one who is building the meals – I’m in charge, I
have the power (there’s always a power element here) and I’m in charge; I get to decide
a meal is successful if my kids eat vegetables.
My kids, if they were in charge, would have defined it differently.
And it matters because we optimise over time; we optimise to success.
The succession of meals that I cook from month to month, is a very different path of meals
than if my son were in charge.
So we do that every time we build algorithms – we curate our data, we define success,
we embed our values into algorithms.
So when people tell you algorithms make things objective, you say ‘no, algorithms make
things work for the builders of algorithms.’
In general, we have a situation where algorithms are extremely powerful in our daily lives
but there is a barrier between us and the people building them, and those people are
typically coming from a kind of homogeneous group of people who have their particular
incentives - if it’s in a corporate setting, usually profit - and not usually fairness
towards the people who are subject to their algorithms
So we always have to penetrate this fortress.
We have to be able to question the algorithms themselves, especially when they are very
important to us.
We have to inject ethics into the process of building algorithms and that starts with
data scientists agreeing and signing a Hippocratic oath of modelling.
We have to stop blindly trusting algorithms to be fair - they are not inherently fair,
they are inherently picking up whatever bias we’ve given them – and start looking into
what they are actually doing.