- When Charles Darwin was deciding
whether or not to propose to his cousin, Emma Wedgwood,
he had a rather interesting approach.
He drew up a list of pros and cons,
as pros he listed things like children, companionship,
and the charms of music and female chit-chat,
and as cons he listed things like terrible loss of time,
the burden of visiting relatives,
and having less money to spend on books.
The list went on for quite a while,
and, by a very small margin, he decided to marry,
signing off, "Quad erat demonstrandum"
Latin for, it being proved necessary to marry.
What a lucky girl.
When it comes to thinking,
we tend to assume that more is better,
you'll make a better decision
the more pros and cons you list,
you'll just spend one more day thinking about that email
before you send it,
you've gotta think up that perfect one-liner
to ask your crush out.
But algorithms in machine learning tell us that
more information isn't always better.
In fact, placing too much value on unnecessary information
can lead to wrong predictions and results,
take this example, Darwin may have found it useful.
It's a study conducted on German couples,
the data points represent their life satisfaction
during their first 10 years of marriage.
Let's say we want a computer program
to make predictions about
what their life satisfaction will be
after the 10-year mark.
The very simplest prediction we could make
takes just one factor into account, time.
This is called a one-factor model,
and would create a simple straight line
on a chart, like this.
It captures the basic trend,
but misses a lot of the data points,
and if we follow it for long enough,
we'll see that couples become infinitely miserable
the longer they stay married,
which doesn't sound quite right.
So, if we tried to capture a more complex,
but slightly more accurate relationship
between time and happiness,
we could take two factors into account,
like time and time squared for example.
This two-factor prediction
passes through more data points,
and the trend seems to align more
with what psychologists say,
that there's a slight comedown after the honeymoon bliss,
but that life satisfaction
more or less levels out over time.
So if using two factors is more accurate than using one,
adding more information
should lead to a more accurate prediction, right?
Well, let's try it with nine factors.
It does a good job of modeling the current data,
as it fits through every point,
but it misses the overall trend.
It shows that couples are deeply miserable
right up til their wedding day,
their marriage is then a series
of severe ups and down,
and after 10 years,
they drop into a sudden, deep depression.
What's more, if the same study were repeated
with different couples,
the inevitable, slight variations in data
would have a huge impact on the nine-factor prediction,
but it would leave the two-factor model
pretty much unchanged.
In practice, obscuring data, adding random noise,
or withholding information,
often makes for better predictions.
By placing too much emphasis
on each individual data point,
we lose sight of what's really important, the trend.
This idea is called overfitting,
and it tells us that sometimes
the less we think, the better off we are.
Overfitting and the wrong predictions it causes,
are usually a result of placing too much emphasis
on what we can measure,
while forgetting what's actually important.
I know this all too well
from my high school and university days,
I was always working to get a good grade,
while not really trying to get a deep understanding
of the actual subject matter.
I'd get an A on a calculus exam,
but if my brother asked me,
"So, like, what is calculus?"
(blinking beats)
(muffled screaming) Well, I'd have no idea.
I was paying too much attention
to what I could measure, consequently,
missing out on actual learning, classic overfitting.
But overfitting isn't just in the school yard,
it's in our biology.
Our taste buds were designed
to help us detect which foods are healthy.
Sugar, salt, and fat
are all essential parts of our diet,
so it makes sense that we're programmed to think
they taste good,
but since we started modifying our food
to contain unnatural, and unhealthy amounts
of these ingredients,
taste has stopped serving
as an accurate guide to what's healthy.
Our brains our overfitting for an outdated metric,
optimizing for taste,
instead of what taste is supposed to provide, health.
As economist, Charles Goodhart, said,
"When a measure becomes a target,
"it ceases to be a good measure."
So what can we do about overfitting?
Well, in machine learning
there's a technique called regularization,
which can be thought of as a kind of complexity penalty.
For example, one method of regularization
is to scale down the weights of all the factors,
until most of them are at completely zero.
This means that only the most important few
have any say in the final decision.
In Darwin's situation,
this might've looked something like
crossing out the less important stuff
like children and companionship,
and keeping only the stuff that really mattered,
like not being able to spend as much on books.
Speaking of Darwin,
right after he decided to marry by proof,
he immediately started fretting about when to marry.
He wrote yet another list of pros and cons,
considering things like happiness, awkwardness,
expenses, always wanting to have traveled
in a hot air balloon,
not sure whether that was a pro or a con,
but by the end of the page,
he resolved to, "Never mind, trust to chance,"
he had his own methods of regularizing.
He married Emma Wedgwood,
and they lived happily ever after,
until he died of congestive heart failure at 73.
Thanks for watching, guys,
so I hope this video helped you out in some way
if you're prone to overthinking.
I know I definitely am,
even while I was making this video,
I usually fret about what to leave in,
and what to leave out,
and what you guys are gonna like,
and in the spirit of not overfitting,
I just thought,
"For this video,
"I'm just gonna go with my first instinct,"
and I had a lot more fun.
So I hope that you can get something out of this video too,
and if you're lucky enough to not be an over thinker,
share it with someone who is.
Oh and I should mention
that this video was based on
this amazing, amazing, amazing book,
"Algorithms to live By" by Brian Christian
and Tom Griffiths.
I've done a few videos based on this book,
which I'll leave for you guys in a playlist
at the end, on the end screen.
Yeah, until the next video, bye.
(upbeat music)