2011 ...
Oh, hey there!
I was just listing all of the times that my team,
Southhampton Football Club, have won the Premier League.
Now, now, I should qualify this by saying that these dates are, sadly,
what is known as something called "synthetic data".
Which is a nice way of saying fake.
Sadly, Southampton have never won the Premier League.
Synthetic data is information that is artificially generated
rather than produced by events in the real world,
which sounds a little, well, worthless.
So perhaps it would surprise you to learn that synthetic data serves some very real productive purposes
and is increasing in popularity.
So let's discuss, first of all, number one, a definition of what synthetic data actually is.
Then we'll take a look at some uses and some benefits of synthetic data.
Why do you want all of this fake data around?
What can it do?
Then we'll take a look at number three, some challenges of this whole approach.
And then number four, we'll take a look at actually how to make some of this data,
which is the generation component.
That is all what we're going to talk about here with synthetic data.
Now, synthetic data is computer generated
and it's derived from existing data sets or from algorithms and models
to replicate the properties and characteristics of real world data.
And it's a broad term.
It covers a variety of processes and techniques
from simple data synthesis all the way through to deep learning models.
But why do we need all of this fake data?
Well, often the reason is that the real data is either hard to come by
or is sensitive confidential information that we can't readily get access to.
So now I'm talking about things like finance.
So financial records, those are going to be difficult to get hold of
and medical histories as well, things that might be confidential.
So that really brings us into this idea of talking about what advantages does that bring?
Well, there are a number of advantages.
So one of them is synthetic data is cheap and easy to produce.
And it's also has the benefit of being pretty good data, specifically that this data can be perfectly labeled data.
So it is exactly defined as we need it.
And real world data is often neither of these things.
But what possible use is it?
Well, that brings us nicely to Section Two: uses and benefits.
So a primary benefit lies in the data hungry world of artificial intelligence and machine learning.
A model can be trained on plentiful volumes of well labeled synthetic data,
with the intention of ultimately transferring the resulting machine learning algorithms to real world data.
And according to Gartner, by 2025, we will need 70% less real data to feed this hungry AI pipeline.
Now, synthetic data will provide domain-specific, well labeled, high volume data at a reasonable cost.
And using synthetic data, things like fraud detection algorithms
can probe trained models for security flaws,
and autonomous vehicles can test drive scenarios on road layouts that don't actually exist.
And synthetic data can be generated to minimize bias
that may exist in real world datasets, helping to make AI models more fair, accurate and trustworthy.
So this all sounds great.
Synthetic data for all.
Why even bother with messy real world data at all?
Well, that brings us nicely to to number three: challenges.
You see, synthetic data can't always accurately account
for the variety of real world factors that may affect a model's performance.
It can't replicate the kind of unanticipated events that may occur in real life.
Look, if ten years ago, I generated synthetic data for the next ten winners of the Premier League,
not many models would have included Leicester City as winners,.
but Leicester City did win the Premier League in 2015,
despite starting that season with 5000-to-1 odds that they would win the title.
Like they say, real life is often stranger than fiction.
So how do you generate synthetic data?
Well, the process is surprisingly straightforward.
In a nutshell, you need to define the type of data you require,
identify the data sources needed, and then generate the data according to your specifications.
And the simplest approach is to use some existing datasets and then manipulate that to create new examples.
So if we start here with an existing dataset, we can perform some sort of manipulation.
It might be that we would add some noise into that dataset, or we would transform some of that data to create new data.
There are also advanced techniques,
things like generative adversarial networks, or GANs,
that we can use for this, which generate data by learning from existing data.
And there are synthetic data generators
which use mathematical and statistical methods to generate data that follows specific distributions.
So synthetic data can be a powerful tool allowing us to generate data that is useful
and an accurate approximation of real world data.
But we do have to be aware of the potential pitfalls and challenges
associated with it, particularly when attempting to replicate real world data.
And most importantly,
don't trust any synthetic data that omits Southampton from a list of future Premier League winners.
2024 ...
I mean, you never know.
If you have any questions, please drop us a line below.
And if you want to see more videos like this in the future, please like and subscribe.
Thanks for watching.