(siraj raval) Hello, world.
It's Siraj.
And what happens when we encounter data with no labels?
Can we still learn from it?
We're going to use a really cool model called
a variational autoencoder to generate
unique images after training on a collection of images
with no labels.
Most of the recent successes in deep learning
have been due to our ability to train
neural networks on huge sets of cleanly labeled data.
So if you have a set of inputs and their respective target
labels, you can try and learn the probability
of a particular label for a particular input.
Intuitively, this makes sense.
We're just learning a mapping of values.
But the hottest area of research right
now is learning from the raw, unlabeled data
that the world is absolutely brimming with, unsupervised
learning.
One type of model that can do this is called an autoencoder.
Autoencoders sequentially de-construct input data
into hidden representations, then
use those same representations to sequentially reconstruct
outputs that resemble their originals.
Autoencoding is considered a data compression algorithm,
but they're data specific.
So it can only compress data similar to what
it's been trained on.
They're also relatively lossy.
So the decompressed outputs will be a bit degraded compared
to the original inputs.
Usually a well known compression algorithm like jpeg
does better.
So what are they useful for?
Absolutely nothing.
Just kidding.
One major use case is data denoising
where we train an autoencoder to reconstruct
the input from a corrupted version of it
so that given some similar data that's corrupted,
it can denoise it.
Another is to generate similar but unique data, which
is what we'll do.
There are a bunch of different autoencoder types,
and each has its own unique implementation details.
But let's dive into one I find particularly interesting,
the variational autoencoder, or VAE.
While deep learning is a great way
to approximate complex functions,
Bayesian inference offers a unique framework
to reason about uncertainty.
It's an approach to statistics in which
all forms of uncertainty are expressed
in terms of probability.
At any time there is the evidence
for and against something, which leads to a probability,
a chance that it is true.
When you learn something new, you
have to fold this new evidence into what you already know
to create a new probability.
Bayesian theory describes this process mathematically.
The idea for the VAE came when these two ideas merged.
So looking at it through a Bayesian likeness,
we treat the inputs, hidden representations,
and reconstructed outputs of a VAE
as probabilistic, random variables
within a directed graphical model.
So it contains a specific probability model of some data,
x, and latent or hidden variables, z.
We could write out the joint probability
of the model this way.
Given just a character produced by the model,
we don't know which setting of the latent variables
generated the characters.
We've built variation into our model.
It's inherently stochastic.
Looking at it through a deep learning lens,
a VAE consists of an encoder, decoder, and loss function.
So given some input x--
let's say we've got 28 by 28 handwritten digit
image, which is 784 dimensions where
each pixel is one dimension.
It will encode it into a latent or hidden representation space,
which is way less than 784.
We can then sample from a Gaussian probability density
to get noisy values of the representations.
Let's start writing this out programmatically.
After importing the libraries and finding
our hyperparameters, we'll initialize our encoder network.
Its job is to map inputs to our latent distribution parameters.
We'll take the input data and send it
through a dense fully connected layer
with our choice of non-linearity to squash dimensionality,
which will be ReLU.
Then we convert the input data into two parameters
in a latent space.
We predefine the size using dense, fully connected layers,
again z mean and z log sigma.
The decoder takes the z as its input
and outputs the parameters to the probability distribution
of the data.
Let's say each pixel is either 1 or 0
since it's black and white.
We can use a Bernoulli distribution
since it defines a success as a binary value
to represent a single pixel.
So the decoder gets the latent representation of a digit
as input and outputs 784 Bernoulli parameters,
one for each of the pixels.
We're going to use these two variables we have here
to randomly sample new similar points from the latent normal
distribution by defining a sampling function.
Epsilon refers to a random normal tensor.
Once we have z, we can feed it to our decoder.
The decoder will map these latent space points back
to the original input data.
We'll initialize it with two fully connected layers
and their own respective activation functions.
And because the data is extracted
from a small dimensionality to a larger one, some of it
is lost in the reconstruction process.
But how much?
We need a measure of this, and that will be our loss function.
I want a generation data unpredictably,
and so my model is stochastic probability.
For a VAE, the loss function looks like this.
The first term measures the reconstruction loss.
If the decoder output is bad at reconstructing the data well,
that would consequently incur a large cost here.
The next term is the regularizer.
Regularizing in this case means keeping
the representations of each digit as diverse as possible.
If a two was written by two different people,
we could end up with very different representations.
That's a bad thing.
We only want one since they both wrote the same number.
We penalize bad behavior this way,
and it makes sure similar representations are close
together.
So our total loss function is defined
as the sum of our reconstruction term and the KL divergence
regularization term.
So this is the dopest part.
The way we normally want to train this model
is to use gradient descent to optimize a loss with respect
to the parameters of both the encoder and the decoder.
But wait, how are we going to take derivatives with respect
to the parameters of a stochastic or randomly
determined variable?
We've built randomness into the model itself.
Gradient descent usually expects that a given input always
returns the same output for a fixed set of parameters.
So the only source of randomness would be the inputs.
What do we do, cowboy?
We're going to reparameterize.
We want to reparameterize samples
so that the randomness is independent of the parameters.
So we define a function that depends on the parameters
deterministically, and we inject randomness
into the model by introducing a random variable.
Instead of the encoder generating
a vector of real values, it will generate a vector of means
and a vector of standard deviations.
Then we can take derivatives of the functions involving
z with respect to the parameters of its distribution.
We'll define our model's optimizer
as rmsprop, which performs gradient descent and its loss
function as the one we defined.
Then we'll train it by importing our digit images
and feeding them into our model for a given number of epochs
and batch size.
For our demo, we can plot out the neighborhoods,
the different classes on a 2D plane.
Each colored cluster represents a digit representation,
and close clusters are digits that are structurally similar.
We can also generate digits by scanning the latent plane,
sampling latent points at regular intervals,
and generating the corresponding digit for each of these points.
So the three key takeaways here are
that variational autoencoders allow
us to generate data by performing
unsupervised learning.
They are a product of two ideas, Bayesian inference
and deep learning.
And we can backpropagate through our network
by performing a reparameterizing step that
makes randomness independent of the parameters we
derive our gradients from.
Mike McDermott is last week's coding challenge winner.
He cleverly applied the multi-armed bandit problem
to the stock market.
Each BOC he created used a different investment strategy
based on reinforcement learning to profit,
and his IPython notebook is a great example
of what well-documented code looks like.
Wizard of the week.
And the runner-up is S. G. Eshwar.
He successfully took state into account, which made it
the contextual bandit problem.
Dope submissions, guys.
I bow to both of you.
The coding challenge for this video
is to use a VAE to generate something
other than digit images.
Details are in the read me Github Links,
go in the comments, and winners are going
to be announced next week.
Please subscribe if you want to see more programming videos.
Check out this related video.
And for now, I've got to deal with my hair.
So thanks for watching.