Let's play guess the fake. Go!
Um, that one.
They're both real, they're balls.
What??
Hello World, it’s Siraj! In this episode, we’re gonna talk about Generative Adversarial
Networks. Deep Learning has done a lot for us, but it’s most incredible successes have
involved discriminative models. Discriminative models are great for classification tasks.
They Discriminate. Not in that way, obviously. Models like Neural nets, support vector machines,
and Hidden Markov Chains where we map high-dimensional, rich sensory input to a class label. A discriminative
model doesn’t care about how data was generated it just categorizes a given signal. They're
the secret sauce behind tools like facial recognition and language translation, and
sentiment analysis. But there’s another type of model that hasn’t been as succesfful.
the Generative model. Generative models are harder to get right due to the dififculty
of approximating some really hard probabilistic computations. A generative algorithm models
DOES care how data was generated in order to categroize a signal. It asks the question:
based on my generation assumptions, which category is most likely to generate this signal?
Anytime you’d like to generate some novel data based on some input data, whether that
be a chatbot’s replies or a song or even a short story, generative models are the way
to go.
So what is a Generative Adversarial Network or GAN? Well, GANs were introduced by a straight
G named Ian Goodfellow who didn’t give a Byte. The basic idea is this. You have two
models: a generative model G that generates new data and a discriminative model D that
estimates the probability that whatever sample data G shows it came from training data rather
than G itself. They’re essentially playing what’s called a minimax game. Minimax is
a well known strategy of always minimizing the maximum possible loss which can result
from a a choice that one of two players can make. So G is fed a bunch of training data
and attempts to generate new data based on that input. Every time G generates a new sample,
D will try to determine if the sample is from the models distribution or the training data
distribution. Think of G as a counterfeiter and a D as the police. They’re adversaries.
G will keep trying to produce fake currency and use it without detection, while D tries
to detect if its counterfeit or not. This keeps on happening, and both models improve
their methods until eventually the counterfeits are indistinguishable from the real thing.
So what type of model do we use for D and G? Well many different types of models can
be used, in the original paper the author used something called a multilayer perceptron
for both to generate photorealistic images based on a training dataset of images. But
there’s an even more interesting paper that came out that use GANs I wanna talk about
it. It came out just last month and it’s called ‘Generative Adversarial Text to image
Synthesis’. These guys created a GAN that can generate a photorealistic image based
on a caption. So when it was given the caption ‘a large blue octupus kite flies above the
people having fun at the beach’ it was actually able to generate an image that showed just
that. I’m dead serious. (Fake laugh)
So how did it do this? Well, they used a dataset of images and their associated captions as
the training data. First, they had to encode the data before they fed it into the models.
They used an encoder which was a hybrid neural network called a character level convolutional-recurrent
network. It takes both the image and caption as input and creates a joint representation
of both in multimodal space. It does this for both the generator and the discriminator.
Both models are a deconvolutional feed forward neural network. Once they had the encoding
they fed it into both models and each generated an image. The generator net creates a synthetic
image from the embedding, an image that is ‘close’ to the real image but not quite.
The Discriminator net receives 3 inputs (the real image with the right text) (the real
image with the wrong text) and (a fake image with the right text). It encodes eacth of
these and compares them to what the generator shows it, looking for similarity between its
inputs and the generators output. Each time step it will try to determine if the generated
image was synthetic or real by doing this comparison. After the determination has been
made, the weights of both the discriminator and generator are updated. So they both get
better over time. By the time training is over, the generator will be pretty good at
fooling the discriminator by producing photorealistic images that look like they must’ve come
straight from the training data.
So let’s build our own GAN. We’re gonna build a simple python script that demonstrates
how GANs work. Both models will be represented by learning gaussian distribution curves that
we plot using pyplot. Once we have our dependencnies in place, we’ll want to define our hyperparamters.
These are all different activation functions that we’ll apply to our models at some point.
We’ll also want to define how much we want to train our models as well as the number
of hidden layers we want them to have. We’ll then write 4 helper functions. The first two
will help use visualize our gaussian curves in our plot. G builds the generative netowork
and D builds the discriminative network. We’ll initialize our networks then write a plotting
function to plot both curves. Finally we’ll train our network using a for-loop. We’ll
get draw samples from a uniform distribution, then train both networks at different intervals.
We’ll plot the graph as we train our model so we can visualize the results. Let’s see
what it looks like as it trains. As it trains, the generator curve will look more and more
like the true data distribution curve as it tries to fool the discriminator network.
And there you have it. You might be wondering, what are the real world use cases for GANs?
Well, gaming is one. It’s a daunting task to create a huge 3D map manually, but with
GANs, programmers can write a part of a map and generate the rest. Another is interior
design. If a designer wants to visualize what a room will look like they can just ask the
machine for a photo of say a living room with beige sofas facing each other and 4 ninjas
in the scene. Pretty much anytime you need help visualizing anything, GANs can help fill
in the gaps and augment our own imagination. I’ve got some great links for you in the
descrption, please subscribe for more ML videos. For now, I’ve gotta go fix a deadlock so
thanks for watching :)