SIRAJ: -Hello World, it's Siraj!
In this video, we're gonna create a neural network
that makes images become really trippy
using Python and TensorFlow.
We humans have been using psychedelic drugs
since prehistoric time.
-That wasn't a normal mushroom!
They help manifest parts of our mind
to conscious experience that wouldn't normally otherwise.
One of the most common experiences
from psychedelics are trippy visuals.
People who have taken psychedelics
repeatedly describe experiences of seeing
both open-eyed and closed-eyed visuals.
But how? What are these drugs doing in our brain
that makes us see things that aren't really there?
The traditional way for us to find out
is to test live human subjects tripping balls
under an fMRI machine,
but recently, we've gotten artificial neural networks
that do the same thing,
Google trained a neural network
on a labeled dataset of images --
everything from squirrels to temples --
and as it trained on these images,
it built internal representations at each layer.
Eventually, the first two layers learned low-level features
like lines and edges, and they got progressively more abstract
so the last few layers were representations
for faces and big shapes.
So when we visualize one of
the higher-level representations it built,
we can see that it contains a mixture of features
like the eyes of a dog and the head of a bird.
Eugh!
And when they gave it a novel image
and asked not to classify it
but to maximize the similarity between the image
and a representation at a particular layer,
the result was a very trippy image.
It all sounds very similar to the drug experience
which is insane.
Our brains are carbon-based
and they use chemical signals as messengers.
A neural network doesn't even exist
in physical space at all.
It's an abstract concept being represented
on binary silicon transistors.
There's no reason to expect that these two systems
would develop the same mechanism
for processing visual information.
Even with existing similarities,
natural selection is very different from using
gradient descent to alter weights
of connection between nodes.
Could it be that, somehow,
encoded in the fundamental rules of the universe
there's this ideal way of doing object recognition?
TRINITY: -What is he doing?
MORPHEUS: -He's beginning to believe.
SIRAJ (AS NEO): -You're damn right, Morpheus.
-We're getting closer to understanding it every day,
and we can learn a lot about the brain
including human development,
treatment for cognitive disabilities, and drug effects,
from studying artificial neural networks.
After we install our dependencies,
we're gonna replicate Google's Deep Dream code
in TensorFlow, and then test it out on a novel image.
NumPy will be used to perform math calculations.
The 'partial' submodule of functools
lets us create new versions of functions
with one or more arguments filled in.
This is good for reusability,
which means less code to write.
Pillow is an imaging library,
and Image will help us modify our images.
TensorFlow is our machine learning library.
urllib will let us download data from the web.
os will let us use
operating system-dependent functionality,
and ziptools [sic] will let us run FREE!
No. It'll just let us unzip files.
Our imports are already at the top of our script
so we're going to download
Google's pre-trained neural network,
create a TensorFlow session,
pick a layer and a pre-trained network
to enhance our input image,
apply our gradient ascent to that layer repeatedly,
and then output our newly Deep Dream'd image.
Let's start by downloading
Google's pre-trained neural network called "Inception"
in our main() method.
We'll store a link to it in the 'url' variable,
create a data directory where we will extract it to,
then use the os module to retrieve the model name
and create a local ZIP file path.
If there is nothing at that path,
we can download it using the urllib module
with the 'url' variable as a parameter
and store it in the 'model_url' variable.
We'll open our ZIP file with the 'wb' flag
so we can write to it in binary,
then write the downloaded data to it.
Then we'll extract that using the ZipFile module.
Now we can create our TensorFlow session.
We'll load our Inception graph file
into our 'model_fn' variable, then initialize the graph
using the Graph() function of TensorFlow.
Now we can initialize a session using that graph.
We'll open our existing saved-in-session graph
using the FastGFile() function
and point it to the saved graph.
Once we've opened it, we can read that graph
and parse it accordingly
using the ParseFromString() method
of TensorFlow's graph definition module.
We need to define our input, so we'll create
an input tensor using the placeholder() method
called 'input' with the size of 32 bits.
Then we'll define image net mean value of pixels
in an image as 117.
By removing this value from our image,
it will help us with feature learning,
so we will subtract it from our input tensor
and store the value in our 'preprocessed' variable.
Then we'll load the 'graph_def' variable we initialized
with the input as the newly processed tensor.
So now we've got our TensorFlow model,
we've downloaded from the web,
and we've loaded it into our session
as a graph with a bunch of layers.
It's a convolutional neural network,
the type of neural net that helps recognize images.
Let's load all those layers into an array
and store them in our 'layers' object.
So for every TensorFlow operation in our graph,
if it's a convolutional layer, load it into our array.
So we've got
♫ layers, lay-layers, lay- ♫
Yeah, we're balling hard right now, I know.
OK, so each of our convolutional layers
output tens of hundreds of feature channels
to pass data in the graph,
and we can collect them all
and store them in the 'feature_nums' variable.
Let's print them out and visualize what we've got
in terminal first.
We can see our number of layers
and the total number of feature channels right here.
Let's now pick a layer from our model
that we're going to enhance.
We'll pick a lower-level layer
and pick one of the feature channels to visualize.
It's time to load our input image
using the Pillow Image submodule's open() method
and store it in our image variable.
We'll format it accordingly using NumPy
and perform Deep Dream on it
with our render_deepdream() function
with a focus on the layer we selected earlier.
In our Deep Dream function,
we can see a couple of our predefined hyperparameters.
We'll start by defining our optimization objective,
which is to reduce the mean of our input layer.
The gradients() function lets us compute
the symbolic gradient of our optimized tensor
with respect to our input tensor.
Now we can split our image into a number of octaves
and say for each octave, let's resize it using NumPy
and add it to our array of image octaves.
Then we can generate details octave by octave
by iterating through each.
Random shapes are applied to the image
to blur tile boundaries over multiple iterations
using the calc_grad_tiled() function.
We're essentially applying gradient ascent here
to maximize our loss function,
which merges our saved representation
in this layer with our input image
more and more every iteration.
So to break it down,
neural networks are a great test for learning
about how the brain functions
and responds to certain stimuli.
They store increasingly abstract representations
of what they learn in their layers,
and we can create trippy images
by applying a gradient ascent process to them
based on any chosen layer
of a pre-trained convolutional neural network.
The winner of the coding challenge from the last video
is Avhirup Chakhraborty.
He created a pretty cool IPython Notebook
to demo his code, and trained his neural net
on both price and sentiment data using Keras.
Badass of the week!
And the runner-up is Victor Ciurana.
Very well-documented code that asks for user input
on stock prediction.
The coding challenge for this video
is to modify this code so it works not just on images,
but video. Details are in the README.
Post your GitHub link in the comments
and I'll announce the winner in the next video.
Also, I created a Slack channel
for all of us programming Wizards
to learn from each other.
The link to sign up below.
For now, I've got to stay sober, so thanks for watching.