Hello, and welcome!
In this video, we will provide a high-level overview of convolution, and demonstrate how
a convolutional neural network could learn to extract features.
There are several ways to understand Convolutional Layers without digging too deep into the mathematics.
Let's explore some of the ideas proposed by the Machine Learning community.
When you start to learn how to program, one of the first things you do to improve the
development process is to build and apply functions.
Functions allow you to reuse important pieces of code in different parts of a program, which
keeps things organized and concise.
This analogy can be extended to Vector Graphics and Computer Aided Design software.
These programs will create and process the first object, but for all the remaining instances,
they just need to store basic information, like position and rotation.
Convolutional neural networks, or CNNs, use a special type of network structure that follows
this type of reasoning.
The network will contain multiple copies of the same neuron, and all of the neurons share
the same weights, bias, and activation function.
As a result, the neuron can be 'reused' so to speak in different parts of the input.
Traditional neural networks use a fully connected layer, where each neuron in a layer is connected
to every neuron in the previous layer.
Each of these connections also has its own unique weight.
This type of connection is general purpose, so it makes no assumptions about the features
in the input data.
It also tends to be expensive in terms of memory and computation.
A convolutional layer works a bit differently.
Each neuron is only connected to a few nearby local neurons in the previous layer, and the
neurons share the same weights and biases.
This type of structure only makes sense when the data is spatial, and has local features
that can be extracted.
So for an image, it would make sense to apply a small window around a set of pixels, in
order to look for some image feature.
By using the same weights, the net assumes that the feature is equally likely to occur
at every input position.
So this window can search all over the image, and can be rotated and scaled.
Feature engineering is the process of extracting useful patterns from the input data in order
to help the predictive model understand the true nature of the problem.
Finding the right features can improve an algorithm's accuracy and performance beyond
what a machine learning model could do on its own.
A feature learning algorithm will determine which features are useful for distinguishing
between different classes.
After this process is complete, the extracted features are ready to be used for classification,
or even regression.
CNNs are exceptionally good at not only finding features, but combining them together to increase
the complexity of the patterns.
The final layers of a CNN use these generated features for the task at hand.
So if you aren't an expert in feature engineering, CNNs are a great tool since they do a lot
of the work automatically.
This is an example of feature learning.
Notice how we start with simple features, and we eventually combine them to form high-level
features like a human face.
Consider this image shown here.
This is a basic binary image with black and white pixels, where a white pixel is represented
by a '1', and a black pixel by a '0'.
So this is a type of 'normalized' image where the pixels are constrained to certain values.
Another common example is an 8-bit greyscale image, where the pixels have intensities that
range from 0 to 255.
This animation shows how a kernel would be applied to the matrix that represents our
image.
Notice that a 3x3 kernel slides over the whole matrix, while multiplying the values element-wise
and summing them up.
Once the kernel has moved across the whole image, we form our convolved matrix.
It's worth noting that a 5x5 kernel could have been used as well, but that would have
been more complex.
You can see the big picture view of convolution through this image.
Notice how the kernel maps a region of the original matrix onto an element of the output
matrix.
As a practical example, applying the left kernel to the image will result in a blurring
effect.
This should make sense based on the kernel's structure.
If each pixel becomes the average of the grid of pixels surrounding it, we'd expect to get
that fuzzy look where parts of the image start to blend together.
This is useful for digital image editing and things like that, but this type of kernel
would erase the detailed patterns that a deep learning model would need.
So let's consider a different type of kernel that could find the edges in an image.
Edges are important because they can be used to build up higher-level patterns.
All this kernel does is take a pixel, and subtract the intensity of the adjacent pixel.
So if two adjacent pixels are similar in intensity, the result will be close to zero.
But high contrast areas, like edges, will have much bigger values.
You can see how this would look when applied to an image.
This type of output is useful for the initial layers of a CNN.
At this point, you should understand the basic ideas behind convolution and feature learning,
as well as the purpose of a convolutional layer in a network.
Thank you for watching this video.