[MUSIC PLAYING]
JASON MAYES: In the last chapter,
you saw how to create your own models completely
from scratch, which is a very useful skill to have.
However, you also saw how long it
can take to train a network from zero as you start to use more
advanced model architectures.
State-of-the-art models may be much larger.
And even when training on the server side with a GPU,
it can take hours, days, or weeks to train.
Now, what if you could re-use a model that's already
trained on a similar task and repurpose
it to work with a similar custom data
faster by reusing the knowledge it already knows?
Well, if you think about it, we as humans do this all the time.
You have a lifetime of experiences contained
in your brain that you can use to help recognize new things
that you've never seen before.
Take this willow tree, for example.
Depending where you are in the world,
there's a chance she may have not seen this type of tree
before.
Yet, if I now ask you to tell me if there's any willow
trees in this image, you can probably spot them pretty fast,
even though they're at a very different angle
and slightly different to the one
that I originally showed you.
And essentially, you already have a bunch
of neurons that know how to identify tree-like objects
and other neurons that are good at finding
long, straight lines.
And you can reuse that knowledge to then classify
a willow tree pretty fast, which is basically
a tree-like object that has a lot of long, liney branches
in the vertical direction.
So in a similar manner, if you have a model that's already
trained on some domain, like recognizing images,
you can reuse it to perform a different but related task.
In this case, you can do the same
with an advanced model like MobileNet shown here.
MobileNet is a very popular research model
that's able to perform image recognition on 1,000
different object types.
From dogs to cars, it was trained on a huge data
set known as ImageNet that has millions of labeled images,
and the model itself is much deeper than anything
you've created on this course.
In fact, you can see, in this animation here,
the huge number of layers that it
has in this MobileNet v1 model.
Now, during its training, it's already
learned how to extract common features that matter
for all of those 1,000 objects.
And many of the lower level features
that it uses to identify such objects can
be useful to detect new objects that it's never
seen before too.
After all, everything is just ultimately a combination
of lines, textures, and shapes.
Now, if you could separate the pretrained lower-level
convolutional layers of an existing trained
model, as shown on the left, from the classification
layers near the end of the model, shown
on the right, which is sometimes referred
to as the classification head of the model,
you could then use this convolutional layer
to produce output features for any given
image based on the original data it was trained on.
Now, assuming the new image data can also
make use of such output features,
then there's a good chance that they can
be reused for a new purpose.
In the diagram shown, this model was trained on digits.
So maybe what was learned about digits
can also be applied to letters like A, B, and C, too.
So now, you could add a new classification head
that's trying to predict A, B, or C instead and then
train only the classification head, whilst freezing
the other convolutional layers, as they were already trained
to prevent them from changing.
Now, the act of doing this is known as transfer learning,
and it's what Teachable Machine does behind the scenes
that you saw in action at the start of the course.
You can also see that by only having
to train the multilayer perceptron at the very end
of the network, it will train much faster
than if you had to train the whole network from scratch.
So how can you get your hands on the subparts of a model?
Well, for more advanced research models like MobileNet,
if you head over to TensorFlow Hub
and then filter for models suitable for TensorFlow.js that
use a MobileNet v3 architecture, you'll
find results like the ones shown.
Now, some of these results are of type image classification,
and others are a type image feature vector, as shown.
These image feature vector results are essentially
the pre-chopped-up versions of MobileNet
that you can use to get the image feature vectors instead
of the final classification.
Now, models like this are often called
base models, from which you can then use
to perform transfer learning.
If you open up the page for one of these feature vector
MobileNet v3 models, you can see, from the JS documentation,
that it's in the form of a graph model.
As you know from previous chapters,
graph models are highly optimized for running as fast
as possible.
However, if you're lucky, sometimes you
may find versions in the layers format, which
can be far more useful for transfer learning,
as you can then combine your new classification
layers with the layers of the base MobileNet model
after training is complete.
You can then call model.save on that combined model
to get one model to use in the future,
instead of needing to load two models.
Also, if a model is in the layers format,
you can manually chop up the layers at any point you wish,
rather than having to rely on someone else
to provide you with a pre-chopped-up model.
Now, it should be noted that transfer learning sometimes
involves the retraining of the base model, too,
for fine tuning, but you shall focus
on the second form of transfer learning
that only retrains a new classification head that you
add to the base model instead.
Essentially, though, if you have a model in the layers format,
you can choose which layers to freeze
and which to unfreeze for training,
which can be very powerful.
Also worthy of noting is that the new model you create
will often be referred to as the transfer model.
So you already know that time of training
is a key advantage of using a transfer learning approach,
as you already have trained the base model to build upon.
But there's also other benefits, too.
Typically, you can get away with showing
far fewer examples of the new thing
that you're trying to classify.
You saw with Teachable Machine that you
could take about 30 images of any object in your room,
and it's then able to detect it pretty well.
Which is really great if you've got limited time and resources
to gather example data of the thing
you want to classify and need to make a prototype really
fast before gathering more training
data to make it more robust.
As such, given the need for less data
and the speed of training a smaller network,
transfer learning is less resource intensive,
which makes it very feasible for the browser environment
too, taking just tens of seconds on a modern machine,
instead of hours, days, or weeks,
for the base model creation.
All right.
So let's put all of this knowledge into practice
and actually perform some transfer learning using
state-of-the-art-models like MobileNet as your base model.
[MUSIC PLAYING]