[MUSIC PLAYING]
JASON MAYES: In the prior section,
you utilized a pre-trained and pre-chopped up
graph model to generate image features you could then
use in a transfer learning based approach with your own custom
classification head.
Now, this is great, but you can go one step
further if you're fortunate enough
to have the layers model available to use as a base
model.
By doing this, it allows you to get three key benefits.
First, it allows you to fine tune the base model
to squeeze out a bit more accuracy, if needed.
Once you trained a new classification head,
you can then unfreeze layers individually
in the base model to refine them slightly
for the new task that might give you slightly better accuracy.
Secondly, you can select for yourself
what layer to use as your features
into a new classification head that you plan to add.
With a layers model, you can sample
the output of any layer in the base model, not just
the last one.
And finally, once trained, you can combine the base model
with your new classification head
and then use model.save to save all of the layers together
so you have just one model.jsn to deal with instead of two
when trying to use the trained model on other websites.
All right.
So let's take a look at loading a layers model
and chopping it up to be used for transfer learning.
For this exercise, you'll start with a completed
example from the last lesson.
Feel free to remix a copy of mine,
which is available on the link on the slide shown.
In this section, you'll be editing code contained
in the script.js file.
So go ahead and open that up.
First, add a new variable called mobilenet base
at the end of your variable definitions
near the top of the file as shown.
Set it to undefined for now.
This is what you'll use to store the chopped up MobileNet
model later on.
Right below this, you can also define a new function
called custom print that takes one parameter, a line of text
that you wish to print.
This function takes a string that has passed,
creates a new paragraph element, sets for text to be the string,
and then adds it to the end of the document body.
Sometimes it can be useful to define your own printing
functions instead of using the console,
and you'll see this function in use later.
OK.
So moving down the code to your existing load MobileNet feature
model function, it's time to make some updates to use
a layers model instead.
First, change the URL to load a MobileNet v2 model I've already
converted for you that's in the layers model format using
the URL shown.
You can then use the await tf.loadLayersModel instead
of load graph model to load this new model
and assign it to the variable called MobileNet.
At this point, you can also update the status text
with the updated string of MobileNet v2
loaded successfully.
Once loaded, you can now call mobilenet.summary,
but this time with three parameters.
The first two, you just leave as null.
And the third one you can pass your custom print
function that you just defined.
What this means is that your model.summary will now
use your custom function to print itself out instead
of going to the console.
In fact, running a code as is right now,
you'll see the model layers printed
at the end of the body in the HTML as shown on this slide.
And if you scroll down to the end of all of this output,
you can find the name of the second to last layer
before it goes to a dense layer for making classifications.
Here, you can discover the name of the second
to last layer, which is global_average_pooling2d_1,
which is what you'll use shortly in your code.
OK.
So go back to the code, and let's now grab a reference
to that second to last layer.
You can do this by calling mobilenet.getLayer and then
passing the string of a layer name that you want.
Store this result in a constant called lastLayer for now.
Next, you can create a truncated version of the model
by calling tf.model to which you pass an object that
specifies the input starting point and the output's ending
point.
In this case, the inputs are the same as the original inputs.
So just use mobilenet.inputs to grab a reference to those.
And then for the outputs, use the lastLayer.output
from the reference you just obtained on the previous line.
The result of this chopping up will
be stored in your MobileNet base variable.
At this point, you can print the MobileNet basis summary
to the console, as you've done in prior lessons.
Or if you prefer to use your new print
function to document that instead,
feel free to use that as well.
Finally, nothing else has changed
within the code at the end, other than the name used
in the warm-up prediction.
Now, it uses MobileNet base instead of MobileNet.
So make sure you update that.
Also, this v2 model uses the same input shape as MobileNet
v3 in the prior lesson.
So everything else is the same as before.
So running the code at this point
and switching to the console once the MobileNet
model is loaded and chopped up, you
can now see the output shown.
Note, that the last layer in this truncated version
is the global_average_pooling2d_1
layer, and the dense layer no longer exists.
You can also see that the output shape of this model
is 1, 1,280, meaning a batch of 1 and in this case,
it has 1,280 feature values.
OK, so moving further down the code,
you can now update your multi-layer perceptron
to take an input shape of 1,280, given
that this is the new size of the output
from the chopped up MobileNet base model
that you just created.
The rest of the code in section stays the same as before.
Next, head to the calculateFeaturesOnCurrentFrame
function that performs for data gathering
when you click the buttons.
The only thing to change here is the model
used to make the prediction.
Ensure you update this to be MobileNet base, and not
the original MobileNet model.
Finally, you can update the code in the train
and predict function.
Here, I've updated the epochs to be just 5,
as after some experimentation I was
able to see that it's able to converge
to a very high accuracy of 99% very quickly.
Moving further down this function's code,
you can add some extra logic to now combine the two models
that you have in memory.
To do this, you can create a new tf.sequential model
and assign it to a variable called combinedModel.
Next, use the Add function to add the MobileNet base
model first, followed by another add for the prediction head
model that you just trained.
All that's left to do now is to initialize this model
with the same optimizer and loss as you did before.
And then you can print a summary of this newly
combined and trained model.
At this point, you can also save it
to the user's hard drive using the same function
you learned about in previous chapters, as shown.
So running the code at this point
will allow you to use for transfer learning
to train a new prediction head for any object you
want to recognize.
And once trained, a new combined model
would be created and offered for download along
with the weights files that you can save to your hard drive.
You can then load that model on any other website
or send to someone else to use however they wish.
Now, after the download is complete,
the web app will then start performing live predictions
using the new trained model as before.
So you can test that it works well.
See the completed code if you're having any issues by checking
the links shown on this slide.
In the example you made on the previous slide,
you used a flow such that you train the prediction head
from the image features generated
by the truncated MobileNet model.
This keeps the truncated MobileNet models separate
when training, so its weights are not
changed when you train the new prediction head.
However, sometimes it's desired to fine tune
the various values in the last few layers of the base model,
too, after a prediction head has been trained.
This is known as fine tuning.
In this case, another approach is
to force all the layers in the base model
to not be trainable by default, by setting
their trainable properties to false,
as shown in this loop on the slide.
But also then grab a reference to the layers
that you do want to fine tune later by checking their names
and pushing the ones you want onto an array
that you can then use.
In this case, you can then combine the new prediction head
with the truncated base model and train
just for head like normal as only the head layers
will be trainable at this point.
Once a prediction head is trained,
you can go back to one of the fine tuning layers
that you saved a reference to, and set its training property
to true, as shown.
You can then perform a second round of training
to fine tune the values of that layer
without affecting the rest of the base model.
This can be useful to get a little bit of accuracy
from the combined model for the task at hand
if you're making some production quality model
and want the very best accuracy possible.
And on that note, great job.
You now know how to use layers models for transferring
too, giving you much more power in selecting just the parts you
need from any layers model and then use that
with your new prediction head.
In the next chapter, you will learn
all about model conversion, which
is how I managed to take the MobileNet v2 model from Python
and convert it into a TensorFlow.js layers model
format so you could use it in the browser.
See you there.
[MUSIC PLAYING]