JASON MAYES: So the second form of premade model
is simply just the raw trained model
itself with no JavaScript class wrapped around it
that typically hides the inner workings to make it easier
to use for beginners.
For this, you would need to write the code
to send data into the model and then extract data from it
yourself using tensors that you just learned about.
Now, individuals or researchers may share their raw models
like this, which can be very useful for finding and using
examples that may still be in development,
or use models that solve less common problems,
where an example may exist but might not
have full documentation.
So there are two types of models supported by TensorFlow.js.
The first one is the layers model,
which is easier to inspect and understand at runtime
but might run slower as they've not been optimized.
This is because they keep their high level building blocks
for ease of use and debugging, which can
help when creating new models.
Next, graph models are highly optimized
and often combine multiple operations
into one batch for very fast execution
but are not easy to understand as a human
if you try to unwrap them.
Now, the only difference between how you use them, however,
is the function you call to load them.
So just be aware that the two formats exist for
now, and you need to use the right function
to load the model when you want to use each.
Now, both model formats are stored in the same way,
consisting of at least two model files.
So let's break down the key concepts
of a saved TensorFlow.js model.
The first file is named model.json.
It contains a bunch of metadata about the model type,
its architecture, and configuration details.
The second file is one or more binary files ending
in the dot bin extension.
Typically, these will have the name shard x of n dot bin.
Now, these binary files contain all of the trained parameters
that the model has learned in order
to perform a certain task.
Now, as TensorFlow.js is designed
to work in the browser, the original, complete binary file
is broken up into these sharded files of four megabytes in size
or less.
This sharding into smaller files allows a browser
to download multiple chunks simultaneously
to speed up the page load time.
Now, all of these files are simply
hosted on a web server or content delivery
network of your choice when you need to use them,
and it should also be noted that, by default,
all of these files will be assumed
to be in the same directory.
If you want to store the binary files in a different location
for some reason, then you'll need
to edit the model.json file to show where that is.
OK, so assuming you have the location of a model.json file
hosted somewhere, how do you use TensorFlow.js to load it?
Well, looking at the TensorFlow.js API
that's available on js.tensorflow.org/api/latest,
you can see a loading subsection that specifies functions you
can call to load the graph and layers models respectively.
Essentially, both of these functions
simply take the model URL as a parameter,
which is just the location of the model.json file
that you just learned about.
Let's try this out.
Now, in order to try this for yourself,
head back to the TensorFlow.js glitch
page, as you've seen before, and find
the tensorflow-js-boilerplate example.
And remix this to make a new copy of this project
that you'll use to load a custom model.
For this example, the only file you need to edit is script.js.
Everything you see coming up will be written in this file.
OK, so in order to load a model, you need a model to load.
Thankfully, I've made a very simple model for you
that I trained on some fictional data.
This model tries to predict the price of a house given
its size in square feet.
So the input to the model is a single number representing
the size of the house, and the output
is another single number representing
the price it thinks it will be.
As this is just an example I made up,
it actually has learned to just multiply
the input by roughly 1,000, and that's
all I did with the fictitious training data.
But it will serve as a simple model
that you can use in this exercise.
Now, I've made this model available at the link
above that you can see on this slide, which you'll
use in your code shortly.
Now, before I proceed, you need to know
if you're about to load a graph model or a layers model.
Ideally, the person who created the model
would provide some example code on how to load the model.
But for this exercise, let's pretend
there's no documentation.
In this case, all you need to do is visit the model.json link
and view the content in your browser.
Once the page loads, you'll see a response
like the one you see here.
At the very start, you can see the model format.
And in this example, you can clearly
see this is using the layers model format.
So now you can move on to write the code to actually load it.
So here you can define the path of the model.json
file you want to load as a constant along
with a variable to store the loaded model once it's ready.
Next, you can create an asynchronous function called
loadModel() that will actually call for TensorFlow.js APIs.
The first line calls await tf.loadLayersModel
and is parsed for location of the model path value
that you just defined above.
As this takes time to load, this is an asynchronous call.
So you must use your wait keyword
to wait for it to finish loading before continuing execution.
Once it's loaded, however, it will be assigned to the model
variable, as you see here.
Now, once the model has loaded, you
can call the model.summary function
to see details about the model that was just loaded.
Finally, to actually execute the code in the function
you just defined, you can call load model on the last line
here to start execution of a code above.
Now, if the model loaded without any errors,
model.summary would have printed some useful information
about the model to the console.
For the example model you just loaded,
it will look something like this.
Now, you'll learn more about what some of these terms
mean later in the course when you create models for yourself.
However, there are some important things for you
to note right now.
First, as the model you just loaded was a layers model,
the summary shows you the types of layers the model uses
behind the scenes.
As you learn what these layers are later in the course,
this can help you understand other people's models
and what they're doing.
Next, it also provides details about the tensor
shapes it will output from each layer.
Take note of the last layers output
shape, as this is the shape of a tensor that
will be returned to you after running data through the model.
You can also see the total parameters contained
in the model, which gives you an idea of how many tensors
the model uses internally itself.
Diving a little deeper, here you can see that the input layer
output shape is null, 1.
The input layer is a special layer representing the inputs
and does not contain any trainable parameters,
as you see on the right.
Now, depending how the model was made,
this may not always be present in the model dot summary.
In which case, check the documentation
for details on the expected input layer shape
if it's not shown.
In this case, however, the number one
in the output shape of the input layer
shows that it expects a single number as an input.
If it had been saved in number two,
then it would expect two numbers as input
because the shape represents the length
of each axis of a tensor.
The null value on the left means a batch of any size
can be parsed as an input.
And what does that mean exactly?
Well, it's quite common in machine learning
to perform many predictions in one
go for efficiency, which is known as parsing
the inputs as a batch.
If you've got seven input values that you want classified,
instead of calling model.predict seven times,
you can pass a batch of seven inputs at the same time,
and they will be evaluated by the model concurrently
whenever possible.
Now, TensorFlow.js expects inputs in models to be batched,
even if that batch contains just one input.
After all, a batch of one is still a batch.
And in order to do that, you need
to store those values in a tensor whose rank is
one level higher than the raw data
itself, as you're essentially putting that data into an array
containing these values.
So data that could have been stored in a tensor 1D
would need to be stored in a tensor 2D
to be used as a batch.
Therefore, this example model needs
to be passed a tensor 2D object, as two axes are
specified as the input shape.
The shape of a tensor that has seven examples to classify
in this case as a batch would be 7, 1,
which is a valid shape input for this model,
as the null value indicates any number is allowed.
If you had just one value, you could pass a tensor 2D object
with the shape of 1, 1, and that would also be valid.
It should be noted, however, that if you
try to pass a tensor with an incompatible shape, like 1, 2,
you'll see a shape error on the Console, like you see here.
This is actually a very common error
and is usually due to a misunderstanding of what
the model needs as an input.
So always check the model.summary if you're unsure.
OK, let's head back to the code to actually pass data
into your model and predict an output, which
is known as inference.
Here, you'll expand your existing load model function
as previously coded.
The first line creates a batch of one
and has a single input value.
Here, you're passing the value 870 as an input,
representing a house whose size is 870 square feet.
To do this, you can call tf.tensor2d
and pass it a 2D array containing a single element,
as shown.
The next line of code is similar,
but here it shows how you can pass
three values at the same time as a batch, with many inputs
to pass through the model simultaneously.
Note that it also uses a tensor2d object,
like you saw on the previous line,
just with more example input data.
Next, for each set of examples, you
can call model.predict and pass the 2D tensor
you wish to use as an input, which will then
perform the inference.
Remember, inference is the act of running the data
through the model to get an output.
So it's this line of code that actually uses the model.
Now, model.predict returns a tensor
object from which you can then print the results that
come back to the Console using the print method on the tensor
that's returned in each case.
If you would like to get the results
in a form you can actually use in JavaScript
instead of just logging to the Console,
you can use the array method instead
that will return an array with the correct output shape, which
you can then loop through just like any regular JavaScript
array.
Finally, don't forget to dispose of the tensors
once you're finished using them.
Here, you can call the dispose method
on tensors you've created to clean up
the memory they've used once you no longer need to access them,
as shown here.
Now, it might be surprising, but you actually
created six tensors when running the prior code.
Two tensors were created when you loaded the model itself.
And if you check the model.summary output,
you'll see that the total parameters is 2,
indicating that two tensors were made internally.
So the act of loading that model will
cause it to create those tensors at runtime.
Next, two tensors were created for your inputs,
and finally, two tensors were created but were
returned to you as outputs.
Make sure you dispose of them all once you no longer need
them.
Otherwise, you might cause a memory leak
in your application, which is where the memory usage grows
to infinity and would eventually crash your computer if you
kept calling this function.
OK, so if you run the code that you just added,
you'll see the output tensors printed to the Console
as shown.
You can see, for your input of the single value of 870,
it predicts a value of around 866,000, which, as expected,
is roughly 1,000 times the size of the input,
representing the house price it thinks is the answer.
In a similar fashion, the second predict using a batch,
you can see that you get a batch of results instead.
This will be in the same order as the inputs presented,
which you can then iterate through as needed.
Great.
So you can now successfully load a model from some URL
and run it in the browser.
One thing I'd like to point out is
that it's really simple to save any model you load
so it can even work offline using the web browser's
local storage functionality.
TensorFlow.js provides a convenient function
that makes this super easy.
Just call await model.save, and pass to this save function
a local storage address you'd like
to use to retrieve it at some later point.
Next time the page loads, instead of loading the model
from a web server, you can just load
the model from local storage instead,
allowing you to use TensorFlow.js models in more
advanced implementations, like progressive web
apps that can work offline.
Now, to check if a model is saved in local storage
next time the page loads, instead of loading the model
from a web server, you can check tf.io.listModels,
as shown first.
This will return a list of models that are currently
stored locally that are usable.
You can then load the model, just like you did before,
but pass it the local storage URL instead.
Great.
So you now know the essence of how
to load and use any raw TensorFlow.js model out there.
In the next section, you'll learn
how to use model gardens like TF Hub that
have a whole selection of models all in one place for you
to load and use, along with some more advanced tensor
manipulation for dealing with models that
may do more than just predict a single number,
as you saw in this example.
I'll see you over there.
[MUSIC PLAYING]