Make JavaScript great again. Hello world it's Suraj and google recently announced the much-anticipated
javascript version of its popular machine learning framework tensorflow
Dubbing it tensorflow.js in this video
We'll use it to build a very simple web app that can detect any object you show your webcam
Last year the Google brain team developed a library called deeplearn.js
Is that was an impressive contribution
To the field allowing developers to build machine learning models in the browser and it had a few impressive demos
Associated with it like one that let any user train a model in real time for image classification
But a lot of the code was deprecated and buggy so an upgrade was in order
They made it faster made the syntax more readable and added lots of new low-level
Functions to it that let developers build very detailed models in the browser
eventually renaming the project to tensorflow.js if we build an AI app
using tensorflow.js
it makes predictions and trains itself entirely client-side that means it can use the GPU of
whatever user
Accesses the app and it doesn't have to be an NVIDIA GPU Jensen
You've badass it can be any kind of GPU from a MacBook GPU to an Android phone GPU
Usually to consume ML
models whether as a developer
Or as an end user we have to install a few packages
Or deal with some sort of technical issue
But because tensorflow.js works in the browser
We don't have to install any dependencies the library is split up into two different
Packages the first is core. Which is a flexible low-level API
It's syntactically pretty similar to the tensorflow Python library
Then there's the layers API which is similar to care offs. It's higher
Level and makes it easier to build models at the expense of fine-grained control over the details
There's a lot of potential for using
client-side data to help train models
It really opens up a whole new world of machine learning one where we can start thinking more about how?
Server-sent events can be used to feed rich data into these models in real-time
While operating in the browser we can create apps that
Continuously learn and even if users have only a small amount of data to give we can use the libraries ability to perform
Transfer learning to augment an existing models capability with new data
We can even load model files trained in tensorflow Python or C++
Directly to the browser and because the data never leaves the client
It's privacy friendly we can train our models on data client-side without ever seeing the actual data itself
There's a lot of potential to use tensorflow.js to create web apps that can learn from their users
You can crowdsource the training of a music generating AI by having the users make their own songs in the browser
Have users submit articles and create an AI that learns to write in the same way they do or
Even learn from users browsing habits to present them with content. That's most suitable for them
When it comes to installing tensorflow.js we don't even have to do anything if we create a simple
HTML file we can access it by simply adding a script tag to our code that points to the library online
Then we can use the library right under that as a test
We can write a basic HTML file that simply prints out a tensor and in our JavaScript console in the browser
We can see that it works. No dependencies
No other scripts to run too legit to quit let's look at some of the core components of the library the reason it's called
Tensorflow.js is because deep learning models are essentially a graph of computations
That is we have some input data
This could be an image or a piece of text all of this could be represented by a series of numbers
We feed that input data into a computing graph at each step. We perform a series of
mathematical operations on that input data slowly
Transforming it and we call the data that flows
through the graph tensors since a tensor is a group of numbers in n
Dimensions that can be one dimension or a hundred dimensions the tensors flow through the graph
eventually producing an output tensor
Which is the prediction that it makes in?
Tensorflow.js the primary unit of data is called the tensor which we can access very easily
it's a self named a
Tensor instance has a shape attribute that defines the array shape
It's found in the tf.tensor function we can create tensors of any size
We just have to define how many dimensions
We'd like as well as the values that make up the tensor if we want to get more specific
And make the code more readable
We can use other functions like TF.tensor 1D or TF.tensor 2D that
Specify that the tensor in question is of one or two dimensions respectively
Tensors are also immutable meaning once. They're created. We cannot change their value instead we can perform
operations on them generating new tensors
But there should be some mutability when building a machine learning model, right?
After all the model must update itself as it learns. That's where variables come into play
Variables are initialized with a tensor of values
But unlike tensors their values are mutable we can assign a new tensor to an existing variable
Using the assigned method they're used to store and update values during model training
But remember that tensors are only half of the equation here we need to perform operations on
Them for this to be machine learning / for us to be cool
Operations or ops for short allow us to manipulate the tensors
The library offers a huge range of ops that allow us to perform fast matrix math on tensors easily
Performing calculations on groups of numbers in parallel and because it's a chainable api
We can call ops on the result of ops on the result of ops on the result of
You get the point, so what does building a model actually look like in tensorflow.js?
We've got two options. The first is to use ops directly to represent the work the model does
This is the low-level part of the API we can define in detail every add
multiplication subtraction
Operation that's applied to our tensors in what order how often et cetera?
Alternatively if we want to quickly prototype a model and don't care too much about the details
We can use the high-level API using TF.model to construct a model out of layers
These are a popular abstraction in deep learning adapted from the chaos library in that a neural network
consists of layers of
operations and while sometimes a layer needs to be written out as multiple lines of code a
high-level version of this allows the developer to define a layer as a single line of code
Specifying the necessary parameters right there
All right, all right all right
Let's get to our demo of having an AI
Predict what type of object is on our web cam we'll use a popular machine learning model called Yolo
Or you only look once the way it works
Is that it first divides a given image up into a grid of 13 by 13
cells each of these cells is responsible for predicting 5 bounding boxes a
Bounding box describes the rectangle that encloses the object
The model will output a confidence score that tells us how certain that the predicted bounding box
Actually enclose is some object
Once it does that it'll look something like this for each bounding box the cell also
predicts a class as in what it thinks that object is it gives a
probability distribution over all the possible classes
And it does this after having been trained on a data set that contains a set of labeled image
classes the confidence score for the bounding box and the class prediction are
Combined into one final score that tells us the probability that this bounding box
Contains a specific type of object since most of the boxes will have a very low confidence scores
We only keep the boxes whose final score is 30% or more
Which leaves us with a final prediction the architecture of Yolo is pretty simple
It's just a convolutional neural network
I have an incredibly detailed video on comm nets on my channel see the link in the video description for more
But it's got. It's got just a couple of these layers repeated over and over and over again
We give the network an input image and in a single pass
It'll output a tensor that describes the bounding boxes for the grid cells
Then all we'll need to do is compute the final scores
For the bounding boxes and throw away the ones that aren't above our threshold
But since we don't know how fast a user's GPU will be we'll use a tiny version of this model called tiny
Yolo, which uses fewer layers and is faster
But also a bit less accurate we can import this model into tensorflow.js
easily then access our webcam
programmatically to retrieve the image frame we can apply each image to our Yolo model in real time and
Have the bounding box layered on top of the image for us to see since the model is already trained
We don't need to train it. We're just repurposing it for use in the browser now
It'll tell me whatever it is that I'm putting in front of webcam TV three things to keep in mind from this video
Tensorflow das is the JavaScript version of Google's popular machine learning library?
consisting of both a low-level core API and a high-level layers API for building and training models
It uses the user's GPU no matter. What kind for both training and
Inference which opens up a whole new world of possibility including training on real-time data
And we can port existing ML. Models to the JS framework and
repurpose them for browser use pretty easily if
You want to learn more hit the subscribe button and for now I've got to use tensorflow.js so thanks for watching