[MUSIC PLAYING]
JASON MAYES: So let's talk about machine learning in JavaScript
now that you know what this is all about.
Currently, over 68% of professional developers
use JavaScript in production, and we
have over 51% of developers using Node.js
as a primary framework in production, too.
Now, as you may have guessed, TensorFlow.js
is an open-source library for machine learning in JavaScript
written by Google, and it actually started its life
when Google researchers needed a way to visualize and use
ML models in the browser, which led
to the release of deeplearn.js Back in 2017
around one year after TensorFlow itself was born.
This first incarnation was very low-level,
mathematical library that allowed researchers
to demonstrate simple models live in the browser,
mostly for educational purposes, and you really
needed to be an ML expert to use it at this time.
It lacked the high-level features
that ML developers were used to working with that made creating
new models more manageable, but its initial success
proved there was a desire to use ML in JavaScript.
As such, the need for production quality
js library that was aligned with the original vision
of TensorFlow emerged, which led to the birth of TensorFlow.js
in 2018.
This version of a library added support for high-level APIs
that mimicked the popular Keras API for TensorFlow,
making ML far more accessible in JavaScript
from existing ML practitioners.
This, in turn, lead to a wave of ML researchers,
porting cutting-edge models to the JavaScript ecosystem,
and now TensorFlow.js is used by all sizes of company,
from startups to large multinationals,
and even individuals, from academics
to hobbyists, who all embrace the unique benefits of using
TensorFlow.js.
In fact, using TensorFlow.js means
developers can run machine learning anywhere
that JavaScript can run, which is basically everywhere.
This equates to zero install on billions of devices globally.
You can just follow a link, and it'll
work on almost any device.
Here are all the environments we currently support--
common web browsers, server-side via Node.js,
mobile native via React Native or Progressive Web Apps,
desktop native via Electron, and even
IoT on devices like a Raspberry Pi via Node.js.
Running client side on device is just one advantage
of using TensorFlow.js, and this is
very different from using TensorFlow Python, which can
only run on the server side.
This, however, brings a unique set of challenges
that you must consider, such as the types of devices
that your model might run on.
With server-side-based ML, you have a fixed hardware setup
that is known in advance along with the expected
performance of a model on that specific machine.
When you run models on the client side on device,
the hardware can change from user
to user, which will affect the speed it can run
and, of course, execution time of the model.
Clearly, a five-year-old smartphone with a weak graphics
processor will most likely run a given model
slower than a current-generation phone
if you try and execute the model on the graphics
processor of such a device.
So let's take a moment to focus on how TensorFlow.js
is structured to understand better how you can get the best
performance from your models and clear up
some of the terminology around the library itself.
TensorFlow.js has two APIs you can work with.
The first is our high-level Layers API
that's actually very similar to Keras, which
for those of you new to the TensorFlow ecosystem,
is essentially an API that was made
available in the Python form of TensorFlow that allows
you to work at a much higher level when making custom
ML models.
So for those of you who have prior experience with this
in Python, you would feel very comfortable
using this TensorFlow.js API.
And for those of you who are new to all of this,
there's no need to worry as you'll
be focusing on using the TensorFlow.js
APIs in this course, which we'll be teaching you later.
Next, you have our low-level Ops API,
which is more mathematical in nature that originates
from our deeplearn.js beginnings. [? It ?] enables
you to go lower level, allowing you
to do things like linear algebra to build pretty much anything
you might want.
And typically, researchers will work at this level,
creating new features for cutting-edge machine learning
research.
In this introductory course, you'll
spend most of your time working at the Layers level
when making custom models later on in the course.
So here, you can see how it all comes together.
At the top you have our pre-made models.
These are built upon our Layers API, which itself
sits on top of the Ops API.
Now, this then understands how to work
within many different environments,
such as the client side, which includes things like the web
browser, for example, and each one of these environments
can execute on a number of different backends.
Now, by backend here, I do not mean server side.
Backend, in this context, refers to the hardware
on which it will be executed.
So let's dive into backend in a little more detail.
On the left, you have the CPU, which
is always available, essentially vanilla javascript,
but it's the slowest form of execution, running
on the computer's processor.
Now, you also have WebAssembly for faster CPU execution
with support for SIMD and multithreading commands,
which allow smaller models to run extremely fast on the CPU,
sometimes even matching GPU performance,
that is, on the graphics card, when running
client side in the browser.
You then have WebGL that can leverage a graphics
card, or GPU, if you will, which supports 97.6% of devices
out there.
And yet that means you can run on more than just NVIDIA GPUs.
You can run on AMD and Intel, too,
so that means you can get great performance even
on something like a MacBook that might not
have an Nvidia graphics card.
And looking towards the future, we're
seeing new web standards being formed namely
around WebNN and Web GPU, which are also
investigating to accelerate performance even further.
And there's a similar story for server-side environments, which
is provided by a Node.js version of TensorFlow.
For those of you new to Node.js, you
can think of it as a special version of JavaScript that's
designed to run on the server side
and has tighter integration with the operating system instead
of the browser.
Note here that our Node.js environment
supports the same TensorFlow CPU and GPU bindings as Python has.
In fact, both the Python and Node versions of TensorFlow are
simply just wrappers around a C++ core,
but server-side TensorFlow is written in behind the scenes.
This allows you to get the same or better performance
from Python as you, too, can leverage
[? Fukuda ?] and [INAUDIBLE] acceleration that's typically
associated with the Python version of TensorFlow.
But as JavaScript developers, you can also
leverage the just-in-time compilation features
of JavaScript in Node, which can lead to some great performance
gains over Python when running on the server side,
and we'll talk more about that later.
Now, some folks you work with may
prefer to use Python for the machine-learning research.
That, of course, is completely fine,
and TensorFlow.js Node supports the ingestion
of Keras and TensorFlow SavedModels
directly within Node without any sort of conversion.
And this is great [INAUDIBLE] Python developers
that directly integrate with web teams
but are most likely going to be using Node.js
as their preferred server-side framework of choice.
Now, if you wish to execute Python models in the web
browser, however, you can use our command line converter
to convert to the format needed to run in the web browser.
We'll cover this in more detail later in the course.
So what's the performance like?
As you can see here, executing a model
named MobileNet V2, which is basically
an image-recognition model, on both Python
and node with GPU acceleration leads to less than 1
millisecond of difference for raw execution time.
However, if you've got a lot of preprocessing, which
is the act of converting data into a form
that the model can use as an input or [? postprocessing, ?]
which is the act of taking the output of a model
and transforming it to something that's useful that you can use
in your app, then Node.js can leverage
for just-in-time compiler of JavaScript
to see a significant boost in speed over Python.
And here, you can see, you have a company
Hugging Face, who are very well known
for their natural-language processing models,
converted their model known as DistilBERT into Node.js,
and this resulted into a two times
performance boost over the Python equivalent.
So if performance is top of mind,
you might want to give Node.js to try, especially if your end
goal is to expose a server-side web API for [? Inference, ?]
for which node is very well suited.
In summary, here are the benefits
to using Node.js on the server side.
First, you can use the TensorFlow SavedModel format
without conversion.
This, in turn, may enable you to run larger models
[? when ?] you can do on the client side.
In some situations, it might not make sense
to transfer a large motherboard to the client
side in the browser if it's gigabytes in size.
Next, it allows you to code in just one language,
and this is great for developers already using JavaScript.
Currently, 68% of developers do use js,
and that can enable code reuse across the stack, which,
for a small startup, can be very beneficial
as your existing JavaScript developers can also
deploy server-side ML models for you,
too, enabling you greater use cases for your business.
And then we've got performance.
As mentioned, our Node.js implementation talks
to the same C++ bindings behind the scenes as Python does,
so you'll get the same server-side hardware
acceleration for both the CPU and the GPU.
However, as mentioned, due to the just-in-time compiler
in JavaScript, if your model requires [? pre- ?]
or postprocessing of data, you can get performance boost
in Node for that for free.
And finally, as this course is primarily
about machine learning in the browser,
let's talk about client-side superpowers
that can only be achieved by running in the web browser.
First is privacy.
As the model runs entirely on the client machine,
no data is ever sent to a third-party server,
maintaining data privacy for the end user, which is particularly
important for industries where it might be a requirement not
to transfer data to a third party, not
to mention growing concerns around privacy these days.
And here, you get the privacy for free with TensorFlow.js.
Second is lower latency.
As JavaScript has direct access to the device sensors,
such as the microphone, camera, accelerometer, and more,
there's no round-trip time to the server
to analyze that data.
Now, latency to the server could be
close to 100 milliseconds on, say, a mobile connection,
and assuming zero latency for using
the machine-learning model itself,
the maximum frames per second would cap out
at about 10 frames per second if you're
going to send images one by one, which is less than ideal.
Now, with TensorFlow.js running on a device,
you can go much faster than that.
Next is cost.
If there's no data sent to the server,
you'll have less bandwidth and hardware server
costs as there's no server side CPU, GPU,
and RAM that are needed to be hired for inference.
This means you just have to pay for hosting of the website
assets and model files, which is far cheaper than running
a dedicated ML server all of the time.
Or what about interactivity?
Web tech was designed for the display of content agnostic
to device type.
From day 1, it supported graphical content
and has evolved to handle even richer formats, like video,
Webex, R, WebGL, and more.
As such, it has far more mature libraries
for graphics and data [? vids ?] versus other languages,
such as Three.js or dv.js, allowing
you to code your ideas in hours instead of days or weeks.
Finally, you've got the reach and scale of the web.
Anyone anywhere can link and load a web page,
and that machine learning will just work.
Even zero installer is required to do this.
And of course, you get more eyes on your cutting-edge research
and [INAUDIBLE] used in other ways across industries.
Now, let's head on to the next video
to learn about the three ways you
can use or create TensorFlow.js models using this library.
[MUSIC PLAYING]