[MUSIC PLAYING]
JOSH GORDON: Hey, everyone.
My name's Josh Gordon.
I'm here with the Developer Show at the TensorFlow Developer
Summit, and Daniel Smilkov, who is
one of the authors of TensorFlow.js.
And it's one of my favorite launches of the year.
So Daniel, thank you so much for joining us today.
DANIEL SMILKOV: Thank you.
I'm really glad you liked the talk.
So TensorFlow.js is a JavaScript library for machine learning
that can run entirely in the browser.
And you can both use your model for reference
or do training in the browser.
So you could take an existing model in Python--
there are a lot of models out there
that people have written--
and you can use one of our tools to automatically convert
that model and import it in the browser.
And then it can build creative applications and visual apps
around that.
The way TensorFlow.js started was
from a different library called deeplearn.js
that we initially released.
And deeplearn.js had a lower level API
that only had lean algebra ops.
With the launch today, deeplearn.js joining TensorFlow
family, we're adding a new layer of high level ops that
is aligned with DF layers and Keras
so people can just use high-level building
blocks and best practices to build
RNNs or convolutional neural networks.
JOSH GORDON: So for instance, if I'm a developer
and I've implemented a model already
on my desktop in Keras or TensorFlow,
or a combination of both, then I can export it?
DANIEL SMILKOV: Yes.
You can do any TensorFlow SaveModel or Keras model.
You can use our Pit package.
We just released the TensorFlow.js Pit package
in Python.
Run that script pointing to the artifact
where your model is saved.
And it will produce a different set of files
that you can then use in the browser.
JOSH GORDON: And one thing I really like about TensorFlow.js
is you have an outstanding examples
directory that demonstrates how to accomplish things like this.
DANIEL SMILKOV: That's one very cool part about the web,
is that people can easily fork these examples,
launch them in the browser, modify them, upload them
in their own servers, and share it with everyone.
JOSH GORDON: OK, so now, let's talk
about some of your great live demos.
There's the Teachable Machine, the Emoji Scavenger Hunt,
and Pac-Man.
DANIEL SMILKOV: All very cool demos.
Teachable Machine is one of our earliest demos.
It was a collaboration with Google Creative Lab folks.
It runs a neural network in the browser.
And then you can train it with your webcam
and associate it with one of the four actions.
And people love it.
JOSH GORDON: So I actually like that so much, I
taught a course last semester, and had
all my students build a Teachable
Machine as their final project.
They loved it, too.
DANIEL SMILKOV: I'm very glad to hear that.
JOSH GORDON: Yeah.
OK, so the Emoji Scavenger Hunt and Pac-Man.
DANIEL SMILKOV: Yes, so the Emoji Scavenger
Hunt is another game that our friends at Google Brand Studio
built. This one actually uses our latest tools.
So it uses the TensorFlow [? Reports ?] tutorial,
takes a pre-trained mobile net.
Then they are retraining it to identify 400 different objects
that you can have in your home.
Then they're using our Automatic Converter tool
to port it in the browser.
And they turn it into a game.
The goal of the game is you can run it on the phone,
uses the webcam.
And you point it to an object.
It gives you an emoji.
And you have to find the real object in real life,
point it, and make the model predict correctly
before the time runs out.
And I just want to point out that everything
runs in the browser.
No data is being sent to any server.
JOSH GORDON: So this is valuable for privacy-preserving
applications, as well.
DANIEL SMILKOV: Yes.
JOSH GORDON: But let me make a quick comment.
So when I think of web browser, I
think of opening Chrome on my laptop.
But I just realized, from what you're saying, of course,
I can develop a TensorFlow.js app
and also open it from a mobile browser.
So you have a last awesome demo, which is Pac-Man.
So tell me about Pac-Man.
DANIEL SMILKOV: So Pac-Man is a demo that kind of
joins the '80s game and the machine learning technologies
from 2018 together.
And it shows you how you can take a pre-trained model
and retrain it with your own private data in the browser.
So what we do is, I use my face to turn it into a controller
and teach a neural network to recognize my different poses as
left, right, up, or down.
And then I play Pac-Man.
JOSH GORDON: So you can control Pac-Man by moving your face.
DANIEL SMILKOV: Yeah.
JOSH GORDON: That's really cool.
DANIEL SMILKOV: So even though it looks like, you know,
just a game, it actually has huge implications
for accessibility.
Think of someone building a Chrome extension
that can then recognize these actions
and allow people to browse the web.
JOSH GORDON: That's a great point,
so for paralyzed patients and stuff like that.
OK, so this is awesome.
To wrap it up, where is the best place for developers
to go to get started?
DANIEL SMILKOV: Go to js.tensorflow.org.
We have a beautiful website with a lot of examples.
Each example you can easily fork, modify, share.
And also, we have tutorials.
Feel free to dive in.
JOSH GORDON: Awesome.
So Daniel, thanks so much.
For the Developer Show at the TensorFlow Dev Summit,
I'm Josh Gordon.
Thanks again, and I'll see you next time.