[Music]
so welcome to this talk on the future of
JavaScript is AI my name is my quick was
broken
my name is Aston Husain you can find me
on Twitter as draw eight not Jewish a
jaw ache I blog about angular and
JavaScript on my website code Croft TV
and I'm also a a cloud advocate at
Microsoft so if you don't know who we
are we are the open source advocates
working on the azure cloud platform if
you go to our get a link there you'll
find a you find all of us basically
there's me in the top there the world we
focus on all different technologies and
areas my focus is JavaScript and and web
well actually I've gone into leadership
now so my focus is people but yeah if
you want to find out more about us grow
into that website and just yeah chase
people up um I also work with a charity
called pal internship and what they do
is they help find Palestinians
internships in high-tech Israeli
startups and I last year I I mentored
about 10 of them it's a really hard work
mentoring ten people by mentor ten of
them and this year I'm very very proud
to say that seventeen of the interns are
all being mentored by one God advocate
for Microsoft so if any of you work with
or for or at a startup or high tech
company in Israel and you have a space
or opening for an intern give me a shout
I'll connect you in with your program
director I can get the ball rolling
anybody here
anybody here know about udemy anybody
here my students on udemy one how I
thank you thank you pays my mortgage on
udemy like this I get over a year ago
now they added in automatic subtitling
and so they correctly sites up the spell
my name correctly
my name is awesome and this talk should
be very awesome
I also co-organized a meetup group in
London called AI JavaScript London
psycho organized with my friend L and a
half prof. and we started early last
year it's been running very successful
thing got about 1700 1800 members now
and we run some pre regular sessions or
all the way through last year and what
happened was at the end of these
sessions people would come up to us and
it kind of give us a link to some hey
look at this javascript a I powered
JavaScript application isn't it cool and
we thought you know it's fantastic it's
really cool let's put them all together
in a website so everybody else can find
them and that's what we did we made a
website called a ijs rocks and if you go
to a ijs rocks what you'll find is a
whole collection of a I powered
JavaScript applications now they're
JavaScript and they're in the web which
means you can play around them but also
means you can usually view source and
see how it's working but the very least
they'll come up with or have a link to
the source code or an article or a video
explaining how it was made and the idea
is to inspire you with an application
and then if you like it you can dig into
and learn a little bit more and learn a
little bit more and that's that's been
my journey into machine learning and
jobs has been has been that process I
recommend doing that yourself and in
this talk today what we're going to do
is we're going to go through three of
the applications which you can find on a
ijs rocks I'm gonna explain how they
work
and then hopefully through that whole
process you're gonna learn throughout
the course of today for the course of
this lecture talk you're gonna learn a
little bit more a little bit more a
little bit more about what's possible
and achievable with AI machine learning
in JavaScript right now I know a couple
of talks already in this conference on
this topics
I suppose my talk if I'm a higher level
give you a bit more of a broader
viewpoint of what's possible so let's
start off with the first application
it's called the moji fire it's not
trademarked it's just the moji fire so
it made sense and this is actually my
application I built this application and
it's going to change everything
what does it do you can give an image it
will detect any faces in the image it
will then detect the emotion in the face
it will then any guesses
it will mode you fire it
that's right we'll grab the appropriate
Modi stick on the face and post it right
back at you yeah Elon Musk so nothing on
me yeah yeah I am open for investment
I hear Singapore's a start-up hotspot
you can use it may different ways so you
can use it as a slack application so in
fact if you go to the moji fire calm and
you just click that add to slack button
it'll add it to your own slack
workspaces right now and this that works
so you basically find them image this is
my son slash mode if I put the pasty
image in yeah I see it can handle
multiple people so if you grab that one
in there we go that back there am i and
I it answers an age-old question that
has been plaguing people for hunt day go
that's the Mona Lisa smile there you go
so the moji fire does quite a lot is a
whole lot of things that it's doing but
i say the most impressive thing that
it's doing is its how the hell is it
calculating emotion emotion right it's
kind of kind of complex things a lot of
us can't really tell other people's
emotion but how is a computer figuring
that out right and I've got good news
for you it's really easy it's incredibly
easy so we got two steps just two steps
okay first step detecting the facial
features we know how to do this now
we've examined in our faces we know the
different points that every single face
has and so basically there's death
different algorithms that you can get
some of them are very arithmetic and
some of them are a little bit more
machine learning based essentially
there's lots of different algorithms you
can use which can if given an image will
extract for you those facial features in
every single face has one of those
features
if I step one and step two is really
really easy you just use and you're all
networks I said just two steps extract
the features and then just train and use
and you all network let's move on I want
to explain you know just just for this
guy
gentleman I'll explain in your network
just for just for him over there I
nobody else knows what they are now
but neural network is has a basis in
biology okay this is a neuron you should
have at least one of these maybe some of
you have a few less then you did
yesterday but it has some dendrites
going in a body a body and some axon
going out if enough electricity flows
into the dendrite the body fires some
electricity out through an axon that's
it
so it does if you stick enough of these
together you get a brain I can think
right that's it that's in your network
that's awful that's a brainer to say if
I asked you to code this up in
JavaScript you probably kind of try and
give it a go I mean a long time so I've
kind of studied there the computer
science basics right now but I still
remember some of it
I've grown node ah I'd create some edges
hi graph theory yeah I know I'm talking
about get some edges going in an edge
going back out and then what your
features what are the things you're
you're pumping into would you would you
was the input some what's the output
let's say this neural network is very
one your neural network you're going to
pump in the day of the year
23 and maybe the temperature on that day
that's the inputs then for each of the
edges you basically create a random
number you initialize with a random
number where math dot R and maybe that
got initializers minus 0.5 maybe that's
got 2.1 you multiply them together you
add them up when you pump them through
an activation function again whatever
that function pumps out that's
electricity going back out that's how
you would simulate that in code that's
it just to see a little bit more about
how that might work
boom just this this is it some input
features multiply them by the weights
you sum them together whatever that
number is you pass into the activation
function and whatever that pumps out you
pump out that's just in your on it's all
you're doing when I when I teach if some
of you coming to my workshop later on at
some point of workshop will just be like
that's what that's what it's all about
right but basically there's loads
different activation functions out there
this is a very very simple one anything
below zero is zero anything above zero
is one there's loads
you can use and basically you just grab
a bunch of those neurons together
okay Network there's a simple
feed-forward neural network and so what
you do you connect them all together but
remember you initialize all those edges
with random numbers right well then for
this car neural network need a training
set let's say we've got a whole bunch of
images that we've already gathered we've
already figured out where all the facial
features are in this image okay this
data sets out that they're available
with all this stuff right we already
figured out what the facial features are
of this image I mean actually we also
know the emotion in it as well we pump
those features in maybe the features are
kind of the XY coordinates of each pupil
and other things we pump those in we
multiply all across then gives us a
number at the end right we we though
what does 3 mean that we've decided
actually that in this neural network 0
maybes unhappy in 10 is happy right
someone has a human being has looked at
that image and they said that's 8 so we
know it's wrong of course it's wrong you
initialize a bunch of random numbers and
multiply them together it's not gonna
it's not finished right it's a start in
office course it's wrong right but what
we need to do is enter just those
numbers based off of how wrong it is so
we know because we know this data that
it's off by 5 this is the magic use an
algorithm use a technique called back
propagation ok given how wrong the
network is how do you tweak those
numbers so next time you run it it's
more right
ok that's back propagation and that's
what you do
and so then it tweaks it perhaps you
keep on running it with your date and
you keep on running with it until it
gets less wrong less wrong less wrong
less wrong and so eventually what
happens is you give an eighth image and
if we correctly predicts that it's eight
okay
that process is training a neural
network all it's doing is tweaking those
weights so the next time you go through
it becomes better and better and better
okay that's how you could do it there
are actually well-known datasets out
there that you can get the freely
available which have a whole bunch of
images 10,000 20,000 30,000 a hundred I
mean which loads of images each image
has been a human being is gone through
and determine what emotion is and you
can build and train your network in
order to detect emotion okay so you
could do okay but a lot what's been
happening over the last two years one of
those lots of things happening in space
over the last years once we've really
been happening in the last two years is
a commoditization
of these services especially it's very
very commonly used models a lot of
different companies Microsoft included
have been making them available via
api's and that's what I use that's what
I use for this service I use something
called the face api which can get from
Azure and it's very very very easy to
use you basically just post an image to
that URL and it gives you some data back
at the backend it's using something
exactly like I've showed you it's been
trained in a much much bigger data set
over much much much longer at a time but
it gives you a whole bunch of
information for each image in the face
image in the for each face in the image
bla it gives you a face rectangle the
face attributes with emotion gets a
whole lot more as well just strips all
out
anger contempt disgust fear happiness
teacher sadness and surprise between
zero and one do you see the odd beard we
have we got bed
yes yeah you cannot be hunter person
happy really sorry about that it's
actually impossible
I've tried I can only be 99% happy
that's basically I used in the moji fire
right
that's how I built it that's how I
detected in motion inside there just to
summarize your networks just incredibly
powerful that incredibly powerful but
actually conceptually they're really
simple to understand I explained you a
really basic one in about five five ten
minutes something like that right okay
basic the other kind of lesson learned
here might be when I do teach my
workshops to get people ask me questions
afterwards
so this taught them how to do training
for neural network from scratch then
they asked me I want to build this and
I'm like oh there's 18 API is available
will you do that right so first off also
especially these days check to see
there's an API that already gives you
the answer that you're looking for next
demo okay and I'm fine so I run
workshops on machine learning with
JavaScript in one of the workshops in
London a guy called Oliver Turner then
built this demo and their center back
seam proud moment when your workshop on
your students build something sends it
back and this is what it does basically
if you do yeah it's doing a search for
unsplash for puppies getting that image
back there then the percentage in the
bottom left are it trying to figure out
what's inside the image you can see here
it's detectable here's like it fountain
mmm
puppies ten percent a terrier bad you
search unsplash for I'm fine again
saying this image and it says
seventy-eight percent t-shirt not bad
okay not bad so the important lesson
here is that the only API request is
being made here is to get image the
actual detection of what's inside the
image is all done in javascript in the
browser okay we heard
I know I know we've heard of tensorflow
already in this in this were these
things called conferences in this
conference yeah I arrived last night
from London and think about well I think
yeah tensorflow is a tent tensorflow
since low is a technology which is open
just like Google for five years ago a
number and it allows you to run what a
tensor is there's actually a
multi-dimensional array that's all
attempts that means a high dimensional
array so you can do kind of matrix
calculations across whole bunch of
servers GPUs scale it out massively
right Brits built-in see it's really
hardcore yeah but that's basically where
it's all about but start of last year
the creative tempo a yes we all like
that here don't we
a little dot J s whenever they add that
to things we get really excited it's
like butter you know plain jacket potato
butter
it's nice right that's what Jay s was
launched now it just a month or two ago
icon with the last tense for some it may
reach 1.0 so house actually reached a
stay at maturity now today I know we
hope we've gone through this before but
this is all you need to do in order to
use 10 so jess is well if you mean you
use this one but that's but the
important thing here is that's all you
need in order to start playing around
with machine learning jobs this is it
when I first thought Enzo J's I thought
it was like node bindings and you had to
install tensorflow not to use it no it's
actually really tense we rerun from the
ground up in JavaScript there's like a
subset of what's available in the proper
turn slope it has enough to do some pre
some pretty cool stuff
and you can do a whole bunch of things
that you can do what I just described to
you earlier on which is a trainer model
the whole thing with the newer
and then the weight and then the
backpropagation all that tuning stuff
you can do that intense flow Jess you
can write oh you can load pre train
models doing pre train models if you go
to ten so jess website you actually find
now there's like the big it's growing
there's a hot there's a set of pre train
models very easy preacher and models you
can just grab from their website and
start using you can also there's also
now scripts available where you can take
a model chained up using tensorflow
proper and then convert those models for
use in pencil for chair so you can start
using them but essentially as long as
good as adjacent file available you can
load it up from within the browser and
that's kind of what all of the use so
use something called mobile net which is
just a very optimized neural network
that can detect one of a thousand things
only a thousand things that's why it
wasn't very good there's only a thousand
things that can detect in the world but
it's mobile so it's been optimized for
size so it can be small to use on a
mobile but that's it only like four
lines of code I say it's crazy four
lines of code in JavaScript to do that
right I'm gonna I'm gonna I'm gonna get
this wrong statistically but I think it
was 2008 was it we still couldn't detect
whether an image was a cat or a dog
right using whatever servers that we
could now in JavaScript we can do this a
bit crazy but it's mobile net right so
it's it's not very good you saw it
wasn't very very good it wasn't that
good right it's useful for a lot of
other things it's useful something
called transfer learning it's useful for
some use cases but not to really know
what's in an image like if your life
depended on it you wouldn't use mobile
net right
you want to use somebody's been trained
up on a much larger data set with much
more features and these things are kind
of gigabytes terabytes big and not not
big enough to download on your computer
but they're available via
I'm gonna pretend to me said api's you
said api's right yeah API is correct I
mean we've got another one called
computer vision I knew you can you can
detect like an insane amount of
information if you give an image one of
the really scary things it does it
shocks me it still scares me actually is
it you can give an image and they can
give you a human readable description of
what's in the image as if a human being
wrote it right my friend Sara Rosen have
made a cool demo with this she thought
wouldn't it be cool if you could use
that to describe the alt text the alt
tag in images because we all do that
right we're all supposed to add alt tags
for images right we're all doing it so
this is completely unnecessary I know
but if you did want to use it you could
try out it's a code pen so you can you
can put in an image and the text at the
top let me know this is TV show no wow
you're in for a treat
just halt and catch fire yeah it's an
amazing TV show it talks to talks about
start something kind of 1985 and talks
about computing back in those days and
she's one of the stars in the show is
Mackenzie Davies this is a when you give
her this image this is the this is the
the caption it gives it was a Mackenzie
Davies and all standing in front of a
building right pretty pretty good for
service but you know she released on
Twitter and the twitter is full of
really helpful people you know all were
very supportive we kindly tell you when
something some work right and I saw some
people said so they said oh this doesn't
work so a star-filled sky takes days for
actually don't know maybe the Asian I
don't know what they were characters but
what it is it's not accurate that's true
no the next one I think is uh
be true could be true no I mean you
don't know dude let's go be dead cats
stuffed on a bed really good ones
I really like the next one because cuz
we're half right
we're half right on the next one yeah
that's good that's 50% that's past mark
where I come from so just to summarize
and that's probably one takeaway to take
from this just tensorflow Jess doesn't
have any dependencies they're gonna play
around machine learning each one's do
quickly you're suffocating the browser
confusing node well actually I take that
back
tend to flow chess in though does have
dependencies but then slow Jess in the
browser you can just stop playing around
with it with nothing mobile there's a
really simple way to analyze images and
it's usually something if you come to
the workshop later on is how we start
off always gonna start off with mobile
net but if you really wanna do some kind
of really deep image analysis you want
to use an API or you want to spend a
couple hundred thousand dollars training
one up or something like that right time
for the last one okay
it's called image to image you find this
on the air Jess rocks website let's just
take a look at actually
this is running in the browser okay
you would outline of a cat and it draws
the rest of the cats go by a guy called
the Zaid he's still at university I
wasn't doing stuff University yeah
amazing work right how does it work
he's an algorithm called pics to pics
there was an example of something call
again
a generative adversarial neural network
and what they how they work is it's to
neural networks that are competing with
each other okay so first off you have a
generator okay and you have another one
called a discriminator the generators
job is given some inputs whatever input
that is in this case it's and outlines
of images of outlines of cats given that
input its job it's a generate an image
of a cat that's this job right and
that's what you do you you get in a
whole set of import outline images you
get a generator to generate outputs it's
not going to do a really good job right
now cuz it's all math dot random right
it's not it's all gonna be noise to
begin with but whatever you take all
those images and you combine them with
pictures of real cats you jumble all
together and you pass it into the scrim
into the discriminators job is to figure
out if an image is of a real cat or of a
generated cat remember the discriminator
has also been initialized with math dot
random so it's not going to do a very
good job as well is it right probably
you might get it wrong the first time
around you might think the real cat is a
fake cat
but then what do you do when it went a
neural network is wrong we train I'll
take it train it we do that
backpropagation thing we tweak it we
tune it we tuned until it gets better
okay but if it got it right it detected
it right that means it generators not
doing a very good job the generator is
wrong we tuned the generator and that's
what's going on as a generator gets
better the discriminating s better as a
generator gets better discriminate and
the fighting versus each other that's
what they're doing until eventually the
generator is doing such a good job
generating cat images the discriminator
can't figure out what's real and what's
what's fake anymore right and then you
know you're done you know trainings
finished and what we do is you throw
away the discriminator we don't need
that anymore we just need a generator
and you can export that if you're
careful you can export it into a small
enough size you can load up in the
browser and that's what they did and
then you can give it your own outlines
and they can generate images of cats
okay that's a gang but you don't just
have to use it with cats works with
anything right if you train it with the
right data set
oh this so your left is the input and
the right is the output okay but this
isn't pix to pics this bit of it
this doesn't work in JavaScript yet but
maybe not so far away um how about this
example one input multiple outputs
remember it doesn't it doesn't just have
to take out lines as inputs you can give
again whatever you want as an input
whatever you want right in this case you
can give it what's called a segmented
image which is just an outline d the
different colors kind of give you a clue
as a the 3d nature of the image and the
depth of the image and you can do
something like this okay she's not
dancing I wish I could get a real-life
one of one of these for me yeah how
about this one there's a segmented
Street image top-left these two are just
different algorithms and that's the
width of it one there again all right
it's generated completely generated
but you don't have to even give it even
an image you don't even have to give an
image you can just give a text as input
okay remember it's just whatever text
you give it's going to generate
something so you give it some text as
input and you also generated generate an
image that's what you doing so these are
two examples here so in the top one the
input text is this flower has long thin
yellow petals and a lot of yellow
anthers in the center stage one is after
six hundred iterations Stage two is
after twelve hundred iterations you can
see well this one's not very good but
the rest are pretty good right and this
is another one using birds so this bird
is white black and brown in color with a
brown beak again stage one is uh 600 not
the best but stage two is very good I
think it's one here it's not very good
maybe so that's it just to summarize
yeah
ganz learn to generate new images okay
they they generate they generate things
that's what's so exciting about them to
cover the creative side of machine
learning right they generate things
discriminators that's why we use mostly
used machine learning you just to tell
us what something is or isn't right but
generators can generate create that's
why so exciting about oh but they take a
lot of computer train a lot more than a
normal Network but the generator can be
exported and run in the browser I think
I'm over time I think if you want to
learn a little bit more about Ted's flow
Jess I am considering writing a book so
you take a screenshot of this if I get a
hundred signups I think I'm about ninety
right now such the tenth
I will start a Kickstarter or something
to decieve see fun writing it if you
want to learn a little bit more about
machine learning especially want to use
some api's I have a module if you go
there it's a tutorial step by step will
shows you how to build something that
emoji fire I have a slack box you're
gonna learn more kind of simple API
route and go that way and thank you for
mystery time thank you
[Applause]