alright hey everybody my name is Melissa
McGregor and I'm a senior UI engineer
right now but in a past life I used to
be a mechanical and aerospace engineer
which is how we get to using machine
learning to improve UI UX roundabout
ways so the goal of this is to make our
Skynet future more user-friendly so I
like to give a quick overview of what
I'm gonna talk about before we get into
it so first how many of y'all know about
neural networks like how they work
oh yes ok how many people have heard of
brain jeaious that's more than I've seen
ever so great it's Charlie here oh yes
no anyways you'll get to see why I ask
later
so then we'll get into more of like how
you actually do machine learning so
we'll talk about choosing features to
use how you actually train a model to
understand our wonky user behavior and
how you actually use a machine learning
model to make updates to your DOM and at
the end of course there's some little
takeaways so just to get you kind of
warmed up and started with neural
networks a neural network is basically a
math equation that makes predictions you
put in stuff you get out stuff so one
thing that I think personally machine
learning has a really bad I guess thing
with is jargon so I'm hoping to break
some of that for you guys so a neural
network is basically just a brain made
of layers of nodes and this is what I
know it looks like so you have as many
inputs as you want you assign those
inputs a certain weight to give them
like I'd say so you give them a weight
so that they know which
input has more value to the prediction
you're trying to make and then you run
it through some crazy math equation that
a lot of people don't really want to
know about and then you get a predictive
value or values so another thing that
comes up a lot in machine learning is
deep learning hmm it's so scary and
fancy but the only thing that deep
learning means is that there are more
than one node in your neural network so
you can have two nodes and you have a
deep learning model oh you can have
three nodes and it's super deep so some
of the places that you might see neural
networks in the wild would be with like
predicting user intent so your user
clicks on a button or a certain part of
the screen and you have certain content
over there that means that that user is
probably interested in whatever you're
showing them that they keep hovering
over or clicking on and based on that
fact you can make adjustments to what
that content is you can change the
jargon or the language around it you can
even throw in memes if you want to just
based on the inputs that you get from
your user and something else you can do
to help our Skynet future is to make
personalized interfaces so let's say
that you have this large deadest data
set of user behaviors and it shows you
that on Tuesdays at 3:05 p.m. all of
your users are drinking mimosas so you
might want to throw in something that's
maybe orange or something that might not
have slipped through they're more sober
minds and then of course you can always
use machine learning to find areas of
improvement so when you get all of your
user data you can take that and look at
some of the problem areas they have so
maybe you're trying to increase
conversion
or you're trying to get more people to
come to your website or you just want
more people spend money on your website
whatever it is you can take your user
behavior data and skim it for any type
of behavior that you're trying to make
them do so basically neural networks in
the wild are a way to kind of implant
little ticks into our users so that they
don't have to worry about as much it
just does stuff for them and that's what
I'll users want and that's what we want
so that they don't break our stuff so
gonna talk about brain jeaious for a
little bit it is another JavaScript
machine learning library and I
deliberately chose the silver tensorflow
a while back when I tried to learn
tensorflow jeaious it confused the
absolute crap out of me so I found brain
juice and it's so much easier to use how
many people know how to work with
objects arrays objects with arrays and
arrays of objects then you can do
machine learning so one of the best
things about brains is if you aren't as
interested in getting into like the
mathematical stuff or you don't really
want to dig too deep into machine
learning you just want to say hey I have
this data and I need a prediction this
is a really good library to do that with
just because objects and arrays are
really easy to work with and again the
tutorials are really straightforward
they get you up and running pretty much
from square one so you can start using
sequential models you can use some RN
ends is just all built-in for you and I
don't have anything to do with brain J s
I'm just a little bit of a fangirl for
it so just throw that out there
so we'll get into the best part choosing
your features so features are basically
the data points that you're trying to
collect from users or from whatever
other hardware systems you have but for
this talk in particular we're focused on
our UI UX so one of the biggest
questions you have to ask is what is
gonna add the most value for a user I
know a lot of us find these new tech
tools and we're like oh my gosh this is
so cool and then we start throwing stuff
into the application that really doesn't
do anything for the user so make sure
that when you're looking for those data
points you're trying to get from your
user that the prediction you get from
that data will add value to the user
then you have to figure out how are you
gonna get that information are you gonna
hook them up to some kind of EEG sensor
and monitor brain waves are you gonna
hack into their webcam and watch them do
whatever people do or are you just going
to get like some Mouse inputs or some
information from input boxes probably
the second one hopefully and then
another thing that a lot of machine
learning projects lose sight of at some
point is what exactly are you trying to
predict so you have all of this data you
have terabytes of data and you're like
yes machine learning we need to use it
okay what are you trying to do with it
so make sure that you actually know what
you're trying to predict before you just
dive into data analysis and then this is
my favorite will anybody care if you
make this update so yeah it's cool to
use machine learning and stuff yeah it
has this applications but at the end of
the day if your users don't care was it
really a successful project just
something there
ponder about so this is the fun part
we're going to get into actually
training a model to understand user
behavior and I have code and a demo that
is supposed to be running right now it
worked a few minutes ago which is why
this is commented out instead of deleted
okay can you guys see that okay
[Music]
and we can ignore the comment for now
but anyways we're gonna go into the
backend so this is one of the ways you
can create a machine learning model
using nodejs with the brain jeaious
library and basically what we do the
main things I mean how many of you are
back-end engineers or full-stack okay
everybody knows about node and servers
then so pretty much the only thing you
really need to do is import this brain
j/s library and blah blah blah okay so
now this is the fun part so this is our
training data for the neural network
model and it's very small because I made
up all of these values you know demo
data set but basically what we're doing
is we are going to make this UI
interface that it kind of bothers people
who might be trying to hack our website
so based on what they type into our
input box there are certain things that
a normal user wouldn't like I don't
expect a user to type in something like
having or group or select especially in
any type of combination so based on
these values which you see it's an array
of objects Oh
and down here that is our input data so
this is what we are saying that hey if
you see any of these inputs we are going
to use them to predict if our our user
is malicious or not so basically if they
use any of these words or terms it
assigns a maliciousness rating and that
is our output data so the way the
machine learning model works is that you
have to have a training input data set
and a training output data set because
machines aren't people they don't really
feel in the blanks yet so what we're
doing is called supervised machine
learning and that's just where you tell
it hey if you see this combination of
things you can expect it to mean this
and that's how we teach our model to
make predictions based off of inputs so
basically we're going to use these two
data sets to train our model which is
incredibly easy so little little jenkin
is here again I wanted to make sure this
actually worked this morning so what
we're doing here is just combining the
input and output data into a single I
think this is an object object or array
whatever we combine it into a single
entity and we do that because that's how
the brain Jas library works and tada
this is how you make a neural network
this one line right here you have a
neural network and basically we just
kept it really simple and went with the
neural network so there's like recursive
neural networks there's some other crazy
things you can use but this is easy and
we gave it three hidden layers what that
means is that basically we give it the
inputs those inputs have weights and
then the model takes over and it's like
this combination of inputs equals this
and it just pushes it through to the
prediction so hidden layers are good
when you
more accuracy but remember that you
don't want too many because it'll slow
your results down it'll it'll just be
really really slow you don't want that
when you run in a server that people are
hitting you know millions of times a day
but this is the best part so you see
this this is how you train a neural
network one function call and we have
our model made that is it in brain J is
easy so we have our model made and now
all we have to do is make predictions
and the way we're going to do that is
we're going to use our model to make
updates to the Dom based on some of the
inputs that our users put in again deep
breath it's demo time so hopefully this
will work
first though how many of y'all have
heard of slip 'some there's a couple
people but okay it's Lauren it's my wife
I disconnected but you should definitely
google slip 'some I don't know why my
wife I disconnected it but such is life
when AV stuff happens but just to show
you kind of what it would look like if
you start typing in words you kind of
notice the colors change and you see my
maliciousness rating is definitely
pretty up there you really don't want
people to be able to type this in so
you'll probably have some kind of
security blocks on the back end and
that's cool but why not start messing up
the user interface for somebody trying
to type it in said it comes from that
principle have you guys seen the like
bootlegged video games where they start
really messing up the deeper you get
into it's basically the same concept you
just make it difficult for them to keep
trying to hack you even though you
already have all of you back in secure
in your database and nobody's using
admin admin so you're good to go
but the way that we do this is on the
front end we have a very small piece of
code somewhere in here
yes here we go so basically what we're
doing here is we get the input from the
user we send it to our back end it goes
through our machine learning model right
here and what it's looking for is it's
going to split up all of the words that
the users typed into the box and it's
going to make a new training input data
set so instead of our original which was
just some made-up stuff to set the base
for our model we actually have real data
now it's wonderful but all that's
happening here is we are assigning some
kind of value to it and we rerun the
model so this line right here
is it this is how you get a predicted
value from a machine learning algorithm
you do another function call that's it
it's pretty great but back on the front
end we get that function well we call
that function it passes the data back
and then we start to mess with our user
so this is where I like to ask you guys
what do you want to change on the screen
when somebody tries to hack you do you
want it to like add a style where they
can't type in anymore do you want it to
move everything around doing live coding
here so what do you guys want to see
today
Comic Sans
[Laughter]
I couldn't hear a lot of them but I
think Comic Sans might be it so we'll
just do something simple I think I have
that installed to font-family
let's see if that's actually going to
work
mmm is it still running
okay still running just so let's see if
it's gonna do Comic Sans hopefully
can't tell if they're changed can you
guys
Thanks
I don't think it changed it to Comic
Sans but basically I think you kind of
get the idea here that you can pretty
much wreck someone's day just with
machine learning and UI UX but on the
flip side of that I see if the other
demo is running
oh I think it might actually work
it's always nerve-racking when you get
up here and it's like I think my demo
works
stupid stuff
okay so this time instead of us you know
making life hard for the user one of the
things we're gonna try to do is make it
easier but I will throw in a caveat this
is just a fun thing you should probably
never make live updates to your UI UX
when a user is actively using it you
don't really want buttons and things to
move around just run that out there but
basically when a user types in something
so tell me something good in Hawaii
no of course one of them wouldn't work
but that's that's fine we roll with you
but anyways I'm not gonna stand up here
and try to debug that all morning for
you guys but basically what that one was
supposed to do is as a user types it
starts to pick up kind of their
sentiment on how they feel today and
based on that it'll either bright color
of the screen - like a yellow because
they're really sad and they need some
sunshine in their lives or it'll darken
it a little bit because they're a little
bit too happy and need to bring it down
but um that is pretty much it for the
code side I'm really glad at least one
out of two worked so I will go ahead and
wrap up so the key takeaways try to find
uncommon ways to implement machine
learning so one of the other projects
that I've worked on with machine
learning is like this EEG sensor that
picks up brainwaves and you can use it
to control stuff like real life stuff
like drones and freaking robots that
climb up the wall just from your brain
signals it's awesome and it uses
JavaScript and then use brain jeaious
because tensorflow j/s is really
difficult for no reason
and of course one of our goals is to
always improve our UI and UX but try to
think about how you could use your data
to improve the UI and UX on an
individual level so one of the things
you can do is get just this large data
storage factory of user data and say hey
I think I want to see if my users will
actually buy stuff on this day in this
country at this time when the weather is
like this machine learning isn't that
powerful and you can get those
predictions just don't jump into buying
a lot of data before you actually know
what you want to use it for because a
lot of companies have done that and then
their machine learning stuff didn't give
them any results so just some takeaways
yeah that's it for me
[Applause]