hi everyone I'm a developer advocate and
applitools and we're going to
about visual validation today but first
we're going to play a little game of
whodunit
clearly somebody in this room murdered
Lord Smythe who at precisely 3:30 4:00
this afternoon was brutally bludgeoned
to death with a blunt instrument I want
each of you to tell me your whereabouts
at precisely the time that this
dastardly deed took place I was
polishing the brass in the master
bedroom I was buttering his Lordships
goes below stairs so but I was planting
my petunias in the potting shed
constable arrest lady Smythe but how did
you know
madam has any horticulturist will tell
you one does not plant petunias until
May is out take her away
it's just a matter of observation the
real question is how observant were you
clearly somebody in this room murdered
Lord Smythe who at precisely 3:30 4:00
this afternoon was brutally bludgeoned
to death with a blunt instrument I want
each of you to tell me your whereabouts
at precisely the time that this
dastardly deed took place I was
polishing the brass in the master
bedroom I was buttering his raw tips
goes for those stairs somewhere but I
was planting my petunias in the potting
shed
constable arrest lady Smythe
can't believe you all let
deadman get up and walk away but
actually I can't believe it because we
all suffer from what's called
inattentional blindness this is where
you're so focused on a specific task
that you miss other things new and
unexpected that might come into your
vision and this happens a lot when we're
developing our code and we do these
little quick I was about to curse these
little quick half effort tests and we
missed a lot of what else is going on in
our application right so how do we solve
it is we usually try to automate these
tests so that it's not relying on us and
when we automate the tests then we're
making sure that every time that test
runs is taking the exact same steps and
it's verifying the exact same things and
that sounds really good and predictable
right what if I told you you were
actually coding in the in intentional
blindness because our scripts are worse
than we are only looking at the thing
that we told it to look at and it misses
anything that we didn't specifically
think of to code in for example what
about if your text is on the screen and
you want to verify that this text is
there right so you write some code and
it verifies that it's there but what if
that text was bleeding off the edge of
the page you see this all the time on
mobile views what about it that text was
covered by another element and so your
user couldn't even see it
what about if it was the same color as
the background and again your user
couldn't see it so all of these things
will be missed with our traditional
assertions and this isn't one off type
things this happens all the time now
that I'm aware of visual testing I see
these visual bugs everywhere and some of
your apps as well here's an example this
is Cineworld
this is the world's second largest
cinema chain
and they had a visual bug on the page
that asks you if you would like to store
your credit card information
now if you saw this in your app you
probably would wouldn't think it was a
big bug
you know but how many of you in here
would actually or your credit card
information exactly no one no hands
worries the reason for that is cuz you
look at this and you think if they
didn't even see this I'm not convinced
that they're testing the backend and and
something actually holds my private data
right so engagement here would go really
low
now your test as written today will
probably pay
cuz it is looking to see if those labels
are there's all the radios the buttons
are there it can click things
functionally this works right so this
gets deployed into production and you
miss his type of thing this is open
table how many people use open table
make reservations yeah so I was born in
Raleigh and they're famous for their
barbecue so I wanted to make a dinner
reservation at my favorite place there
with some of my friends seven people
this reservation so I went to click a
time that I wanted and this modal popped
up and I was a little bit confused at
first because there was nothing in the
center there so I didn't know what I was
supposed to be doing as I looked around
I noticed in the upper right corner
those two buttons let's say select so I
scratch my head a bit okay what are
these buttons one why are there two of
them too why are they not aligned in
three what am i selecting again and so I
did what any of you would do I open
chrome dev tools I began to dig around
and figure out what were these buttons
and actually the labels were there they
were just as far away from the buttons
as possible again
think that these people don't test take
they're cold but they might be tested it
the tests were passed in this case
because the tests are looking in the Dom
right and all of this stuff is there
it's present functional but a user
cannot use this right and most of their
users are not engineers who know about
chrome dev tools and are gonna dig in
the Dom so this is money that's being
lost
these aren't one offs happens to your
favorite tech companies as well
Amazon someone not only wants to
purchase something but they want to
multiply the number of items the amount
of money they're going to spin and you
have this goofy bug
Facebook on their marketplace page I'm
going shopping I want to buy things I
cannot even read the text again your
test yeah
Texas present Instagram this is my
favorite one the reason this is my
favorite one is because this is
sponsored content someone paid Instagram
to show their polls to people and it
looks like this again your test is this
text present yes yes yes and deploy
Twitter all of the tweets are there the
text is present we can't read this this
is very annoying the great news is that
we don't have to live like this
visual testing can also be automated now
what is visual testing it's where you
take a screenshot of your application in
the state that everything is perfect and
as you want it and then on any
regression runs it takes another
screenshot and compares the two there
are any differences it alerts you now
visual testing is not something that is
very new it's been around for a while
but we as an industry we're using some
antiquated methods but we were comparing
them pixel by pixel and making sure that
the picture was exactly the same the
problem with this is that pixel
comparison is really flaky not really
designed for testing purposes for
example the top picture is picture of my
app when it's in its perfect state the
second picture is a regression run the
regression run failed token see walk
the search button is bold here in the
second picture but not the first well if
you put your mouse over the button it
bowls so if you were running this
front-end test and it just so happened
that the mouse landed right there when
the screenshot was taken now you have a
field feel it another one who can see
what why this one feel
the cursor so after the word bad the
cursor is solid in the first picture but
not in the second cuz cursors blink
right so if you happen to take the
picture in the split second that is not
the same than that feel so this became
very very annoying for people apples
came out with a way to do visual
validation that uses machine learning to
be able to only capture the things that
we would care about as human beings
here's another little spot a difference
game you all don't get to play because
you let the dead man walk away in the
first game so I did it for you I ran
this through the pics of the pixel and I
ran it through the athlete tools so with
pixel to pixel you'll notice here pretty
much everything is highlighted very very
sensitive even to white spaces using
this I can't really tell exactly what's
different in those pictures versus using
it with applitools it's faster things we
would as human beings okay
so let's add this to an existing test
base first let me show you the
application that we're going to be
testing it's a single page application
and it has 8 of my favorite books on
testing and the way that this works give
us some more real estate the way that
this works is you just type in one of
the book titles and it filters that for
example if I put in test and I see all
the books that have tests as part of the
title that's it okay and let's look at
the test now so we have
two tests here one that is looking for a
specific book with the exact title agile
testing so we search for that we make
sure that there's only one book that's
returned in that it is the right title
and then the second test this one is
looking for multiple ones so the one I
showed you in the example you put in the
word test here are the books that we
expect to come back we validate that all
of them are actually visible and they
are at the right titles okay
so let's run this test this big one and
I'm using Cypress here anybody use
Cypress yeah
did y'all see that announcement
so they just released 4.0 yesterday and
now cross browser support I know all
right so we've run this test and this is
past beautiful everything is lovely
okay let's switch back to application
mode and let's say we want to like jazz
this page up a bit you know we want a
little bouncy effect or something so we
had a little CSS here and we have test
so we feel very confident and we go
ahead and commit this right so we run
this little test again and let's see oh
no with one line of CSS I have flipped
this application on its head
but the test is still passing why is
this because we said make sure there are
five books are there five books check
make sure that those five books are
these titles that's true right it's
missing these other things the
inattentional blindness so let's add
visual testing to see if we can catch
these kinds of bugs that our traditional
frameworks would miss okay so to add
visual testing
the first command is to open your eyes
beautiful a tree so we say sigh dot eyes
open
all right and then after the test let's
go ahead and make it after each
and in here we're going to want to close
our eyes when we're done very intuitive
all right okay now within the test
itself let's go ahead and add some
visual testing here so we're going to
say side eyes check window
entire thing this will do a screen
capture and do the visual testing and
with that screen capture it's gonna make
sure that this title all of the titles
are visible so really I don't even need
this line anymore it's gonna make sure
it's only five of them so I don't need
this so now I'm doing my favorite thing
which is to delete code I no longer need
this array oh my gosh this is getting
really good and I don't need a variable
cuz I'm only using it that one time so I
can put that there so look how nice my
test looks now yes yes okay all right so
let's make sure we put the books back
yes okay so we're gonna run it this
first time and what's going on here did
I mess something up okay
okay so what's going on here is it takes
a picture it's going to send this image
out to the applitools cloud and so I
don't have to store this stuff on my
computer I can if I wanted to there's
options for that but by default this
just goes out to the applicants clown
and applitools will then create this as
a new baseline saying okay this is a new
test that you have we'll store that
there all right and then every time I
run the regression runs is going to
compare the two now this is still
running because I'm on like my Wi-Fi so
it's a little bit slower but let's go to
the applicant tools dashboard so we can
see now you don't have to do this the
very first time you run the test I'm
really done but I just want to show you
this okay so we have this search here
and we see that there's no baseline so
this new image will be the baseline
image that will be store for us once
that looks
techgirls juice
alright I'm gonna let that cook but you
see the little thumbnail here right
alright so
okay there is all right so let's talk
about this a little bit this is really
powerful this is great like I want to
run this again with the books flips and
let's make sure that it actually
captures that okay so let's uncomment
out that we flip the books and let's run
our test again
okay great so this is now going to take
another screenshot again since this out
to the cloud and now it's doing the AI
comparisons to make sure that these are
the same or not and then it should alert
us when this feels
all right so see we got a little error
here and Cyprus you have to go into like
chrome dev tools to see the error but
I'm just gonna show you here so we come
back to our dashboard and we see that
now we have this test marked as
unresolved so AI is saying that failed
but hey something has changed here human
come and see if this is what you want it
like this was an intentional change or
not
of course flipping our books upside down
is not an intentional change so we say
oh no and we just do a thumbs down and
now this test is marked as filmed okay
so this will capture the things that we
miss with our inattentional blindness we
don't have to think of every single
assertion to add to our code base this
covers everything but on the flip side
of that that's also a con if we think
about it in some cases right because it
captures everything so not just these
books it also captured the text that I
put into here the name of the the title
on the application anything else that
was going on a page so that's great in
some cases but it could also be a
nuisance in other cases where it's
capturing too much right so we could
solve for this by scoping this down we
don't have to take a picture of the
entire page it's very very flexible so
if I wanted to do this other test
instead let's uh let's disable this one
and here I'm gonna do
okay so this one is the one where is
just one book now let's say that
I only want this one book captured I
don't want to take a picture of the
entire page so we can simplify this
again we get to delete some code and
then let's do the side odd-eyes check
window and this time we can specify some
information so we can say target a
specific region and give it the selector
for that region that you want targeted
so I already know that it has an idea of
PID 3 so I'm saying just do this one
book all right so if we run this again
did I miss up my search
yes all right so it takes a picture of
this but this time only the book so it
finds that selector it takes a picture
of that and then sends it off to
applitools to do the comparison
so let's look at what that looks like
all right so we have a new test here and
we see that it's only the book that's
captured not all the rest of the noise
so this is pretty powerful as well we
can scope it down as we want but there's
some drawbacks to this approach too
right
so for example let's say that I actually
typed in the word a dual testing here
and it didn't work the filter didn't
work none of the books were filtered
away all of them were still there would
this test pass or what it failed how
many say pass I mean say fail how many
are on the edge of their seats waiting
okay so this would actually pass because
it's going to find that specific region
that I told it and that book is there
and it looks beautiful so this would
pass so we could couple our visual
assertions with the traditional
functional ones as well so in this case
we could say okay only do the visual
testing on that one book but then make
sure that there's only one book showing
so that way you're mixing and matching
it doesn't have to replace everything
you already have you can do both okay
you can also do testing on dynamic
layouts so let's look at another test
let's go ahead and just make one really
quickly
and this one we're gonna say should have
a nice layout and this time we're not
even on a search okay so we're just
going to look at the layout it's
who works on vs cold here I don't like
that part I don't like that feature
alright
so have a nice layout here so let's go
ahead and run this as is so what this is
going to do is it is going to at this
point is running as normally but we're
going to add some things here to see you
know what I want to test some dynamic
content it might not always be the same
every time so the way that this works is
is going to use machine learning to
observe any patterns that it can find
with this application and the patterns
that I'm guessing it's seeing is okay we
have some squares and they are this much
width apart and they all have like a
purple thing at the top and some white
text in it and a picture and some bold
text and blah blah blah so it's figuring
those things out and then we can
actually go into our application and we
can change the order of let's just
change the order of a couple of books
all right now in order to say hey
applitools we want to do some some
dynamic testing here we are going to say
inside of here we can do match level
and say this equals a layout match level
okay and then we'll run this again
and there are four different types of
match levels so the one we've been using
by default was streaked that just does a
regular comparison there is exact which
does pixel to pixel but we say no the
pixel to pixel right so don't use that
one and there is this one layout which
does the dynamic content and then
there's also a content one that does
color blind comparison all right so
let's look at this test I've run it
twice so I should have a baseline yes
alright so notice here that cucumber
book and the first picture was in the
third place on the second row but in the
second picture it is the second place
and this still passed because it noticed
okay the layout and everything is fine
nothing looks messed up here and if I
wanted to preview this in a different
match level just to see it if I did it
in strict mode it would have captured
that these books were different alright
so this is really powerful we use this
when I was working at Twitter after the
ugly bug we found and we have like new
sites and things they use this where
their content is gonna be different
pretty much every time the app loads but
structurally you want to make sure
everything looks good alright and
finally last thing I want to show you is
that you can do cross browser testing
with this as well so let's get rid of
that
and I'm going to I should've did that
just pass in like all of the things I
want to test on so I want to test on
firefox and chrome and ie and I want to
test on iPhone 10 and in landscape mode
and in galaxy so I can just add this
here and this will run all of those
tests across all of those configurations
so I mean it let's let's run
let's run both of those tests that we
had our first two tests
okay
so notice this just opens in electron or
whatever this is not going into all of
those different browsers at least not
from our view so what this is doing it's
really magical in my mind it is
capturing the DOM and all of its assets
so all of the J's files all of the CSS
files and it's taking it in that state
and then blasting it across all of the
configurations that we said so you don't
have to execute those scenarios over and
over again across all of these platforms
it just takes it once it's already in
that state and shows what it'll look
like in all of them so if we go back to
our dashboard let's see what we have
here
okay wallah so in a matter of seconds
we've run these 12 tests it's a lot to
look at I can even group this down to
like by device or like by browser and
then I can manage this a little bit more
and I see that that test has run so this
is iPhone in whatever mode that is which
one is that landscape and this is a
portrait and then this is the book on
both as well so that is visual testing
in a nutshell very very powerful stuff
you can open your free account out on
applitools site let me know if you have
any questions I'll be around for the
rest of the day and you can also you can
also hit me up on Twitter at Tech girl
one 9:08 if you don't or if you're too
shy to come talk to me in person alright
thank you so much
[Applause]