[Music]
hi everyone Wow
thank you for having me III have to say
I'm not a I'm not a front-end developer
I've been switching back and forth
between front-end and back-end
currently I'm in my back-end face and
the presentation is not loading wait
turn Wi-Fi off there we go hopefully but
one thing I did notice both in Jing's
presentation and in Jason's is this did
everybody knows that they were both
showing us how responsive the things are
and I was thinking to myself oh wow and
they were like it this is simple true
but what if I change the CSS that that
that that's problematic because I have
to check it every time time and time
again hi I'm a guilty our testing is is
what is it's been my passion since about
2000 I'm a developer advocate and a
software developer mostly software
developer at applitools
and what we do is we specialize in
visual testing tools and I want to talk
about fear definitely fear and we'll
talk about fear of what in a sec but
first I want to talk about web
developers first let's start with the
back end developers back-end developers
have been there since before the web and
I'm a back-end developer right now they
use words like you know thread pools and
message queues and Scala's and you know
they're back there and without them come
on we wouldn't be able to do our front
end so round of applause there you go
what about front and front end is
divided front and let's talk about
front-end JavaScript with developers
they're wizards they deal with things
like JavaScript and angular and react
and state management and Dom and and all
that kind of stuff and they're really
wizards and the way they handled things
what about front-end CSS developers
fronting CSS developers are beautiful
that's for you come on a round of
applause for you
there we go not only are you all
beautiful you create beautiful things
using CSS and I'm constantly amazed by
these things and how you manage to work
with them so you deal with responsive
design media queries and flexbox
and something has wonderful is happening
in the web development world a really
wonderful thing and what is it
developers are starting to write tests
that test their own code this has been
true for something like five years or so
obviously a lot of people be testing
before that but it's become slightly
mainstream this night last five or seven
years but why why are developers writing
tests the word I use is a go programa
phobia anybody know what a go programa
phobia is probably not given that I've
just invented it it's fear of one's own
code okay it's and I'll explain that you
know the manager he says you know just
add that feature and and the way they
think about it is just you know adding
that Lego to the existing code and when
we're done the truth as we all know is
something like this that's fear the fear
of doing that kind of thing doesn't
always happen usually doesn't but when
it does out removing code we want to
refactor this is especially true in CSS
we think about it as just you know
removing that one single line or in CSS
just
need that rule that rule shouldn't be
there let's just take it out but we all
know what happens in the real world when
I take out that CSS somewhere some page
is gonna break yeah
refactoring I really want to refactor I
really want to refactor that JavaScript
but I'm afraid this will happen to me
and it doesn't matter whether our code
is a horribly written pile of spaghetti
or a wonderfully beautiful crafted set
of loosely coupled modules we really
fear our own code which we fear changing
it because changing code may mean
breaking the app and web developers as
all developers have have discovered that
developer testing or testing in short
removes the need for this manual testing
after code changes and developer testing
is writing code that tests your own code
as part of writing the code itself so
it's not QA automation engineers doing
that for you it is you doing that for
yourselves and testing abolish is that
fear it makes that fear mostly go away
because you code and then you write the
test for that code and it doesn't really
matter whether you're writing it before
you write the code TDD style or after I
don't read here and you know you run
those tests and they break and you fix
the pods etc and it works and once you
change your code you just run the tests
again and you're done testing abolishes
fear that is the most important thing
testing abolishing the fear of one's own
code who's writing tests who's
abolishing that fear
well back-end developers have been
writing tests for quite some time
they've had that methodology even before
the web the methodologies and tooling
are mostly established they've been
working on it for quite a while
front-end JavaScript developers well the
Weber's is young in terms of
methodologies it's not totally there yet
it's slightly confusing nobody really
knows exactly what it's but the tooling
is almost there it's happening it's
happening we've been happening for quite
some times a lot of mentors and teachers
can't see das just in serials Kevin Lamb
pain Gleb ajamu Tov Shai Resnick and
others hopefully including me we are
slowly building for JavaScript
developers a methodology based on unit
testing component and multi component
testing using JS DOM and things like
Cypress and selenium and webdriver IO
what about you you're sad
no tests for you you don't know and why
why is that well I'll explain a second
front let's talk about front-end
JavaScript tests they're functional
let's say I have a function you put an
input it does its thing you grab the
output you check that the output is
correct you're done you've written your
unit tests what about the you know UI
tests not a problem
you open the browser using also
automation tools you write you do the
click click clicks and the type type
types and then you check that this field
is the correct value in that field is
the correct value and this field is the
great value same stuff and that nothing
changed but how do I functionally test
something like this I could check the
width of the columns when I you know
when I do the responsive design it's
possible but it's a huge amount of code
and you are not testing everything and
you're saying quite rightly we can't do
functional testing and you're right you
can't I tried a few years ago right wigs
we did I was my front end face and I
tried writing functional tests yeah whoo
it worked not no problem the tools in
tooling as I said is there and then I
tried to do visual testing couldn't
succeed testing CSS and testing visuals
is a hard problem you're not why because
it's visual not functional anything to
do with vision is problematic but don't
believe the rumors writing tests for CSS
is possible and the point is not the
writing test is for CSS as possible is
that you should be starting to write the
tests for CSS it is time for that how
I've been doing a lot of talks on this
issue and talking to a lot of people
people think the developers think it
goes like this
you navigate to a page using web
automation tools like the selenium or
Cypress take a screenshot and then you
tell the tool whatever library you're
using for testing
does this look good right they really
think that or or maybe does this conform
to a certain design system okay this is
my design system does this UI conform to
it and the answer is no unfortunately
we're not there are machine learning
algorithms in our AI
and we need AI for this type of stuff is
just not that good yet but right now so
how does it work
we're not actually doing visual testing
we're doing visual regression testing
what chain we're checking that what has
been still is okay that nothing has
regressed or changed and this is a
typical visual test it's that simple I'm
using a Cypress I love Cypress it's my
go-to automation tool but there are tons
of other selenium-webdriver i'll test
cafe all of them support visual testing
to once at one point or another first of
all we set the viewport because setting
the viewport as we all learned and we
all know in responsive design is crazy
important then we use Cypress to visit
and then it opens that you know browser
to whatever site we want then we do the
typing and the clicking to get to
whatever page in whatever state of the
UI we want and we call the tool with
visual testing tool just call it and it
does its magic this is where the magic
happens
and what is that magic the tool first of
all takes a screenshot of the app it
compares the screenshot with a baseline
that was previously given and I'll
explain what when it was given it checks
for differences if there are no
differences good your test pass if there
are differences it doesn't know whether
that difference is a bug which it
usually is or a feature I've just added
a button I've just changed the styling
so you need to check manually accept or
reject that difference and where does
that baseline come from that baseline
comes from a previous test the first
time this tool runs it accepts all the
screenshots the
and ask anything that's the baseline
when you accept a difference because
it's a feature not about that becomes
the new baseline and so on and so on
this is visual testing in a nutshell
that's how you test your CSS thank you
very much questions now just joking I
wish there's an elephant in the room
because it's visual it's hard it's not
that easy for elephants in the room
let's talk about elephant number one I'm
good
we've seen dogs we've seen cats now we
have elephants first of all taking a
screenshot is hard it's actually the
smallest elephant but it's still there
are you using Cypress that's easy
taking a screenshot in Cypress is great
Cypress runs under chrome chrome has
great screenshot taking facilities you
want only the viewport only the window
fine you want the whole full page fine
you want a specific region or a specific
element we're good we don't always need
the full page sometimes want to take a
specific segment of that page otherwise
if you're using if your JavaScript
developers are using selenium or
webdriver IO or others that's a problem
ouch you can only do window how do you
take a full page when you can only take
a screen shot of a window so what if my
JavaScript if our JavaScript developers
use selenium we just you know I won't do
visual testing I'll just do this
manually all day and the answer is no as
I said don't believe the rumors taking a
screenshot is possible if you're using
Cypress you're done if you're using
selenium or others one there are open
source and commercial
tools that know how to do it in weird
and wonderful ways basically take a
screenshot viewport of the viewport then
CSS transform the body up take another
screenshot sisters transform the body up
etc etc until we stitched all the
screenshots together and we're done
so tooling knows how to do that just
look for it that's the easy part
elephant number two I think this is the
biggest one but comparing the
screenshots is hard so you have your
baseline screenshot and you have your
current screenshot how do you compare
them most open-source tools and others
do pixel by pixel just check pixel by
pixel to see whether anything has
changed that's easy to do but look at
this these are two screenshots of a
certain page from 2004 I think this is
the my Windows machine and this is I
will use Mac but nothing more this is
what the top one is my machine and the
bottom is my friends machine they look
exactly the same but notice they're not
why is that why is the same machine same
Chrome or Firefox or whatever same well
Windows operating system giving us
different results and the answer
incredibly enough is graphics drivers
graphics drivers graphics a car graphic
cards etc the difference are really
slight but they are there let's do
another one this is a picture from
Chrome we found it out during the move
from chrome 67 and 68 it blew my mind
this is chrome 67 this is chrome 68 same
JPEG look exactly the same right nope
this is the difference in the pixels
yeah most of the pixels are different
basically it's probable they probably
change the library that they used to
render the JPEG or something I don't
know but there is
the difference remember the testing
bookstore will see a demo shortly this
one written on the same and this what I
was doing while while preparing the talk
I found that out using a pixel-by-pixel
Maya library the same one same machine
5-minute difference between the two
these are the two screenshots you can
see zero difference right nope really
slight I have no idea why really I was
actually surprised and somebody who
deals with these pixel differences a lot
I have no idea why there was a
difference there but there was like in
one of 10 times I ran the tests which is
why all the tools have this it's a
slider which says do you want it do you
want false positives or false negatives
it's it's basically it's basically
saying how much percentage of error is
okay if you say no percentage of errors
you'll get a lot of false negatives if
you move that slider to the right you'll
get a lot of false positives or the
other way around I can never figure it
out but you have to tweak it constantly
and per page as I said that's it and
that's the biggest problem because we
don't want to constantly manage our
screenshots nope it's not the same
because a lot of these tools employ
advanced screenshot comparison
algorithms AI machine learning just
really really complex algorithms what
they do is they don't look at the pixels
they try and find the elements in the
page and look at the elements and
compare them if there are slight
differences there they just ignore them
they're actually seeing as a human would
see or at least trying to do that so no
false positives no false negatives if
you're using advanced comparison
algorithms it is possible and that is
the biggest advance and
testing that we've seen in the last few
years and all of these if there are
commercial tools they all have free and
OSs plans so go ahead and use them
elephant number three managing those
comparisons is hard okay so we've got a
difference and it's like we're doing a
hundred screenshots and a lot of them
let's say I change the header all of
them have changes there that's the part
where you accept or reject the
difference you have to do that manually
and as I said hundreds of comparisons is
that a problem let's see one way to deal
with it is we're doing deltas
small Delta's so a lot of times it's the
same difference in all the screenshots
so I just use the tool and say yeah fine
update update the baseline they're all
fine because if it's a bug there is no
problem I don't need to do anything but
if it is a feature if there's an
addition I just acceptable go away the
other way is taking like dragging from
the folder that has the current
screenshots dragging it to the baseline
screenshots and dealing with that every
way looking at the day of saying okay
this is above this is a feature this is
about this is a feature and dragging
whichever way you want other tools
mostly commercial deal with it using a
dashboard where you can accept or reject
the differences as you wish it's
basically a salt problem each tool in
its own way but you have to think you
need to manage those screenshots and
this comes to this one the big one
testing all those responsive wits and
browsers we're talking 1024 retina iPad
we've seen that and it's getting more
and more interesting because more and
more sites are becoming responsive not
only that it's not just responsiveness
its Chrome Firefox Safari etc etc
different pixel density
we're talking different pixel densities
also do we abandon ship because oh my
god how do we do that
one solution is just run the test
multiple times I have a 4h on the test
on all the sizes I want I phone iPad etc
on each I run the test multiple times
and change the viewport accordingly and
take the screenshot and we're done
definitely doable
the other solution well that's the
solution but it takes a lot of time if I
have a hundred tests now I have 500
tests the other solution is do the same
but paralyze the testing that takes a
lot of infrastructure but it's
definitely doable and a lot of companies
do that solution number three is let a
cloud provider do that for you send over
not the screenshot but sent over a
snapshot of your Dom in your CSS etc and
then run all those tests all those
screenshots on a lot of browsers in the
end it really is as easy as this I want
to show a demo because I do have some
more time No let's see a demo
let's let's start with the application
under test this is the application under
test I can filter right here test and it
filters it obviously it's all responsive
but you've seen that already now let's
look at the test let's look at the
regular OSS tool as I said it's just as
it were but viewport visit get type type
the testing and we're done I will be
using an apple tools eyes where I have
to open the test and close the test but
it's basically very simple simple I run
the actions and then do a check window
very similar same tooling as I said they
are all the same let's run the test we
can run it non interactively but Cypress
is really cool so let's do that
there we go the test is running
hopefully are we good yes Wi-Fi is
working at least the ethernet is working
it's running it did the check window but
now it's waiting for the answer from
from Apple tools from eyes and let's go
to the dashboard and there it is we've
just passed notice that it doesn't give
one screenshot although we did give one
screenshot it's running for tests in
Chrome and one test in eye 11
this is actually an iPhone X emulation
so you get the responsive design as I
said we're give we're letting the cloud
do our work for us okay now let's break
something sorry about the artificiality
of it this is it no that's not the
extent of my CSS I know it better but
okay so let's change the title to read I
will close it
there we go and we'll run it again so
it's running now you can you can see
that the testing book store is now red
and it's running it's throwing over
those dumb snapshots to the cloud and
waiting for the results of those tests
let's go to wherever I need to go and we
have an error in all the pages okay as
you can see it's this error I'll show it
like this okay now we need to say
whether this is a bug or a feature let's
say that it's a feature we wanted to
change it to red sorry my design
abilities are really low what I can do
is I can group all of them and approve
the two that it grouped on and save the
changes I won't save the changes so
maybe if I give the talk another time I
can do that again so basically this is
the level of sit it's it's basically
that easy remember just write the code
that uses the tool does the visits does
the clicks wherever you want to take a
screenshot you use the tool to take that
screenshot and then you do have the
managing of the screenshots so
as I said it really is as easy as that
as long as you take care of those four
elephants that's important I'm in
summary so do we believe the rumors no
writing tests for CSS is possible and so
join the backend and the front-end
JavaScript developers start testing your
CSS you can cure your CSS ego programmer
phobia it's a hard problem don't don't
worry it is a hard problem because it's
a visual one and visual stuff is always
hard taking the screenshots managing
those screenshots comparing the
screenshots and take testing across
various browser widths and Heights and
and platforms is really really difficult
but it is solvable with open source and
commercial tools so who knows what this
is come on raise your hands anybody read
do you and thank you by Frank Herbert
we're gonna get a movie soon I must not
fear we're talking about fear fear is
the mind killer fear is the little death
that brings total obliteration I will
test my CSS I will permit it to pass
over and through me and when it has gone
past I will turn the inner eye to see
its path where their fear has gone where
the fear is gone there will be nothing
only my tests will remain thank you very
much
[Applause]
[Music]