Hi, my name is Alan Richardson you can find me at eviltester.com
I just wanna explore difference between testability and automatability. Our
ability to test, versus our ability to automate. In terms of how the application
supports us doing that. Because very often when people are talking about
automating an application, they say the term testability, as if more testability
makes it easier to automate. And that is not really the case. I mean fundamentally
pretty much any application, if we can test it then it's testable. It has some
testability because we are able to test it as humans. Now we can increase our
ability to test an application. Our ability. By using tools and understanding
the technology. That's not in reference to the testability of the app. That's our
Test Ability in terms of how we can test it. And if we have more Test Ability then
we'll probably be able to test apps, that other people can't. But those are
different concepts. Even though the words are very similar. I make the
statement that "some technologies are easier to test than others". Right, I think
that means they have a higher intrinsic testability. There's just something about
that technology that makes it easier to test. But this is a subjective statement.
I find web applications more testable than a mobile app. Simply because I don't
like the mobile ergonomics. I don't like using that. For a web app I can sit back,
multiple screens. And I know what tools I can use for a web app to make it more
testable. So I can look at the HTTP traffic. The browsers have got
tools built in. Mobile apps don't really have that. I have to set up remote
debugging, that's a bit for pain. Green screen apps have to go through a
terminal emulator. And I'm relying on the terminal emulator. And it's just not as
testable an environment. Now an application's testability doesn't just
depend on our Test Ability. We can increase the testability of an
application by adding specific features to help us test it more easily.
This makes the application itself more testable. Some of those features that we
add might help the usability of the app. So there might
be a knock-on effect that the users can have. That may be a benefit of making an
app more testable. The testability is not automatizability. Right, testability is
"does the application have features to make it easier for a human to test"?
Automatizability or automatability (I just find it hard to pronounce) Is about does
the application have features that make it easier to control and interrogate by
another application. That is, is it easier to automate because of those
features. There are different things. Testability - humans. Automatizability -
automatability: machines, computers, programs, other applications. Now the
features that we add to an application to make it more testable are not
necessarily the same features that we would add to make it more automatable.
They might be similar. There might be overlap. For instance, one feature we
might have is logging. We might add logging to an application to make it
more testable. So improved logging allows me as a tester to see messages that I
would not otherwise be able to see. The state of the application at different times. So I
observe the variables internally and get more insight into the
application. If the logging was parseable and accessible by another program. If it
was in JSON. If it was easy to get through an API. If there's a certain call
to make. Then that might improve our ability to automate. But having it in
JSON doesn't necessarily make it human readable for me. I might prefer sentences,
paragraphs, an HTML view. We have different requirements in terms of how
we consume that information, that logging information, there's overlap, but they're
not the same thing. Testable, testability, does not mean automatability. And also
what we have to remember, and have to consider, is every feature that we want
to add might come with risk. Right we add testability features to show more
information on screen. That might impact usability because now the screen is more
cluttered, or more confusing, because we got all this extra stuff. So we need to
have the ability to switch it off, if we add that. There might be security
implications because we give away too much information about the internal
state: variable names, whatever else, maybe hackers can use that. So we need to be able to
switch that off entirely for environment so it can't ever be switched
on. They might impact performance. So sometimes you want to switch on under
specific users. We might only want to on specific environments. Right? There are risks
that come with these that we have to consider when we also explore the
features that we want. And there's the same risks with automatability there
might be security concerns because we've got an unprotected API. We've got a way
of getting the innards out. And particularly with automatability and
it's even more of a security risk than it is with humans. Because it's designed
to be automated, and hackers love it when they can automate this stuff. So
fundamentally testability is not automatizability not automatability.
Testability is for humans and human interaction. Automatizability is for
application interaction. I find that some technologies are inherently more
testable than others because of the tool support and usability. But generally
when we're talking about testability we're talking about the application, and
the feature set that the application has supporting us in our ability to test it.
So we can specify new features. We have to be clear on that. That might have a
knock-on effect on our ability to automate, there's overlap. But chances are
we need different requirements for humans than machines. And every time we
do it, there's a risk that we have to consider. There might be knock-on
benefits but there's also risks. So that's quick summary. Hopefully. Of
testability versus automatability.