Hi, I'm Eric Minick with IBM Cloud. I want to talk a little bit about Continuous Testing.
This is a practice where we're trying to drive really rapid feedback to development as changes come in.
How quickly can we identify that there's a problem, let development know, get it fixed.
This really supports the DevOps principles; it really supports continuous delivery.
Now, the way we see this done most often leads to some trouble because we think:
All right, how we can do it fast? We need to automate some stuff. How are we going to automate it?
Well, our manual processes, well, we're working at the UI level. So, we should grab functional test automation,
either write some scripts or do some playback and record,
and we're going to test a lot at the UI level.
And we might also say, Well, part of our product is a public API, right?
So we should test those APIs, I guess, as part of that.
We'll do a little bit of API testing.
And I guess good code quality probably helps from some unit testing, we'll do little unit testing too, right?
So, we see this sort of approach.
This is also the approach that probably leads to more test automation programs kind of failing than any other.
And the problem is, that the UI is relatively malleable, it changes a lot.
And so when you start moving buttons around, and renaming it, and changing elements,
your UI tests just keep breaking.
And so, as you started out you have a few UI tests, and they're going well,
and you start getting more and more and more and everything keeps breaking.
And you end up with this tension between, "Can I test it?" and
"Can I, you know, actually change things to be responsive to the market?"
- and that's all kind of anti-DevOps.
So, that's sad when your weighted too heavily towards automated UI testing,
but even that, that is way better than manual UI testing.
It's much faster and better, so it's the only way we can do it.
So, we know we need automation here, but we want to flip this approach.
Right, we want to say the bulk should be unit testing.
Every interesting piece of code that we write should have unit tests along with it, right.
This is where we see principles like code coverage,
Let's make sure we get 90% code coverage, 100% code coverage, on our unit testing.
So, the code that the developer writes comes with unit tests.
But then the real key here, what I find most interesting, is to put a lot of weight on the API testing.
Particularly in a world where we're moving to more microservices,
but even very true as we're looking at service-oriented architectures and things like that.
API testing does a couple of things:
One, it captures our business logic at run time
and some of the interconnection between various pieces and parts of our applications.
The next thing is, the easiest way to break all of our microservices
is to have APIs start changing, and then a different microservice breaks.
If we're putting our APIs live and calling those APIs and making sure their steady,
right, they're at least backwards compatible.
That helps us have healthier microservice architectures.
And so, this is a level where things aren't supposed to change a lot,
whereas at the UI level we're still going to have our automated UI tests.
Things are supposed to change a lot, they are supposed to be more reactive to the market.
So, let's do less testing, less automated testing perhaps, at kind of that most dynamic level.
More at the most predictable levels, the code levels.
This is really powerful. This is stable.
These are where we see the most successful transformations towards continuous testing.
Now, if we were to stop here, we would have a nice automated testing story,
but we would have left out the people - and people are absolutely critical to testing.
Manual testing absolutely still has a role in how we deliver high quality software.
And so, there's a couple of key pieces here, right.
We're going to want exploratory tests.
And this is where we have (let's see if I can spell "exploratory")
this is where we have our smartest, you know, good testers going in, exploring the software,
trying to break things, right, looking for things we might have overlooked in our other tests.
This is also a place where you often have product managers, and other people like that, working on the user experience,
right, making sure that as they go in and they see the new features, it all works and it feels good, right.
You'll get your designers involved here.
So, you got these elements, and then typically you're going to have some real users actually try the software,
and I'm just going to call this a beta.
This could still be, you know, what's normally called user acceptance testing.
It could be in a more consumer-oriented app,
it could be something like, rolling it out to 5% of your users and seeing if they like it or not.
You're not going to do that in a medical app most of the time, because you don't want to have bugs hurt people.
So, manual testing still has a role. It is not following a test script, right,
if they can be written down in the test script then it could be a UI test,
but more likely the things underlying that all API calls and you want to put your testing here at the API level.
So what I challenge you to do is make sure that you're building out your unit tests,
making it look less like this more like that, and that you're looking at all of your UI level tasks
and saying, "Could we test the same thing at the API level?"
Then reserve your functional testing, or your UI level testing,
for real end-to-end validation that everything actually works together.
You need some of it, don't overdo it, and make sure it's the end-to-end sort of capabilities,
not small feature-function that could be handled by the APIs.
So, I hope that's useful introduction to continuous testing and the biggest "gotcha" that we see out there.
Thank you. If you have questions, please drop us a line.
If you want to see more videos like this in the future, be sure to like and subscribe.