[ Applause ] Our next presenter is Archana Chaudhary.
Archana, I'll let you come up here and give your talk.
>>Archana Chaudhary: So thank you for giving me the chance to present on a roundtable topic
today.
We were about 10 or 12 people in a room and we discussed the topic of documenting the
automation test coverage.
So we all have a lot of -- we all have a lot of legacy test cases that we may have taken
from some other team prior to us.
And then there are, like, a lot of Java, jUnit test cases or API test cases or maybe UI regression
test cases, and there is no way to know for us that what is the coverage like from those
automation test cases so that we can give an idea to the new people who are adding the
test where to start from.
So in the discussion, what came up, the first thing is why should we even think about documenting
the coverage?
And some of the questions were that we want to know what are the work flow and features
that are already covered by the automation tests that we have.
And they are at times when we want to map what has been automated to what was actually
in the specs and requirements.
So that, like, for my team, we have in the definition we've done that we should have
the automation complete.
What does that mean?
How many of these specs and requirements have we covered by the automation tests which are
done?
Then there are -- Most of us work in monolithic codebases, meaning that even if we are working
in smaller scrum teams, we are contributing to essentially the same main code base.
And what if a team is developing a set of integration tests and another team is developing
a set of other integration tests but they are duplicating the efforts?
So documenting the test cases in some way help reducing in the duplication of the automation
effort.
It will also give the developers an idea of suppose they want to run the tests for a particular
module and they come and ask a question: Which tests can I run for this part of the application?
Good documentation will avoid those kind of questions, and there should be a way by which
the developers will be able to pinpoint to a specific area and will be able to use the
automated test framework to validate their own work.
So some of the typical problems that we know, like were discussed in the discussion were
that when we are dealing with manual tests from, like, written by legacy manual QA testers,
there is not a one-on-one core relationship with what is a manual test to how you can
automate it.
So a manual test documentation does not serve as a documentation for what has been covered
in the automated tests.
And code coverage does give us some information about what coverage that we have in the module,
but it does not specifically test whether -- like it's not specific to the functional
parts of the application.
It can pinpoint to how the code base is organized.
But if your organization -- if you're looking like from an organization from feature perspective
or from work flow perspective, code coverage may not be able to give you that data.
So some of the recommendations that came up from the group was that using good class names
and -- of course will be the most helpful thing because all the reports, test reports,
will show the class names as the first thing, which somebody can take a clue from.
Writing some level of Java docs on your test cases will be helpful.
Now, there is cost and benefit for writing Java docs.
You can go extensive and your documentation itself will become a maintenance so that balance
needs to be maintained.
Some people are using the logging techniques in the page object patterns that in the page
object patterns, when they are -- for each and every action that a page is taking, they're
logging that.
So when the test is running, it is very easy to debug to see, okay, exactly which particular
action caused the failure, and then it is easier to go back and to debug and triage
the particular test.
So a second recommendation was that we should write tests which are smaller.
So, yes, big integration end-to-end tests not only becomes flaky but are also difficult
to document in the sense that you don't know how much is that test covering.
So writing it smaller, like smaller test cases, breaking it into smaller pieces will help
pinpoint to the specific problem.
If there's a problem only in the initial part of your work flow, the test will catch it
rather than the test running and then failing or you are figuring out, okay, end-to-end
test flow breaks but we don't know which part broke.
And then it's very important that the development engineers who are writing the features themselves
take the ownership of writing the tests because in this way, there is no handover between
the software developers and the software test engineers or the quality engineers who have
to sit down together and to understand what -- like, tell me what to write the test for.
So instead of having more communication channels, standup meetings and all, the best thing is
to have the development engineers write the test.
But if it is not possible, have as many as daily standups and things like that so there
is a communication going between the developers and the people who are writing the test to
know that the tests are meaningful and are covering what the developer wants to cover
in the tests.
Somebody mentioned the tools like TestRails and all.
There was also a methodology -- I didn't get that properly.
This talk happened at 2:00, so there was not much time to prepare the presentation.
But if you need to know more, there is a tool called TestRails, and I'm sure the person
who mentioned it must be in the crowd somewhere here.
We did take a tangent in the discussion and we talked about the Android native apps, and
one of the persons in here is working on tool which is based on the technology.
I'm not sure if I captured the words correctly, Emma and the .em files, which are basically
to do coverage from the black box testing that is happening on a mobile Android device.
So it is like a code coverage, but it's gathering the data from the manual tests that are happening
in a particular run when you are running your test on the Android device.
And the person mentioned that combining this data with the code coverage data that you
are getting from your unit test will give you a good -- somebody offer what is the coverage
from your test, like overall testing.
Now, this is specifically for the unit -- for the manual test that are run on the Android
devices.
Yeah.
I think I had one more present- -- like, one more slide.
But maybe it -- We also talked about one more topic, which
-- sorry.
Yeah.
It's okay.
I think there was a talk about how the business analysts when they are giving the requirements,
there are some people who are using the tools in which the business analyst requirement
are automatically converted into automated tests, and now these automated tests are used
by developers to do test-driven development.
So that now there is a flow not directly from a business analyst to a developer, but through
the automated test, which actually gives them a way to know what to actually code, instead
of just giving -- like, getting pure English business analyst documentation.
Thank you.
Any questions?
[ Applause ] >>Sonal Shah: Yes.
>>> Two quick questions.
On recommendations, did you guys discuss getting stakeholder buyoff on the time it's going
to take to actually document the tests?
And then the second one is, -- oh, I've got to remember now, along a similar vein, how
do you justify adding the time into the sprint for the documentation time it's going to take.
>>Archana Chaudhary: We did not discuss these topics.
But I can share you with my personal experience that is happening in the group.
So as far as the shareholder input, I am the engineering manager of the group, and because
this is a problem, I do allow that to happen, because it ultimately helps when you look
at the same test cases one year from now.
It may not seem most useful right now.
But one year from now, when you are still working on the same product, if there is a
new set of developers or a new set of test engineers who come on board, it will be easier
for them to pick up what has already been done.
The second thing was how did we -- like, how do we incorporate it in the sprints.
What we have done right now is that we have taken a few stories which are technical dev
stories and have put -- okay, let's put Java Docs into each and every page object file.
And that has helped in two ways.
That the people who took over these stories were the new-comers for the team.
So not only it helped them understand what the page objects that are written already,
so that when they have to actually write the test, they will know what is available.
And it will help the future people to know better.
Okay?
>>> Thanks.
>>Archana Chaudhary: Thank you.
>>Sonal Shah: Anybody else?
All right.
Thank you very much, Archana.
>>Archana Chaudhary: Thank you.