When changes are made to a software system
defects may sometimes be introduced to other parts of the application that were previously working.
To guard against this, it’s a good practice to run a group of specific test cases
to verify that the system is still operating as expected after the change.
We call this regression testing.
Rally supports regression testing by enabling you to assign one or more test cases into a test set,
which can be copied, estimated, and scheduled as a whole into an iteration.
Using test sets allows you to schedule a group of tests that have a common purpose
when the individual test cases are not directly associated with a particular story or defect.
This is convenient for running confidence checks,
and for other purposes such as accepting builds,
checking browser compatibility,
or validating functionality of critical areas of the application
after software changes have been made.
If you want to schedule a new or existing test set directly into an iteration,
one place to create it is from the Iteration Status page on the Track menu.
As we’ve learned in the Test Planning with Rally training,
a test plan folder with the test cases we need may already exist.
It’s easy to create a test set and schedule it directly into an iteration,
then pull existing test cases from a test folder into the test set for the iteration.
But if you want to create a reusable structure of test cases that you can copy to future iterations,
the Work Views page on the Plan menu is the best place to start.
In this example, we’ll start there.
First, let’s clear any views from previous visits to this page.
Select the Clear View option from the Saved Views drop-down list options.
Select Show Filters
We can leave the Projects value as is,
and specify the work item type that we want to show or create.
In this case, it’s a Test Set.
As with other work items, you’ll need at least editor permissions for your project to create a test set.
Select the +Add New button,
specify Test Set as the work item type,
enter the name (in this case, Browser Compatibility Regression Tests),
and select Create with Details.
Add description details,
assign a color, owner, and project.
You may specify a schedule state, and even a flow state for the Team Board view.
The delivery team can estimate the test set as part of iteration planning,
just like they do for user stories and defects.
Optionally, assign a tag to facilitate searching, grouping or filtering,
and to facilitate reuse, keep the release and iteration unscheduled.
You may also assign one or more milestones for this test set,
and indicate whether this is expedited work.
If any custom fields have been configured for test sets, you’ll see them here as well.
We don’t have any in this example.
Select Create to save the new test set.
Select the Tasks collection to add new tasks or copy existing tasks to a test set.
In this example, you might want to add a task for manual tests
and then another for automated tests if this matches your workflow.
Select the Test Cases icon to add new test cases or link existing test cases to this test set.
If you’ve done test planning, you can filter by test folder to find the test cases you want.
In the Use Rally Test Cases for Acceptance Testing training video,
we discuss creating test cases directly from a user story or defect.
The process is exactly the same when creating test cases directly from a test set.
Likewise, to remove a test case,
select it and then select Remove from the toolbar.
This will remove the association between the test case and the test set,
but it doesn’t delete the test case.
Test cases may be associated with more than one test set,
and in fact they will be any time they are run with different test sets in other iterations.
Let’s get a list of test sets using the Work Views page.
Select Test Set as the type of data we want to list.
Add the release and iteration columns.
This will help us determine if we’ll be able to copy a test set of interest.
If you want to create a repeatable set of tests and associated tasks to schedule in future iterations,
you can copy a test set as long as it's either unscheduled,
or it's scheduled in an iteration that has not yet been accepted.
To copy an eligible test set, select the checkbox and then select Copy from the toolbar.
Update the test set name
so that it can be easily reused.
Once you've created
your test set and scheduled it into an iteration,
you can complete each task in the set and execute each test case
by running through the test steps and recording results.
For each test case, record the results you got when you followed the steps.
Specify the build number, the verdict and the test set that the result is associated with
so that you can get statistics on your results for the test set.
In the same way that you can see the Test Run details for a user story or defect,
you can select the Test Run icon for the test set
to see the details and status of results for its test cases,
or filter results by Type of tests and Method.
By hovering over the Test Set Status
you can see more statistics about percentages of passing and failing tests,
and other types of test results.
Congratulations! Now you understand how to implement regression testing with Rally.
To learn more about Rally and agile practices, check out our blog and community.
Like this video and subscribe to our channel so that you’ll never miss new content as it’s added.