After building & validating the testing models several test cases are generated. The next biggest task is to decide the priority for executing them by using some systematic procedure.
The process begins with identification of "Static Test Cases" and "Dynamic Test Runs", brief introduction of which is as under.
Test case: It is a collection of several items and corresponding information, which enables a test to be executed or performing a test run.
Test Run: It is a dynamic part of the specific testing activities in the overall sequence of testing on some specific testing object.
Every time we invoke a static test case, we in-turn perform an individual dynamic test run. Hence we can say that, every test case can correspond to several test runs.
Why & how do we prioritize?
Out of a large cluster of test cases in our hand, we need to scientifically decide their priorities of execution based upon some rational, non-arbitrary, criteria. We carry out the prioritization activity with an objective to reduce the overall number of test cases in the total testing feat.
There are couples of risks associated with our prioritization activities for the test cases. We may have the risk that some of the application features may not undergo testing at all.
During prioritization we work out plans addressing following two key concepts:
Concept – 1: Identify the essential features that must be tested in any case.
Concept – 2: Identify the risk or consequences of not testing some of the features.
The decision making in selecting the test cases is largely based upon the assessment of the risk first.
The objective of the test case prioritization exercise is to build confidence among the testers and the project leaders that the tests identified for execution are adequate from different angles.
The list of test cases decided for execution can be subjected to n-number of reviews in case of doubts / risks associated with any of the omitted tests.
Following four schemes are quite common for prioritizing the test cases.
All these methods are independent of each other & are aimed at optimizing the number of test cases. It is difficult to brand either of the methods better than the other. We can use any one method as a standalone scheme or can be used in conjunction with another one. When we get similar results out of different prioritization schemes, level of confidence increases.
Scheme – 1: Categorization of Priority.
Scheme – 2: Risk analysis.
Scheme – 3: Brainstorming to dig out the problematic areas.
Scheme – 4: Combination of different schemes.
Let us discuss the priority categorization scheme in greater detail here.
Easiest of all methods for categorizing our tests is to assign a priority code directly to every test description. This involves assigning a unique number to each & every test description.
A popular three-level priority categorization scheme is described as under
Priority - 1: Allocated to all tests that must be executed in any case.
Priority - 2: Allocated to the tests which can be executed, only when time permits.
Priority - 3: Allocated to the tests, which even if not executed, will not cause big upsets.
After assignment of priority codes, the tester estimates the amount of time required to execute the tests selected in each category. In case the estimated time happens to lie within the allotted schedule, means successful identification of tests & completion of the partitioning exercise. In case of any deviation of time plans, partitioning exercise is carried out further.
There is another extension to the above scheme i.e. new five-level scale using which we can classify the test priorities further.
The Five-Level Priority scheme is as under
Priority-1a: Allocated to the tests, which must pass, otherwise the delivery date will be affected.
Priority-2a: Allocated to the tests, which must be executed before the final delivery.
Priority-3a: Allocated to the tests which can be executed, only when time permits.
Priority-4a: Allocated to the tests, which can wait & can be executed even after the delivery date.
Priority-5a: Allocated to the tests, which have remote probability of execution ever.
Testers plan to divide the tests in various categories. For instance, say tests from priority 2 are further divided among priority levels like 3a, 4a and 5a. Likewise any test can be downgraded or upgraded.
Other considerations used while prioritizing or sequencing the test cases
a) Relative Dependencies: Some test cases are such that they can run only after the others because the one is used to set up the other. This is applicable especially for continuously operating systems involving test run to start from a state created by the previous one.
b) Timings of defect detection: Applies to cases wherein the problems can be detected only when many other problems have been found and already fixed. For example it applies to integration testing involving many components having their own problems at individual components level.
c) Damage or accidents: Applies to cases wherein acute problems or even severe damages can happen during testing unless some critical areas had not been checked before the present test run. For example it applies to embedded software involving safety critical systems, wherein the testers would not prefer to start testing the safety features prior to first testing the other related functions.
d) Difficulty levels: This is one of the most natural & commonly used sequence to execute the test cases involving moving from simple & easy test cases to difficult and complicated ones. This applies to scenarios where complicated problems can be expected. Here the testers prefer to execute comparatively simpler test cases first to narrow down the problematic areas.
5) Combining the test cases: Applies to majority of cases in large-scale software testing exercises involving interleaving and parallel testing to accelerate the testing process.
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment