Pairwise Testing

Show us your love by sharing this post.

Pairwise Testing is very interesting and pretty cool thing in black box testing. It enables testers to find bugs effectively and efficiently by saving time and effort. It’s a technique that helps to convert large number of test cases (exhaustive testing) to minimal.

Sometimes equivalence partitioning and boundary value analysis techniques may not uncover defects in the application under test.  Let us take an example.

Consider a vehicle which is capable of travelling on ground, on air and in water, in manual and automatic mode. Below is the maximum speed limit:

On ground maximum speed is 150 km/hr., in air 200km/hr. and in water the maximum travel speed is 100km/hr.

To test the above scenario for bugs, for exhaustive testing the number of test case required is 18 i.e. 2*3*3. It is not possible to test vehicle using all possible combinations, as it takes more effort, time and money for fuel expenses.

Table 1 depicts the total number of exhaustive test cases for testing.

EXHAUSTIVE TESTCASES

TESTCASE ID

MODE

MEDIUM

SPEED

1ManualAir100
2ManualAir150
3ManualAir200
4ManualWater100
5ManualWater150
6ManualWater200
7ManualGround100
8ManualGround150
9ManualGround200
10AutomaticAir100
11AutomaticAir150
12AutomaticAir200
13AutomaticWater100
14AutomaticWater150
15AutomaticWater200
16AutomaticGround100
17AutomaticGround150
18AutomaticGround200

Table 1

In order to avoid more number of test cases and to find bugs by executing minimum number of test cases, pairwise testing approach comes in.

Pairwise testing principle states that most bugs are caused due to interactions of 2 factors. Table2 depicts test cases generated by combining 2 factors using PICT by Microsoft.

COMBINATION DEPTH is 2

TESTCASE ID

MODE

MEDIUM

SPEED

1ManualGround100
2ManualWater200
3AutomaticAir100
4ManualAir150
5AutomaticWater100
6AutomaticAir200
7AutomaticWater150
8AutomaticGround150
9ManualGround200

Table 2

For the above scenario, using pairwise testing, the number of test cases (effort) can be reduced to 50% in comparison to exhaustive test cases.

Chart 1 shows pictorial view of comparison between Exhaustive and Pairwise test cases count.

 Chart 1

If you go for 3 or n factors then there will be increase in number of test cases. Table 3 in which test cases have been generated using 3 factors combination.

COMBINATION DEPTH is 3

TESTCASE ID

MODE

MEDIUM

SPEED

1ManualWater100
2AutomaticAir100
3ManualWater150
4ManualGround150
5AutomaticGround150
6AutomaticWater150
7AutomaticWater100
8ManualAir100
9ManualGround100
10ManualAir200
11AutomaticWater200
12AutomaticAir150
13AutomaticAir200
14ManualAir150
15ManualWater200
16AutomaticGround100
17AutomaticGround200
18ManualGround200

Table 3

Now, next question which comes to your mind is, will Pairwise testing approach uncover all the bugs? We will see this briefly.

There are some instances where Pairwise testing might fail or it will not be able to find bugs for your scenario under execution. Following are the instances where pairwise testing might fail to identify bug.         

  1. Pairwise Testing will not identify all important and critical combinations. Choose right values yourself.
  2. Understand your data. Pairwise testing may not fit for all data sets.
  3. If 2-way is not able to identify faults for your data sets, higher order combinations like 3-way/4-way might be necessary.

Most important consideration is while implementing pairwise testing, is that there are no best practices to uncover faults.

Conclusion

Tester needs to pay more attention while designing test cases using pairwise testing approach. It is important to understand the data and the way it interacts and care should be taken while assigning priority.


Available tools for pairwise testing

http://download.microsoft.com/download/f/5/5/f55484df-8494-48fa-8dbd-8c6f76cc014b/pict33.msi

Further Reading and References

2 thoughts on “Pairwise Testing”

  1. NaveenKumar,

    Nice article about an important topic that too few software testers know about. This test design approach tends to improve software testing efficiency (because it prevents a great deal of wasteful repetition in software tests) and improve software testing effectiveness (because it closes a large number of gaps in coverage that would otherwise exist).

    You may find a couple of these links useful to:

    1) A photograph-rich, text-light presentation on both pairwise / allpairs testing and more thorough combinatorial software testing: http://www.slideshare.net/JustinHunter/combinatorial-software-testdesignbeyondpairwisetesting

    2) A more scholarly but still practical introduction to pairwise and more thorough related test design methods that I co-wrote with 3 PhD’s: http://hexawise.com/combinatorial-and-pairwise-testing

    3) Our Hexawise testing tool (which has a starting price point of $50,000/ year for corporate licenses but which we give away for free – with no small print – to thousands of users); any companies who want to use 5 or fewer licenses don’t pay: http://www.hexawise.com/free

    Thanks.

    – Justin

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

New Year. New Hope. Learn TruWeb. 78% OFF for Limited Time of my Udemy course..BUY NOW
× How can I help you?