Pairwise Testing

Pairwise Testing is very interesting and pretty cool thing in black box testing. It enables testers to find bugs effectively and efficiently by saving time and effort. It’s a technique that helps to convert large number of test cases (exhaustive testing) to minimal.

Sometimes equivalence partitioning and boundary value analysis techniques may not uncover defects in the application under test.  Let us take an example.

Consider a vehicle which is capable of travelling on ground, on air and in water, in manual and automatic mode. Below is the maximum speed limit:

On ground maximum speed is 150 km/hr., in air 200km/hr. and in water the maximum travel speed is 100km/hr.

To test the above scenario for bugs, for exhaustive testing the number of test case required is 18 i.e. 2*3*3. It is not possible to test vehicle using all possible combinations, as it takes more effort, time and money for fuel expenses.

Table 1 depicts the total number of exhaustive test cases for testing.

EXHAUSTIVE TESTCASES

TESTCASE ID

MODE

MEDIUM

SPEED

1 Manual Air 100
2 Manual Air 150
3 Manual Air 200
4 Manual Water 100
5 Manual Water 150
6 Manual Water 200
7 Manual Ground 100
8 Manual Ground 150
9 Manual Ground 200
10 Automatic Air 100
11 Automatic Air 150
12 Automatic Air 200
13 Automatic Water 100
14 Automatic Water 150
15 Automatic Water 200
16 Automatic Ground 100
17 Automatic Ground 150
18 Automatic Ground 200

Table 1

In order to avoid more number of test cases and to find bugs by executing minimum number of test cases, pairwise testing approach comes in.

Pairwise testing principle states that most bugs are caused due to interactions of 2 factors. Table2 depicts test cases generated by combining 2 factors using PICT by Microsoft.

COMBINATION DEPTH is 2

TESTCASE ID

MODE

MEDIUM

SPEED

1 Manual Ground 100
2 Manual Water 200
3 Automatic Air 100
4 Manual Air 150
5 Automatic Water 100
6 Automatic Air 200
7 Automatic Water 150
8 Automatic Ground 150
9 Manual Ground 200

Table 2

For the above scenario, using pairwise testing, the number of test cases (effort) can be reduced to 50% in comparison to exhaustive test cases.

Chart 1 shows pictorial view of comparison between Exhaustive and Pairwise test cases count.

 Chart 1

If you go for 3 or n factors then there will be increase in number of test cases. Table 3 in which test cases have been generated using 3 factors combination.

COMBINATION DEPTH is 3

TESTCASE ID

MODE

MEDIUM

SPEED

1 Manual Water 100
2 Automatic Air 100
3 Manual Water 150
4 Manual Ground 150
5 Automatic Ground 150
6 Automatic Water 150
7 Automatic Water 100
8 Manual Air 100
9 Manual Ground 100
10 Manual Air 200
11 Automatic Water 200
12 Automatic Air 150
13 Automatic Air 200
14 Manual Air 150
15 Manual Water 200
16 Automatic Ground 100
17 Automatic Ground 200
18 Manual Ground 200

Table 3

Now, next question which comes to your mind is, will Pairwise testing approach uncover all the bugs? We will see this briefly.

There are some instances where Pairwise testing might fail or it will not be able to find bugs for your scenario under execution. Following are the instances where pairwise testing might fail to identify bug.         

  1. Pairwise Testing will not identify all important and critical combinations. Choose right values yourself.
  2. Understand your data. Pairwise testing may not fit for all data sets.
  3. If 2-way is not able to identify faults for your data sets, higher order combinations like 3-way/4-way might be necessary.

Most important consideration is while implementing pairwise testing, is that there are no best practices to uncover faults.

Conclusion

Tester needs to pay more attention while designing test cases using pairwise testing approach. It is important to understand the data and the way it interacts and care should be taken while assigning priority.


Available tools for pairwise testing

http://download.microsoft.com/download/f/5/5/f55484df-8494-48fa-8dbd-8c6f76cc014b/pict33.msi

Further Reading and References

About the Author

3 thoughts on “Pairwise Testing”

  1. NaveenKumar,

    Nice article about an important topic that too few software testers know about. This test design approach tends to improve software testing efficiency (because it prevents a great deal of wasteful repetition in software tests) and improve software testing effectiveness (because it closes a large number of gaps in coverage that would otherwise exist).

    You may find a couple of these links useful to:

    1) A photograph-rich, text-light presentation on both pairwise / allpairs testing and more thorough combinatorial software testing: http://www.slideshare.net/JustinHunter/combinatorial-software-testdesignbeyondpairwisetesting

    2) A more scholarly but still practical introduction to pairwise and more thorough related test design methods that I co-wrote with 3 PhD’s: http://hexawise.com/combinatorial-and-pairwise-testing

    3) Our Hexawise testing tool (which has a starting price point of $50,000/ year for corporate licenses but which we give away for free – with no small print – to thousands of users); any companies who want to use 5 or fewer licenses don’t pay: http://www.hexawise.com/free

    Thanks.

    – Justin

    Reply
  2. Hello Naveen,

    What is the relationship between the number of test cases and the parameters + cardinality + order of combination (strength) in Pairwise test?

    I am using the Microsoft PICT to generate Pairwise tests. For 4 parameters with cardinality of 5, 4, 4, 4 it generates 24 test cases for test strength of 2 (pairwise). For test strength of 3, it generates 115 test. How do I mathematically calculate the number of tests it will generate?

    Thanks

    Reply

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Hamster - Launch JMeter Recent Test Plans SwiftlyDownload for free
+
Share via
Copy link