Close

October 23, 2012

Pairwise Testing

Pairwise Testing is very interesting and pretty cool thing in black box testing. It enables testers to find bugs effectively and efficiently by saving time and effort. It’s a technique that helps to convert large number of test cases (exhaustive testing) to minimal.

Sometimes equivalence partitioning and boundary value analysis techniques may not uncover defects in the application under test.  Let us take an example.

Consider a vehicle which is capable of travelling on ground, on air and in water, in manual and automatic mode. Below is the maximum speed limit:

On ground maximum speed is 150 km/hr., in air 200km/hr. and in water the maximum travel speed is 100km/hr.

To test the above scenario for bugs, for exhaustive testing the number of test case required is 18 i.e. 2*3*3. It is not possible to test vehicle using all possible combinations, as it takes more effort, time and money for fuel expenses.

Table 1 depicts the total number of exhaustive test cases for testing.

EXHAUSTIVE TESTCASES

TESTCASE ID

MODE

MEDIUM

SPEED

1 Manual Air 100
2 Manual Air 150
3 Manual Air 200
4 Manual Water 100
5 Manual Water 150
6 Manual Water 200
7 Manual Ground 100
8 Manual Ground 150
9 Manual Ground 200
10 Automatic Air 100
11 Automatic Air 150
12 Automatic Air 200
13 Automatic Water 100
14 Automatic Water 150
15 Automatic Water 200
16 Automatic Ground 100
17 Automatic Ground 150
18 Automatic Ground 200

Table 1

In order to avoid more number of test cases and to find bugs by executing minimum number of test cases, pairwise testing approach comes in.

Pairwise testing principle states that most bugs are caused due to interactions of 2 factors. Table2 depicts test cases generated by combining 2 factors using PICT by Microsoft.

COMBINATION DEPTH is 2

TESTCASE ID

MODE

MEDIUM

SPEED

1 Manual Ground 100
2 Manual Water 200
3 Automatic Air 100
4 Manual Air 150
5 Automatic Water 100
6 Automatic Air 200
7 Automatic Water 150
8 Automatic Ground 150
9 Manual Ground 200

Table 2

For the above scenario, using pairwise testing, the number of test cases (effort) can be reduced to 50% in comparison to exhaustive test cases.

Chart 1 shows pictorial view of comparison between Exhaustive and Pairwise test cases count.

 Chart 1

If you go for 3 or n factors then there will be increase in number of test cases. Table 3 in which test cases have been generated using 3 factors combination.

COMBINATION DEPTH is 3

TESTCASE ID

MODE

MEDIUM

SPEED

1 Manual Water 100
2 Automatic Air 100
3 Manual Water 150
4 Manual Ground 150
5 Automatic Ground 150
6 Automatic Water 150
7 Automatic Water 100
8 Manual Air 100
9 Manual Ground 100
10 Manual Air 200
11 Automatic Water 200
12 Automatic Air 150
13 Automatic Air 200
14 Manual Air 150
15 Manual Water 200
16 Automatic Ground 100
17 Automatic Ground 200
18 Manual Ground 200

Table 3

Now, next question which comes to your mind is, will Pairwise testing approach uncover all the bugs? We will see this briefly.

There are some instances where Pairwise testing might fail or it will not be able to find bugs for your scenario under execution. Following are the instances where pairwise testing might fail to identify bug.         

  1. Pairwise Testing will not identify all important and critical combinations. Choose right values yourself.
  2. Understand your data. Pairwise testing may not fit for all data sets.
  3. If 2-way is not able to identify faults for your data sets, higher order combinations like 3-way/4-way might be necessary.

Most important consideration is while implementing pairwise testing, is that there are no best practices to uncover faults.

Conclusion

Tester needs to pay more attention while designing test cases using pairwise testing approach. It is important to understand the data and the way it interacts and care should be taken while assigning priority.


Available tools for pairwise testing

http://download.microsoft.com/download/f/5/5/f55484df-8494-48fa-8dbd-8c6f76cc014b/pict33.msi

Further Reading and References

About NaveenKumar Namachivayam

Hi there! is a passionate and experienced Performance Test Analyst based in Chicago, USA with skill sets in Manual Testing, Test Automation, Performance Testing, and Test Estimation. He loves to learn and experiment new trends and models in Software Testing. Please stop by at his personal blogs: Excel Blog, DealsBrook, NaveenKumarN.in, and Affiliate Insights blog and Software Testing Memes.

2 Comments on “Pairwise Testing

Justin Hunter
October 30, 2012 at 8:58 pm

NaveenKumar,

Nice article about an important topic that too few software testers know about. This test design approach tends to improve software testing efficiency (because it prevents a great deal of wasteful repetition in software tests) and improve software testing effectiveness (because it closes a large number of gaps in coverage that would otherwise exist).

You may find a couple of these links useful to:

1) A photograph-rich, text-light presentation on both pairwise / allpairs testing and more thorough combinatorial software testing: http://www.slideshare.net/JustinHunter/combinatorial-software-testdesignbeyondpairwisetesting

2) A more scholarly but still practical introduction to pairwise and more thorough related test design methods that I co-wrote with 3 PhD’s: http://hexawise.com/combinatorial-and-pairwise-testing

3) Our Hexawise testing tool (which has a starting price point of $50,000/ year for corporate licenses but which we give away for free – with no small print – to thousands of users); any companies who want to use 5 or fewer licenses don’t pay: http://www.hexawise.com/free

Thanks.

– Justin

Reply
NaveenKumar
November 1, 2012 at 11:49 am

Many thanks Justin for your information. I will check it out. Thanks!

Reply

Leave a Reply

Your email address will not be published. Required fields are marked *