Introduction

There are two challenges in designing a good set of tests for software:

  1. You need to minimize the number of tests while still providing strong coverage.
  2. You need to ensure that you are getting the right answer for the right reason.

For even a relatively simple system the number of possible test suites actually exceeds the number of molecules in the universe (which is 1080 according to Stephen Hawking in"A Brief History In Time"). The key challenge then is to select a small subset of tests which, if they run correctly, yield a very high degree of assurance that all of the other combinations/permutations will also run correctly.  


Definition

Pairwise Testing is a test design technique that delivers near 100 percent test coverage by narrowing down a large number of combinations/permutations to a small subset of those combinations/permutations to represent the entire set of combinations/permutations for testing. 


Pairs are created by the cross of each variable/state with each other variable/state done two variable/states at a time. These pairs are then merged into tests in such a way that each pair is included (i.e. covered) in at least one test.


The technique consists of the combined configuration of four components (Functions , Variables , States and Constraints).


Function

A Function contains a set of Variables and Constraints that define the total combination of pairs that will be reduced to a small subset of tests .


Variable

A Variable defines a name that gives meaning to a set of states. Examples:

  • Switch
  • Security Level
  • Action  

State

A State is a single value/constant that represents one possible state of a Variable. Two or more States are often necessary to represent all possible states of a Variable.  Examples:

  • Switch = {On, Off}
  • Security Level = {Red, Yellow, Green}
  • Action = {Sound Alarm, Sound Warning, Sound All Clear}


Constraint

A Constraint is a rule, that when applied across variables and states, ensures that combinations that are impossible to create in real life are not produced.  When constraints are not factored into the test design process, then a finite set of combinations/permutations that are defined, may contain tests that can not be physically created.  For example: for a given alarm system under test, if the state of its variable, Switch, is Off, then to test any state of its variables, Security Level and Action, would be impossible.


Within the GenRocket Pairwise Testing Platform, there is the option to choose from one of the five possible Constraints: 

  • Exclusive - Given options A, B, C..., at most one option may be true or all must be false.
  • Inclusive - Within a set of one or more options A, B, C..., at least one option must be true.
  • Mask - If A is in a given state, then B..Z either do not exist or they will be ignored.
  • One and Only One - In the set of A,B,C... one and only one may be true, but never all false; thus, there will always be just one true.
  • Requires - If A is true then B must also be True; it's a dependent relationship.  Requires is a uni-directional constraint.