Introduction to Combinatorial Testing: 1st Edition (Paperback) book cover

Introduction to Combinatorial Testing

1st Edition

By D. Richard Kuhn, Raghu N. Kacker, Yu Lei

Chapman and Hall/CRC

341 pages | 92 B/W Illus.

Purchasing Options:$ = USD
Paperback: 9781466552296
pub: 2013-06-20
SAVE ~$15.59
eBook (VitalSource) : 9780429185151
pub: 2016-04-19
from $37.48

FREE Standard Shipping!


Combinatorial testing of software analyzes interactions among variables using a very small number of tests. This advanced approach has demonstrated success in providing strong, low-cost testing in real-world situations. Introduction to Combinatorial Testing presents a complete self-contained tutorial on advanced combinatorial testing methods for real-world software.

The book introduces key concepts and procedures of combinatorial testing, explains how to use software tools for generating combinatorial tests, and shows how this approach can be integrated with existing practice. Detailed explanations and examples clarify how and why to use various techniques. Sections on cost and practical considerations describe tradeoffs and limitations that may impact resources or funding. While the authors introduce some of the theory and mathematics of combinatorial methods, readers can use the methods without in-depth knowledge of the underlying mathematics.

Accessible to undergraduate students and researchers in computer science and engineering, this book illustrates the practical application of combinatorial methods in software testing. Giving pointers to freely available tools and offering resources on a supplementary website, the book encourages readers to apply these methods in their own testing projects.


"I was prepared at the outset to dislike this book. I expected a different subject. It was written by researchers who may or may not have been able to relate to practitioners. It was about a subject that seemed virtually impossible to make practical. But in the end, it turned out to be on a quite different subject, and the authors overcame all my concerns. I thoroughly recommend it to anyone involved in the practice of software testing."

—Computing Reviews, March 2014

Table of Contents

Combinatorial Methods in Testing

Software Failures and the Interaction Rule

Two Forms of Combinatorial Testing

Covering Arrays

The Test Oracle Problem

Quick Start: How to Use the Basics of Combinatorial Methods Right Away

Combinatorial Testing Applied

Document Object Model, Carmelo Montanez-Rivera, D. Richard Kuhn, Mary Brady, Rick Rivello, Jenise Reyes Rodriguez, and Michael Powers

Rich Web Applications, Chad M. Maughan

Configuration Testing

Runtime Environment Configurations

Highly Configurable Systems and Software Product Lines

Invalid Combinations and Constraints

Cost and Practical Considerations

Input Testing

Partitioning the Input Space

Input Variables versus Test Parameters

Fault Type and Detectability

Building Tests to Match an Operational Profile

Scaling Considerations

Cost and Practical Considerations

Test Parameter Analysis, Eduardo Miranda

What Should Be Included as a Test Parameter

Combination Anomalies

Classification Tree Method

Modeling Methodology

Selecting the System under Test

Combinatorial Testing and Boundary Value Analysis

Managing System State, George Sherwood

Test Fact or Partitions with State

Test Fact or Simplifications

Sequence Unit Replay Model

Single Region State Models

Multiple Region State Models

Measuring Combinatorial Coverage

Software Test Coverage

Combinatorial Coverage

Using Combinatorial Coverage

Cost and Practical Considerations

Analysis of (t + 1)-Way Coverage

Test Suite Prioritization by Combinatorial Coverage, Renee Bryce and Sreedevi Sampath

Combinatorial Coverage for Test Suite Prioritization


Prioritization Criteria

Review of Empirical Studies

Tool: Combinatorial-Based Prioritization for User-Session-Based Testing

Other Approaches to Test Suite Prioritization Using Combinatorial Interactions

Cost and Practical Considerations

Combinatorial Testing and Random Test Generation

Coverage of Random Tests

Adaptive Random Testing

Tradeoffs: Covering Arrays and Random Generation

Cost and Practical Considerations

Sequence-Covering Arrays

Sequence-Covering Array Definition

Size and Construct ion of Sequence-Covering Arrays

Using Sequence-Covering Arrays

Cost and Practical Considerations

Assertion-Based Testing

Basic Assertions for Testing

Stronger Assertion-Based Testing

Cost and Practical Considerations

Model-Based Testing


Access Control System Example

SMV Model

Integrating Combinatorial Tests into the Model

Generating Tests from Counterexamples

Cost and Practical Considerations

Fault Localization

Fault Localization Process

Locating Faults: Example

Cost and Practical Considerations

Evolution from Design of Experiments


Pairwise (Two-Way) Testing of Software Systems

Combinatorial t-Way Testing of Software Systems

Algorithms for Covering Array Construction, Linbin Yu and Yu Lei


Algorithm AETG

Algorithm IPOG

Cost and Practical Considerations

Appendix A: Mathematics Review

Appendix B: Empirical Data on Software Failures

Appendix C: Resources for Combinatorial Testing

Appendix D: Test Tools



A Summary and Review appear at the end of each chapter.

About the Series

Chapman & Hall/CRC Innovations in Software Engineering and Software Development Series

Learn more…

Subject Categories

BISAC Subject Codes/Headings:
COMPUTERS / Programming / Games
COMPUTERS / Programming Languages / General
COMPUTERS / Software Development & Engineering / General