Many enterprises regard system-level testing as the final piece of the development effort, rather than as a tool that should be integrated throughout the development process. As a consequence, test teams often execute critical test plans just before product launch, resulting in much of the corrective work being performed in a rush and at the last minute.
Presenting combinatorial approaches for improving test coverage, Testing Complex and Embedded Systems details techniques to help you streamline testing and identify problems before they occur—including turbocharged testing using Six Sigma and exploratory testing methods. Rather than present the continuum of testing for particular products or design attributes, the text focuses on boundary conditions. Examining systems and software testing, it explains how to use simulation and emulation to complement testing.
- Details how to manage multiple test hardware and software deliveries
- Examines the contradictory perspectives of testing—including ordered/ random, structured /unstructured, bench/field, and repeatable/non repeatable
- Covers essential planning activities prior to testing, how to scope the work, and how to reach a successful conclusion
- Explains how to determine when testing is complete
Where you find organizations that are successful at product development, you are likely to find groups that practice disciplined, strategic, and thorough testing. Tapping into the authors’ decades of experience managing test groups in the automotive industry, this book provides the understanding to help ensure your organization joins the likes of these groups.
Does Your Testing Look Like This?
Last-Minute Flailing
Fiascos Uncovered Weeks before Launch
Huge Warranty Problems
Customer Dissatisfaction
Benefits of Improved Testing
Product Problems Revealed Early
Improved Reliability = Lower Cost
Happy Customers
Confidence in a Fine Product
Cost-Effective Testing Solutions Not Waiting Until
the Last Minute
Overview
Goals of Testing
Types of Testing
Levels of Testing
Basic Principles
Looking at the Evidence
The Question
Not Phrases
Instead
Contradictory Perspectives of Testing
Organic/Inorganic
Quantitative/Qualitative
Objective/Subjective
Deterministic/Probabilistic
Variable/Attribute
Continuous/Discrete
Wide-Ranging/Focused
Many/Few
Structure/Unstructured
Ordered/Random
Nominal/Overstress
Single Environment/Multiple Environments
Compliance/Characterization
High-Level/Detailed
Growing/Imposing
Bench/Field
Abstraction/Verisimilitude
Reproducible/Nonreproducible
Repeatable/Nonrepeatable
Linear/Nonlinear
Fine/Coarse
Combinatorial/Exhaustive/Stochastic
Focused/Fuzzy
Instant/Enduring
Inside System/Outside System
Ambiguity/Clarity
Sensitive/Insensitive
Long-Range/Short-Range
Costly/Cheap
Flexible/Inflexible
Parameter/Tolerance
Standard/Failure
"Good"/"Bad" Testing
Parallel/Sequential
Fractal/Smooth/Ordered
The Use of Noise
Realistic
Can Use Taguchi Approach
Sometimes Difficult to Simulate
Where to Apply Noise?
Exogenous Shocks
Self-Generated Noise
How to Perform "Bad’’ Tests
Do Not
Do
Documenting the Testing
Components of a Test Plan
Components of a Test Report
The DVP&R Format
Failures
Test Administration
Test Management
Test Scheduling
Test Human Resources
Test Device Resources
Test Quality
Test Costing
Test Risk
Calibration Issues
Advanced Concepts
Test Impacts
Construct Validity
Types of Bias
Reliability and Confidence
Life Testing
Testing Team Early Involvement
Software Test Documentation
IEEE
Defect Life Cycle
Configuration Management
Configuration Management Areas
Planning
Elements of Configuration Management
Importance of Configuration Management
Links to Testing
Risks
Software Testing
Overview
Software Testing—The Problem
Test Metrics
Software Boundary Testing
Static Code Analysis
Dynamic Code Analysis
Systems Testing
End-to-End Testing
Big-Bang Testing
Top-Down Testing
Bottom-Up Testing
Managing Multiple Test Hardware
and Software Deliveries
Test Configuration Management
System Variations and Test Demands
Functional Testing
System Response to Multiple System Failures
Ranges of System Performance
Simulation and Emulation
Simulation
Simulation Levels
Simulation Activities
Objectives of Simulation
Simulation as Verification
Simulation as Test Preparation
Conflict between Simulation and Test Results
Span of Tests
Software
Unit Test
Component Test
Subsystem-Level Integration Testing
System
Production Test
Static Analysis
Structural Analysis
Simulation
Prototyping and Simulation
Reliability and Confidence
Limitations of Reliability and Confidence
Concept of "Life" and the Product
Establishing Product Life Exposure
Exit Criteria
When Is Enough, Enough?
Compared to Plan?
Testing Ethics
Final Words
Bibliography
Index
Biography
Kim H. Pries, Jon M. Quigley