Leveraging the Wisdom of the Crowd in Software Testing: 1st Edition (Hardback) book cover

Leveraging the Wisdom of the Crowd in Software Testing

1st Edition

By Mukesh Sharma, Rajini Padmanaban

Auerbach Publications

180 pages | 36 B/W Illus.

Purchasing Options:$ = USD
Hardback: 9781482254488
pub: 2014-09-19
SAVE ~$12.99
eBook (VitalSource) : 9780429189722
pub: 2014-09-19
from $31.48

FREE Standard Shipping!


Its scale, flexibility, cost effectiveness, and fast turnaround are just a few reasons why crowdsourced testing has received so much attention lately. While there are a few online resources that explain what crowdsourced testing is all about, there’s been a need for a book that covers best practices, case studies, and the future of this technique.

Filling this need, Leveraging the Wisdom of the Crowd in Software Testing shows you how to leverage the wisdom of the crowd in your software testing process. Its comprehensive coverage includes the history of crowdsourcing and crowdsourced testing, implementation practices, and future trends.

The book discusses best practices in implementation—explaining what, when, and how to crowdsource in a testing effort. It also includes case studies that illustrate how both product and service companies have successfully applied crowdsourcing in their testing programs.

Explaining how to use the combined advantages of crowdsourcing and cloud computing for software testing, the book examines various engagement models in which you could implement crowdsourced testing. It addresses effective defect management in crowdsourced testing and considers both the business and engineering aspects of crowdsourced testing.

The book explores the challenges, limitations, and situations when crowdsourced testing will not work and provides powerful best practices for mitigating the constraints and challenges, including how to build a crowdsourcing platform to test software products. Covering career opportunities for crowd testers, the book concludes by taking a look at the need to build a crowdsourced testing ecosystem, who the players of such an ecosystem would be, and who would need to champion such an effort.


This book is unique in explaining how to combine the power of crowdsourcing with the craft of software testing to share practical experience, lessons learned, and provide guidance for others to hit the ground running. The rich set of examples provided here help set the context for all practitioners to help improve understanding of a fast-moving field. The depth of experience comes through with guidance on how to build a platform, create an ecosystem, and avoid the many potential problems and pitfalls that present a threat to successful crowdsource testing initiatives. The Did-you-Know conclusion offers pragmatic suggestions and offers a synopsis of advice to move forward on crowdsourced testing.

Quality is an amorphous, ephemeral condition, especially in software—and very hard to achieve in an increasingly online and mobile world. High quality requires meeting the needs and requirements of a broad and diverse set of people—and to do so requires a diverse set of analysts—a crowd—to help ensure everyone’s needs are met. This book offers a formula for helping everyone become successful in leveraging the crowd to improve the quality of the software experience.

Ross Smith, Testing Director, Skype Division, Microsoft Corporation

Table of Contents

Introducing the Crowd

What Is a Crowd?

History of Crowdsourcing

History of Crowdsourcing in Software Product Development

What Are the Traits of the Crowd That Make It Relevant to Be Engaged in Various Stages of Product Development?

Did You Know?

An Overview into Crowdsourcing

You Perform, We Provide the Platform

What Is Crowdsourcing?

Understanding the Varied Forms of Crowdsourcing

Crowd Creation

Crowd Voting

Crowd Wisdom

Crowd Funding

Is Crowdsourcing the Same as Outsourcing?

What Is Crowdsourced Testing and Where Does It Fit in This Picture?

Understanding Crowdsourced Testing through Examples

The Rising Popularity of Crowdsourcing

Did You Know?

Why Leverage the Crowd for Software Testing?

Are You Ready to Stand Out in a Crowd?

A Peek at Software Quality

Software Quality in the Current Day and Age

The Need for Alternate Solutions to Assure and Control Software Quality

Tying the Gap in Quality to the Definition of Crowdsourced Testing

A Case Study on Facebook Relevant in the World of Agile Operations


Challenges Posed by Scale



Leveraging Crowdsourced Testing in Various Testing Areas

Functional Testing

UI Testing

Usability Testing

Accessibility Testing

Performance Testing

Security Testing

Globalization Testing

Compatibility Testing

Did You Know?

Is Crowdsourced Testing a No-Brainer Solution to All Quality Problems?

What Not to Crowdsource in Your Test Effort

Challenges and Limitations of a Crowdsourced Test Effort

Did You Know?

How to Successfully Implement Crowdsourced Testing

Best Practices in Implementing a Successful Crowdsourced Test Effort

When Do We Crowdsource Our Testing Effort?

What Do We Crowdsource in Our Testing Effort?

How Do We Crowdsource Our Testing Effort?

Examples: Practical Implementations of Crowdsourced Testing

Engagement Models in Leveraging a Crowdsourced Test Effort

Did You Know?

Defect Management in Crowdsourced Testing

What Is Defect Management?

What Are Some Core Challenges in Defect Management in a Product Team?

Is Defect Management Any Different in a Crowdsourced Test Effort?

What Can Go Wrong If Defect Management Is Not Effective Specific to Crowdsourced Testing?

Can I Leverage Best Practices in Defect Management from a Formal Testing Effort in My Crowdsourced Testing?

Are Defect Tracking and Measurement Important in Crowdsourced Testing Too?

Are There Any Case Studies to Help Understand Crowdsourced Testing Defect Management in Real Time?

Did You Know?

Win Your Team’s Support in Implementing Crowdsourced Testing

Did You Know?

Let’s Understand Crowdsourced Testing Holistically

Chapter 1: Introducing the Crowd

Chapter 2: An Overview into Crowdsourcing

Chapter 3: Why Leverage the Crowd for Software Testing?

Chapter 4: Is Crowdsourced Testing a No-Brainer Solution to All Quality Problems?

Chapter 5: How to Successfully Implement Crowdsourced Testing

Chapter 6: Defect Management in Crowdsourced Testing

Chapter 7: Win Your Team’s Support in Implementing Crowdsourced Testing

Chapter 8: Let’s Understand Crowdsourced Testing Holistically

Additional Myths and Facts

Is Crowdsourced Testing a Viable Career to Build?

Who Can Become a Crowdsourced Tester?

What Should I Do to Prepare and Be Successful in Crowdsourced Testing Assignments?

Did You Know?

Future of Crowdsourced Testing

Sizing the Crowdsourcing Industry

Future of Crowdsourced Testing

Market Trends

Business Trends

Technology Trends

End User Trends

Did You Know?

Building an Ecosystem around Crowdsourced Testing

Crowd Testers

Crowd Seekers

The Platform

Common Knowledge Repository on Crowdsourced Testing

Tools to Help with the Crowdsourced Testing Effort

Who Is Responsible for Building This Ecosystem?

What to Watch for as We Let the Ecosystem Develop

Did You Know?


About the Authors

As founder and CEO, Sharma Mukesh started QA InfoTech with a vision to provide unbiased Quality Assurance solutions for business partners worldwide. Responsible for the company’s global operations, marketing, sales, and development efforts, QA InfoTech, under Mukesh’s leadership, has grown to five Centers of Excellence with over 700 employees. Mukesh is an active test evangelist, spreading the word on software testing through articles, conferences, and most recently his book on a career in testing. Through his passion for excellence, keen eye for detail, and customer commitment, QA InfoTech stands apart in the exceedingly competitive software testing industry.

As sr. director of engagement, Rajini Padmanaban leads engagement and relationship management for some of QA InfoTech's largest and most strategic accounts. She has more than 12 years of professional experience, primarily in the software quality assurance space. Rajini actively advocates software quality assurance through evangelistic activities including blogging on test trends, technologies and best practices, providing insights on software testing to analyst firms such as Gartner, IDC. She is also an active speaker in the Star conferences run by SQE.

Subject Categories

BISAC Subject Codes/Headings:
COMPUTERS / Information Technology
COMPUTERS / Software Development & Engineering / General
COMPUTERS / Internet / General