Usability Testing of Medical Devices covers the nitty-gritty of usability test planning, conducting, and results reporting. The book also discusses the government regulations and industry standards that motivate many medical device manufacturers to conduct usability tests.
Since publication of the first edition, the FDA and other regulatory groups have modified their regulations and expectations regarding how medical device manufacturers should approach usability testing. Reflecting these changes, this Second Edition provides updated guidance to readers with an interest or direct role in conducting a usability test of a medical device or system. Key updates involve the 2011 FDA guidance on human factors engineering, requirements set forth by the third edition of IEC 60601 and closely related IEC 62366-1:2015, linking usability test tasks to risk analysis results, and analyzing root causes of use errors that occur during usability tests.
Written by seasoned human factors specialists, Usability Testing of Medical Devices, Second Edition is an informative, practical, and up-to-date handbook for conducting usability tests of medical devices. The book helps ensure a smooth and painless development process—and thus, safe and effective medical devices.
Table of Contents
What is usability testing?
What is a medical device?
Why conduct usability tests of medical devices?
What is a use error?
What is a close call?
What is a difficulty?
What are common regulator comments on summative (i.e., validation) test plans?
Is usability testing of medical devices required?
Do you have to test minor design changes?
How do you defend usability testing methods to market researchers?
Risk management and usability testing
What is the relationship between usability testing and risk management?
Can usability testing identify use-related hazards?
What is a dangerous use error?
Is usability testing a reliable way to assess the likelihood that a dangerous use error will occur?
Do you have to evaluate every risk mitigation?
The commercial imperative
How does testing affect the development schedule?
Does usability testing offer liability protection?
Can you develop marketing claims based on test results?
What should a request for quotation for usability testing include?
What does a usability test cost?
What is the return on investment?
Anatomy of a usability test
What are the common elements of a usability test?
What is the proper duration of a test session?
Do you have to be a usability specialist to conduct a test?
Does it take a "brain surgeon" to evaluate medical devices?
Why test if you cannot change the design?
How do you set expectations?
What can postpone a usability test?
What ethical dilemmas might arise?
Types of tests
What is the difference between formative and summative usability testing?
What is a benchmark usability test?
What is an "out-of-the-box" usability test?
Can a test session include more than one participant?
Can you conduct a group test?
How do you conduct a "quick-and-dirty" usability test?
Are there special considerations when testing a "legacy" device?
Writing a test plan
What should a test plan include?
Does usability matter to regulators?
Do usability test plans require IRB approval?
How do you protect intellectual property?
Must a regulator approve a summative usability test plan?
Choosing a participant sample and recruiting participants
What is an appropriate sample size?
Can advisory panel members play a role in usability tests?
Should children participate in usability tests?
Should seniors participate in usability tests?
How do you test a device used jointly by a child and parent?
How do you conduct a usability test involving people with impairments?
How do you recruit test participants?
How do you recruit physicians?
How do you recruit nurses?
How do you recruit laypersons?
How do you prevent no-shows?
What does building a usability test laboratory entail?
What is the benefit of testing in a medical simulation facility?
How do you test in actual use environments?
How does testing a home health care device differ from testing a device used in clinical settings?
Should you test in a participant’s workplace?
Can you conduct a usability test over the Web?
Can you test a device while it is in actual use?
How do you conduct a nocturnal test?
What if a device cannot be moved?
Why and how do you distract test participants?
What use is a mannequin?
What role can a standardized patient play?
How do you simulate surgical procedures?
How do you simulate blood?
How do you simulate skin and injections?
How do you simulate impairments?
How do you simulate hardware interactions?
How do you simulate other medical devices?
Do you have to test everything?
What tasks should test participants perform?
How do you make tasks flow naturally?
Why focus on potentially dangerous tasks?
How do you choose tasks when evaluating use safety?
What if there are no high risks?
How do you assess risk control measures associated with very unlikely use errors?
How do you assess effectiveness?
Should tests include maintenance and service tasks?
Can you test long-term usability?
How do you test alarms?
How do you test warning labels?
How do you test instructions for use?
How do you test symbols?
How do you test legibility?
How do you evaluate packaging?
How do you test the appeal of a device?
How do you prioritize tasks?
Is there value in directing participants to repeat tasks?
Conducting the test
What is the value of pilot testing?
Who should observe the test sessions?
What kinds of usability problems arise during a usability test?
What can go wrong before, during, and after a test?
What risks do test personnel assume?
Are there times when the testing staff should be all female or all male?
Should user interface designers conduct usability tests of their own designs?
When and how should you assist test participants?
Can you modify a test in progress?
Can you reliably detect use errors?
Can you deliver training to test participants?
Should you provide access to learning tools?
Interacting with participants
When is it appropriate to ask participants to think aloud?
What is the proper way to pose a question?
Is there a place for humor in a usability test?
How do you minimize participant fatigue?
How do you protect participants from harm?
What if the test participant gets hurt?
Documenting the test
What data should you collect?
What use are task times?
What is a good way to video record a session?
How do you video record participants’ interactions with a moving device?
Analyzing test data
What kind of statistical analyses are most useful?
How do you handle outliers?
What role does root cause analysis play?
What makes a good test report?
Should test reports include design recommendations?
Can usability test results be misleading?
How do you deliver bad news?
How do you explain a lack of statistical significance?
What makes a good highlight video?
How does design validation differ from design verification?
Do you need to conduct a test prior to filing for an investigational device exemption?
Can a clinical trial supplant summative usability testing?
Can you conduct a usability test in parallel with a clinical trial?
Can you conduct a summative usability test without conducting a formative usability test?
Michael E. Wiklund is general manager of the human factors engineering (HFE) practice at UL-Wiklund, as well as professor of the practice at Tufts University, where he teaches courses on HFE. He has more than 30 years of experience in HFE, much of which has focused on medical technology development. His work has involved optimizing the safety, effectiveness, usability, and appeal of various products. Widely published, he is a certified human factors professional and one of the primary contributors to today’s most pertinent guidelines on the HFE of medical devices: AAMI HE75 and IEC 62366.
Jonathan Kendler is design director of UL-Wiklund’s human factors engineering (HFE) team. He has worked in the HFE profession since receiving his bachelor of fine arts degree in visual design from the School of the Museum of Fine Arts, Boston. He earned his master’s degree in human factors in information design from now Bentley University. His design portfolio ranges from small, handheld medical devices to room-size diagnostic scanners. In addition, he has co-taught applied software user interface design at Tufts University, and delivered HFE workshops to medical and nonmedical clients.
Allison Y. Strochlic is research director of UL-Wiklund’s human factors engineering (HFE) team. She earned her bachelor of science degree in HFE from Tufts University, and her master’s degree in human factors in information design from Bentley University. A board-certified human factors professional, she is a member of the Human Factors and Ergonomics Society New England Chapter, User Experience Professionals Association, and Association for the Advancement of Medical Instrumentation Home Use Environment Committee. She has served as a part-time lecturer at Tufts University, and delivered multiple presentations to industry and academic audiences.