Navigating Human Factors Validation Studies

Medical device manufacturers are more than proficient when it comes to innovating and developing new technologies. But alongside this focus, device manufacturers must now navigate the regulation of human factors activities to be conducted during device development. FDA draft guidance on human factors and usability engineering provides manufacturers with insight to regulators’ current thinking about the application of standards and other reference documents such as ISO/IEC 62366, HE 75, and ISO 14971 to the device development process. However, even with this guidance, it can be challenging for manufacturers to go it alone. And when it comes to approving devices,  FDA has given manufacturers a clear ultimatum: Any use error attributable to device design will not be tolerated. 

This means that now, more than ever, manufacturers must focus on designing devices that are intuitive to the intended users—devices that will not create dangers for patients under normal use conditions. One of the best things manufacturers can do to set themselves up for success is leverage formative research during human factors validation testing and production.

Part Art, All Science

Validation tests are very different from formative research. During formative research, we may say there are no right or wrong answers, and researchers may use any number of methods to gather information about the use of a device. We track success rates for key tasks during formative usability tests, but the true focus is on learning about users’ experiences with a device. We strive to understand not only how users currently use the device, but also how they would like to use it and what impact environmental factors might have on the use of the device. Getting answers during formative research is as much art as science. Eliciting insights from research participants and identifying opportunities for improvements requires researchers to adapt and respond to participants’ behavior. 

However, in validation tests there are particular methods to use as well as right and wrong answers. These are actually not so much answers as observed behavior. Participants in a validation test either execute a task successfully without introducing any risks to the simulated patient or they fail to do so. A validation test requires a much more rigorous design that takes into account the specifics of who should be conducting tasks, how those tasks should be administered, and under what environmental conditions the tasks should be attempted.

To construct a validation test that will account for all of these factors, we have to view the formative research not as an isolated event, but rather as the first part in the overall scientific process. The structure of the validation test is as dependent on the outcomes of formative research as the design of the device itself. 

When conducting formative research, a manufacturer examines and answers the who, what, where, and how that surround the device’s use: Who are the users? What will the device be used for? Where will they use it? And how will it be used?

But it’s not good enough to simply know or divine the answers. Manufacturers need to successfully transition those outputs from formative research into the inputs for a human factors validation study and production of an intuitive device. That’s why the outcomes from formative research are so important—they make the fundamental case for why a device includes particular functions or constraints, why those functions or constraints are designed in a particular fashion, and why they are appropriate for particular users, uses, and use environments.  

Research Under Any Other Name

During initial conversations with clients, we’ll frequently hear that they have done little by way of formative research other than an occasional usability test. Formative research can take many forms, and in our experience manufacturers often sell themselves short when it comes to acknowledging the work that has been done. When we begin to pry, we soon learn that the manufacturer actually has a good start on gathering relevant information, even if that information may be distributed within their organization.

For example, take the case of a manufacturer  that wished to conduct a human factors validation test to demonstrate safe and effective use of its new infusion pump. The manufacturer indicated that it had conducted several usability tests internally throughout the development process to inform the design of the pump. However, when asked to support its definition of critical tasks, distinct user groups, and representative use environments for the validation study, the manufacturer was uncertain how to go about doing that. After further discussions, we helped the manufacturer realize that as an organization it had in fact conducted a number of formative research activities that could be leveraged to inform and support the design of a robust human factors validation test. The manufacturer had performed the following activities:

These are all steps in formative research that should be considered when designing a human factors validation test—the manufacturer just didn’t realize the importance of leveraging this information during the validation phase. Formative research is often misinterpreted as only formative usability testing, which is conducted primarily to inform the design of a more usable and useful device. Usability testing is a critical step in formative research but is not typically sufficient to inform the design of a proper validation test. 

The key to getting the most out of all this formative research is not just doing it but documenting the outcomes and leveraging those outcomes appropriately when it comes time for human factors validation. That way, when confronted with questions the researcher or manufacturer can cite the specific outcomes that led to the definition of relevant parameters, instead of speaking in general terms. A complete human factors validation report should include references to outcomes from formative research to justify the design of the validation study and preempt any questions about assumptions or definitions. As long as we can provide justification for why we test certain demographics, why research participants will be attempting certain tasks, and why the participants are being tested in a certain environment, we have the baseline we need to start the validation study.  

In formative research, manufacturers are not just making changes to a device, but also beginning to design a study that will validate that the device can be used safely and effectively according to the best-available information. Outputs from formative research become the inputs to the design of validation tests—equally as important as the plastic, metal, and electronic components from which the device is constructed.

Documenting and approaching formative research as the first step in the human factors validation process required by FDA will increase the probability of a successful validation study. Although the regulation of human factors validation in medical device design requires manufacturers to spend more time and effort during development, the results more than make up for the energy expended. 

When this process is followed properly, safer and more effective medical devices are produced, reducing risks to patient safety and health. In addition to safety and effectiveness, medical device manufacturers will ultimately design a device that is more intuitive, which will expand the shelf life of the device and make it more marketable to users.

Korey Johnson is associate director at GfK User Centric (Chicago), a global user experience research and design firm. He has a master's degree in psychology with an emphasis in human factors from University of Idaho. 

Gavin Lew is executive vice president at GfK User Centric. He holds a master's degree in experimental psychology from Loyola University Chicago.

Related Content

Case Study: Applying Human Factors for User Interface Development

When Reprocessing Devices, Human Factors Matters

You Need Human Factors Folks and Designers: Adventures in Medical Device Usability