Those of us who work with medical device software are well aware of the radiation therapy software disaster in the mid-1980s, in which patients received 100 times the intended dose of radiation due to a software defect. The incident prompted FDA-regulation of medical device software.¹ It is well established that even the simplest software can have complex and unpredictable behavior, even when it is developed in the structured, systematic manner prescribed by the principles and practices of modern software engineering. Risks creep into the best designs. Your best defense is a robust risk management process with valid decision making.^{2, 3}

Complete and correct risk analysis is the basis of risk management for medical device design. It always has been, and will continue to be, crucial to the safety and effectiveness of medical devices, as long as we continue to develop new or modified products

Medical Device Risk Management

During the late 1990s, there was a recognized need to codify engineering best practices for risk management, so they were available to the largest possible medical device engineering audience. The first comprehensive international consensus standard for risk management of medical devices (ISO 14971) was published in 2000 and amended in 2003. A second edition was published in 2007. ISO 14971 has been an FDA-recognized risk management consensus standard since 2001.⁴ In 2005, the Global Harmonization Task Force (GHTF), a voluntary international group of representatives from medical device regulatory authorities and trade associations from Europe, the United States, Canada, Japan, and Australia, issued its final guidance on implementation of risk management within a medical device quality management system.⁵ Both of these documents provide medical device manufacturers with a structured, systematic approach to risk management.

Medical device risk management (Figure 1) is a necessary, although not sufficient, component of medical device design control (Figure 2).^{6, 7} Envision risk management as consisting of iterative risk analyses, followed by evaluation of the overall residual risks and generation of a risk management report. Risk analysis proceeds iteratively (Figure 3) as the medical device design evolves. It consists of the following:

Once the design is frozen, the acceptability of the overall residual risks is determined to evaluate whether the product is viable.

A variety of tools can be used for risk analysis (Figure 4). The tool chosen depends on two critical factors. First, will the risk analysis be inductive (bottom-up) or deductive (top-down)? Second, what kind of data is available? Is the data objective (e.g., historical, quantitative, failure rate data from clinical or actuarial sources) or subjective (e.g., a best guess, using expert opinions). Correct risk management is essential for medical device innovation. For many medical device manufacturers commercializing new or improved devices, an economically reasonable inductive risk analysis can be only accomplished with mostly subjective data. The alternative is exhaustive clinical, field, and accelerated environmental testing.

Two important elements of both ISO 14971 and the GHTF guidance are as follows:

Top management, not the design engineer or engineering manager, is responsible for incorporating effective risk management into the quality management of medical devices.^{5, 8}

Nowhere in the ISO standard or GHTF guidance does it state that only a bottom-up risk analysis is necessary and sufficient; a combination of top-down and bottom-up analyses gives the greatest and most economical risk coverage.

The dFMEA

The design (or device) failure modes effects analysis (dFMEA) is an inductive risk analysis tool that addresses design-related risks to the end-user (e.g., the patient, the patient’s family, and the patient’s healthcare providers). Annex C and Table E1 of ISO 14971:2007 provide a set of critical questions and a convenient listing of hazards that need consideration in an FMEA. Annex A of the GHTF guidance provides a convenient tabulation of risk values for an FMEA, segregated by whether they are low, medium, or high. Given these, Figure 4 suggests why the FMEA is a popular tool.

There is an important caveat regarding these convenient questions, tables, and risk categories. They are generic, not comprehensive, and do not necessarily all apply to a particular medical device. The clear expectation is that the designer will think critically through what is and is not relevant to the safety and effectiveness of the medical device and its users.

Risk is defined in both ISO 14971:2007 section 2.16 and the GHTF guidance (section 2) as the “combination of the probability of occurrence of harm and the severity of that harm.” Severity of harm (S) is how critical the hazard is (i.e., will it kill you or will it be simply annoying?). The probability or frequency of occurrence (O) is an estimate of how often that hazard is expected to occur. The product of these (S × O) is the practical measure of risk, and the numerical value is a risk prioritization number (RPN = S × O) that can be used to decide the order in which to mitigate these risks (i.e., highest first, lowest last). The RPN may also be used as a mechanism to allocate the engineering resources required to deal with each risk. What we are trying to accomplish in the dFMEA is the management of design risk. Design risk originates from the specific design of a product, not from its manufacture, distribution, or maintenance.

Nowhere in the standard or guidance is detectability (D) included in the definition of risk because detection is not an element of risk but a risk mitigation or risk control measure. The use of detectability, which legitimately can be used in a process failure modes analysis (pFMEA), is not permissible in a dFMEA. The ingress of detectability in the dFMEA may have been the result of misunderstanding the difference between a pFMEA (development, manufacturing, distribution, or maintenance) and the dFMEA (final risk to the end-user). The misunderstanding has persisted in some quarters because it appears to offer a convenient means of showing acceptable design risk. However, it does not. The use of detectability in the dFMEA is not only the use of a faulty risk management process in a quality management system; it is also the use of a flawed corporate decision-making process. Detectability does not change the actual design risk; only redesign may accomplish that.

Managing risk multiplied by detectability is not managing design risk. The whole purpose of ISO 14971 is to provide a framework “to manage the risks associated with the USE of medical devices.”⁹ Unless the end-user can detect the specific risk in a sufficient amount of time to avoid the risk and recall the correct procedure for avoiding that specific risk, detectability is not even useful as a risk control measure.

Conclusion

Risk consists of severity of harm (S) and frequency of occurrence (O). Do not include detectability (D) in a design FMEA because it corrupts the risk analysis and causes you to underestimate the real risk to the users of your medical device. Reducing design risk requires changing the design, designing engineering controls, or, as a last resort, designing effective warnings. Yes, lowering the risk prioritization number (RPN = S × O) with detectability (RPN × D) certainly reduces your engineering workload and reduces your time to market, but it does nothing to reduce the risk to patients, their family members, and their healthcare providers.

References

1. S Casey, Set Phasers On Stun And Other True Tales Of Design, Technology, And Human Error, 2nd ed., (Santa Barbara, CA: Aegean, 1998).

2. Mike W Schmidt, “The Use and Misuse of FMEA in Risk Analysis,” Medical Device and Diagnostic Industry [online] March
2004 [cited 24 February 2013]; available from Internet: www.mddionline.com/article/use-and-misuse-fmea-risk-analysis.

3. Stan Telson, “Develop Defensively: Control Risk and Predict Results,” Medical Device and Diagnostic Industry [online] May 2005 [cited 24 February 2013]; available from Internet: www.mddionline.com/node/1226.

4. Federal Register, 66 FR:23032, May 7, 2001.

5. GHTF/SG3/N15R8, “Implementation of Risk Management Principles and Activities Within a Quality Management System” (Global Harmonization Task Froce, 2005).

6. Code of Federal Regulations 21 CFR 820.30

7. ISO 13485:2003, “Medical Devices—Quality Management Systems—Requirements for Regulatory Purposes—Part 7: Realization Requirements” (Geneva: International Organization for Standardization, 2003).

8. ISO 14971:2007, “Medical Devices—Application of Risk Management to Medical Devices—Part 3.2: Management Responsibilities” (Geneva: International Organization for Standardization, 2007).

9. ISO 14971:2007, “Medical Devices—Application of Risk Management to Medical Devices—Introduction” (Geneva: International Organization for Standardization, 2007).

G.M. Samaras is a biomedical scientist and engineer currently in private practice (Pueblo, CO). Trained as an electrical engineer, he has doctorates in physiology and industrial engineering. He is a licensed professional engineer, board-certified human factors engineer, and an ASQ-certified quality engineer. Samaras has a number of biomedical patents and publications in physiology and engineering. He has worked at CDRH as a reviewer and manager, was a medical school and engineering graduate school professor, and founded an biomedical engineering firm that he ran for a decade. Reach him at [email protected].