Integrating Risk Management into the Design and Development Process

Originally Published March 2001

Andrew Snow

Successful risk management is essential to the design and development of safe and effective medical devices. Unfortunately, it is too often viewed by manufacturers as an isolated activity that must be performed merely to fulfill a regulatory requirement. This is the result of three common mistakes: a misunderstanding of the regulatory requirement, confusion about what risk management really is, and a failure to recognize the benefits of effective risk management.

This article clarifies the regulatory expectations and explains the fundamental concepts of risk management. Reviewing the methods of risk management provides manufacturers with tools that will improve their design and development efforts. The end results include device designs that are safe and effective, a shorter and more-efficient design and development timeline, and fewer postlaunch problems.

When it is performed correctly, risk management involves the development and transfer of safe, reliable, and effective devices to manufacturing, while at the same time reducing, controlling, and monitoring risk throughout a device's life cycle. Fundamental to achieving this success is integrating the methods of risk management into the design and development effort so that the following occur:

REGULATORY EXPECTATIONS

To understand how to achieve these results, it is necessary to understand the regulatory expectations for risk management—what it really means and how to do it. This is best achieved by first defining the regulatory requirements for risk management, and then exploring methods for establishing a successful risk management process.

Risk has always been an important part of regulatory requirements for devices, and manufacturers must realize that risk underlies the basic flexibility of the quality system regulation (QSR). Manufacturers have to decide how to implement QSR within their organizations. The preamble provides the guiding principle that the type and extent of controls implemented must be commensurate with the risk associated with the product produced.¹ Nowhere is this principle truer than with design controls.

Risk analysis is specifically mentioned in QSR part 820.30(g), Design Validation.² This part of the regulation states that design validation "shall include software validation and risk analysis, where appropriate." Requirements qualified with "where appropriate" are deemed necessary and appropriate unless the manufacturer can justify otherwise. That is easy enough for software—if there is no software, software validation is not appropriate. In order to justify not doing risk analysis in other cases, however, the manufacturer needs to establish that there are no risks; to do that, the manufacturer needs to perform a risk analysis. Clearly, it is always appropriate to do some form of risk analysis.

This is the area where a narrow interpretation of the regulation can cause confusion. To that end, it is unfortunate that the term risk analysis is buried way down in part 820.30(g) of the regulation, which further contributes to a common misunderstanding of the term. It is tempting for many manufacturers to think that as long as a risk analysis is in the file, all expectations have been met. This is incorrect. To understand why, one must take a closer look at what is meant by risk analysis.

First, the term risk analysis is confusing when placed in the context of current global risk management standards. However, at the time the regulation was developed, it was the term of choice and encompassed all of the activities now understood as risk management; that is, that companies should identify hazards, estimate risks, evaluate the acceptability of risks, and, where unacceptable, implement measures to control those risks and verify their effectiveness.³ In addition, when design changes are made, manufacturers need to evaluate their effects on any existing risk, and then determine if new hazards have been introduced as a result of those changes.

Second, risk management is defined in the standards as a lifecycle activity that starts—at the latest—when design control begins. FDA agrees: in the design control section of QSIT, the agency asks for clarification on how a company managed its risks during the design and development phase of the project.⁴ FDA wants to see how risk management activities were addressed in design plans and how risk was considered throughout the design process.

This approach is consistent with ISO 13485, the application of ISO 9001 to medical devices, where risk analysis is mentioned in the general requirements for design controls (section 4.4.1).⁵ "Throughout the design process," the requirement states, "the supplier shall evaluate the need for risk analysis." That is, some form of risk management is always required, it should be addressed as part of the design plans, and it must be considered throughout each stage of the product development process. This means that risk management outputs help define safety requirements as part of design inputs. The risk outputs also determine failure modes to be considered during design validation and identify potential new risks that might result from design changes.

Underlying these requirements is the notion that the types and extent of design controls used for any one product should be commensurate with the risk and complexity of the design. This is an important concept, because the largest potential source of failures associated with design are systematic errors in the design process. Examples of systematic errors would include a failure by the manufacturer to consider or properly identify requirements, the selection of inadequate designs, or a failure to adequately verify or validate. One of the primary benefits of implementing design controls is preventing these types of errors.

Manufacturers often protest that they don't need specific risk analysis techniques, using the all-too-common thought process that because they always take risk into consideration during product design, risk management simply becomes a natural part of the process. But while manufacturers may claim that it is their intention to always consider risk, they do not always follow through on that intent.

Ensuring that risk is considered is one of the major benefits of risk management. It provides a framework within the design control process for reducing systematic error and creates a decision-making process for assessing the adequacy of design safety; when taken together these factors lead to design improvements. In order to see how to integrate this concept into the design and development process, however, a better understanding of the risk management process is required.

RISK MANAGEMENT PROCESS

Risk management is more about the process than it is about the outputs. The documented results must be accurate, complete, and conclusive; more importantly, however, the manufacturer must be able to show how those outputs were used to drive the design control process and create a safe design. Too often, risk management outputs are placed in a design history file and forgotten. Instead, the risk management process should reflect the evolution of a safer device as the design progresses from concept to production.

The fundamental concepts of risk require the recognition that there is a causal relationship among the harm, the hazard, and the cause of the hazard. The cause may occur in the absence of failure or as a result of one or more failure conditions. Often, the hazard is inherent in the nature of the product. Attempting to overcorrect a hazardous event, however, may create further adverse consequences. Therefore, it is the source of the harm and not the actual harm itself that must be dealt with. During risk analysis, both the severity of the consequences of the hazard and the probability of the hazard occuring are evaluated. These two components— severity and probability of occurrence—make up risk.

Often, many of the risks associated with a product are related to the requirements and benefits derived from the product itself; the hazards and risks must then be weighed against the benefits derived. This evaluation determines the acceptability of the risks: if the risk is unacceptable, mitigations or risk reduction measures can be implemented. Doing so, however, might alter the end product negatively, or worse, may create new potential hazards. This back and forth process—assessing risk and reducing it, then evaluating new risk against the benefits derived—is the essence of risk management.

The most widely recognized standard for risk management is ISO 14971.⁶ It makes a clear distinction between risk analysis and risk management. While this standard is not mandated by the quality system regulations, and there are others (e.g., IEC 60601-1-4, for programmable medical electrical equipment, and EN 1441), it applies to all types of devices and has the common elements required for effective risk management. For a thorough review of the risk management process one can rely primarily on the revised version of ISO 14971.⁷

Risk analysis is only one element of the risk management process. Figure 1 shows the overall process for risk management. It comprises five core activities: planning, risk analysis, risk evaluation, risk control, and postproduction control. The process begins with planning, and is followed immediately by risk analysis. When the risks are evaluated and decisions are made regarding their acceptability, risk assessment is complete. Risk management begins when risk controls are implemented, their effectiveness is verified, and the overall safety of the device design is deemed acceptable.

Figure 1. Overall risk management process. Adapted from ISO 14971.

The process of risk management is never really completed; manufacturers must continue to review risk management information as field experience is gained and postproduction design changes are made. This process continues for as long as the product is on the market. The process loops back to evaluate new hazards as design changes are implemented during the postproduction period.

Risk Management Planning. Clearly, the first step in the process is planning how to manage risk for a given project. A risk management plan should include the following:

The risk management plan should go hand in hand with the design and development plan. At the beginning of a project, the nature of hazards and their causes are often unknown, so the plan may change as more is learned about the device. The plan may be project specific, or it may be specified as part of operating procedures and policies. Risk management activities can be included as part of other design reviews or performed as independent reviews. Ultimately, hazards and their mitigations should link directly to verification and validation plans. It is important for management to determine responsibilities, establish adequate qualified resources, and review risk management activities and results to ensure that an effective management process is in place.

When setting policies for risk acceptability, several factors should be taken into consideration. While the overall guiding principle is that risks should be outweighed by benefits, decisions usually can be justified by doing three things: one, comparing the product to other similar devices on the market; two, following appropriate guidance (e.g., the single-fault philosophy); and three, using product-specific standards.

Comparisons with other products should take into account similarities and differences in intended use, hazards, risk, safety features, and historical data. The single-fault philosophy, detailed in IEC 60513, implies that medical electrical equipment have two means of defense against any one hazard, so that a single fault cannot result in the hazard.⁸ The underlying assumption is that the equipment is reasonably reliable, so that the probability of any one single failure is low. Product-specific standards usually specify requirements, which, if implemented and tested, will result in an acceptable level of risk.

Risk Analysis. The risk management process continues with a definition of the intended use of the device and detailed description of the characteristics that affect device safety. The statement of intended use should also include foreseeable misuse. Annex A of ISO 14971 provides questions that guide the determination of intended use and characteristics of the device.⁹ Human factors issues and user interfaces should also be taken into consideration at this point.

Next, hazards associated with the device are identified. Brainstorming is a useful tool for identifying hazards and can be supplemented with Annex D of ISO 14971 and Annex B for in vitro diagnostic devices.¹⁰ Requirement documents are also good sources of hazards as there are many hazards associated with the nonfulfillment of a requirement. For example, in dialysis equipment there may be requirements for fluid removal and hazards associated with inadequate or excessive fluid removal.

Estimating the risks associated with each hazard completes the risk analysis part of the process. As previously defined, risk is the probability of a hazard causing harm and the severity of the consequences. Risk estimation can be quantitative or qualitative. Some typical tools for risk analysis are summarized in Table I. Risk can also be defined using the chart shown in Figure 2.¹¹ This chart can be converted to a table for a more qualitative assignment of risk based on categories.

Table I. Survey of common risk analysis tools.

Figure 2. Risk acceptability chart.

Manufacturers should develop a qualitative categorization of severity based on an evaluation of both the long-term and short-term effects of harm. Typically, several discrete levels are developed (see Table II, which was adopted from IEC 60601-1-4).¹²

Table II. Qualitative severity categories.

If sufficient data are available, quantitative probabilities of hazards can be estimated. Frequently this is not the case, however, and qualitative descriptors (e.g., incredible, improbable, remote, occasional, frequent) must be used to define probability. In estimating probabilities, the manufacturer needs to consider the initiating causes of a hazard, and decide if the hazard occurs in the absence of a failure or from a single or multiple failure. To help estimate this, historical data, analytical or simulation techniques, and expert judgment can all be used. Historical data or simulation techniques are preferable and can act as independent checks of each other. In some cases, however, it may only be possible to use expert judgment.

Risk Evaluation. Once the components of risk have been determined, evaluating the risks associated with each hazard and determining their acceptability completes the risk assessment. Tables III and IV—which are adopted from Annex E in ISO 14971 and IEC 60513, Technical Report on the Fundamental Aspects of Safety Standards for Medical Electrical Equipment—illustrate the decision criteria.¹³

Table III. Qualitative risk categories.

Table IV. Risk acceptability.

Generally, risks that are as low as reasonably practicable (ALARP) are acceptable if the benefits justify any residual risks. At the same time, however, intolerable risks are not acceptable and must be reduced at least to the level of ALARP risks. If this is not possible, the project must be terminated.

The concept of practicability involves both technical and economic consideration. Technical in this case refers to the availability and feasibility of solutions that mitigate or reduce risk; economic refers to an ability to reduce risks without making the use and application of the device financially infeasible.

If the risk is not broadly acceptable, risk reduction must be considered. There are three possible scenarios in this case: one, risk is reduced to the broadly acceptable region and there is no need to consider it further; two, risk reduction is practicable, but it cannot be reduced into the broadly acceptable region; or three, risk reduction is not reasonably practicable.

In the latter two scenarios, residual unacceptable risk is evaluated against the device benefits to determine its acceptability. Benefits should include consideration of the patient's current state and prognosis, the likelihood of improvement or deterioration if alternative treatments are used, and the degree and likelihood of improvement with the proposed treatment. If the benefits do not outweigh the risks, the risk is unacceptable.

Risk Control. Once the decision is made to reduce risk, control activities begin. Risk reduction should focus on reducing the hazard severity, the probability of occurrence, or both. Manufacturers should use one or more of the following types of risk controls, in the order listed:

Product-specific standards address inherent safety, protective measures, and information for safety (e.g., labeling) for many types of devices. These standards should be used to define requirements for design and testing, where applicable. One way of evaluating the control options is to estimate their potential impact on the severity and probability of hazard occurrences. The technical and economic practicality of implementing the options should also be evaluated. If it is not practicable to reduce risk further, then the risk/benefit analysis must justify any residual risks.

An implementation of risk reduction measures consists of the following:

Once the risk reduction measures have been fully evaluated and implemented, the overall risk acceptability of the device should be determined. This is done using the criteria established in the risk management plan (which is based on individual hazards). The risk acceptability evaluation should be documented in the final risk management report. The risk management report contains or refers to the following: