Developing a Comprehensive Quality System

Originally Published MDDI June 2004

Regulatory Outlook

An effective quality system must integrate risk management, customer satisfaction, and continuous improvement. 

Barbara-Helene Smith
Regulatory Affairs Consultant

After almost two decades of enforcing the good manufacturing practices regulation, FDA enacted the quality system regulation (QSR) in 1996 to harmonize with the ISO 9000:1994 quality system standard. Just as medical device companies began feeling comfortable with the QSR, the ISO standard was revised. 

ISO 9001:2000 modified the focus of the international standard from a procedural and documentation-based system to a process-oriented system, with increased emphasis on customer satisfaction and continuous improvement.¹ But FDA, and many in industry, do not consider this new focus to be required for regulatory compliance. Hence the development of ISO 13485:2003.² This new standard for the international medical device industry essentially adopts the requirements of ISO 9001:1994, but it will not fully satisfy compliance with the QSR.
But does this mean that customer satisfaction and continuous improvement are not essential? Not at all. Indeed, they are important considerations that should be included in any comprehensive quality management system. But since they are not regulatory requirements, many companies' quality systems may not put enough emphasis on these important considerations.

Similarly, risk management is a critical concept for an effective quality system. Risk management is not new to the medical device industry. It is covered by the ISO 14971:2000 standard.³  While FDA recognizes this standard as a guideline, it does not treat the standard as an explicit regulatory requirement. Consequently, companies may not pay risk management its due. To be effective, a quality management system must incorporate risk-management considerations. In addition, FDA will likely pay much greater attention to risk management in the future.

Regardless of regulatory requirements, an effective quality system should address risk management, customer satisfaction, and continuous improvement. This article outlines ways to implement this important effort. 

Risk Management

Risk management comprises three primary elements: risk assessment, which includes risk analysis and risk evaluation; risk control; and postproduction monitoring. In the medical device industry, risk assessment initially takes place during design control. The QSR requires design control procedures (21 CFR 820.30) to include provisions for the following elements:⁴

•Design and development plan.
•Design input.
•Design output.
•Design verification and validation.
•Design reviews.
•Design transfer.
•Design changes.
•Design history file (to incorporate the required documents).

Design input defines the device requirements based on user needs and the intended uses of the device. From these requirements, product specifications are established.

Risk analysis identifies potential foreseeable hazards derived from product testing, complaints, adverse events, recalls, audit observations, and other product or process deviations that are involved with the use, misuse, or abuse of the product. Risk analysis then estimates the risk of each. Fault-tree analysis (FTA), which is a top-down approach, identifies potential product hazards to include those associated with software and human factors.^5,6 FTA is commonly used in reliability engineering to determine specific causes of a system failure. 
Risk evaluation is a risk–benefit assessment that compares each hazard against acceptable defined risks.

Failure modes and effects analysis (FMEA) assists manufacturers in preventing defects, enhancing safety, and increasing customer satisfaction by identifying and evaluating the failure potential of a product or process. With this analysis, an action plan can be implemented to eliminate the failure or reduce its likelihood. FMEA, a bottom-up approach, examines components and processing steps individually and then interactively. Some medical device companies also use hazard analysis critical control points (HACCP) as a risk analysis and risk control tool.

Once device specifications are established, design verification confirms that the design output meets the design input requirements. Design validation is an important step in the design control process to ensure that the finished device meets customers' needs and intended uses.

If a risk cannot be entirely eliminated, it must be controlled. Risk control reduces a risk to an acceptable level either through product design and manufacturing or by implementing protective measures.

If it is not feasible to incorporate changes to the device or the manufacturing process, the company must inform the user of the residual risks. Information can be provided to users and patients as a warning or caution in the product labeling. In some cases, physician or user training can also suffice. After all risk control measures have been analyzed and verified, the company must decide whether the overall risk is acceptable.

Postproduction monitoring evaluates how well the company met its goal both in satisfying customer needs and expectations as well as in identifying previously unrecognized hazards. 

Negative feedback should prompt the company to reevaluate the product and initiate improvement to reduce or eliminate any identified risk. Manufacturers must not wait for a catastrophic failure to make a change. Both the product and the manufacturing process should be assessed continually. 

Customer Satisfaction

Once a device is on the market, customer satisfaction can be measured using surveys, field reports, complaints, and reorders. Most companies focus on customer complaints and returned products. However, contract negotiation, order handling, and shipping errors are other areas that can be measured.

Many companies use quality assurance personnel to oversee the product quality issues (e.g., product performance and defects), while customer service representatives document business-related issues (e.g., billing and shipping errors). Device companies rarely combine the two to measure total customer satisfaction. 

Product performance, accuracy, and value; knowledgeable and courteous personnel; and company responsiveness are factors that are important to customers.⁷Every customer complaint or returned product suggests customer dissatisfaction. The problem may be a business issue. Shipping the wrong device or the wrong quantity, late delivery, damaged product, or billing errors all lead to customer dissatisfaction. Except perhaps for damaged product, these complaints do not involve regulatory issues; however, they may be factors in whether customers will reorder a device.

It is also important to remember that not all dissatisfied customers complain. The complaints received may be just the tip of the iceberg, and active customer feedback should be encouraged. Timely responses are essential to indicate to customers that the company is committed to making improvements. A lack of communication can result in lost market share and revenue.

Continuous Improvement 

From a regulatory standpoint, product quality issues must be evaluated and, if necessary, investigated. Most device companies do a satisfactory job documenting problems, but they often lack a comprehensive procedure for investigating causes and implementing preventive steps. A complaint or returned product must be investigated if any of the following conditions apply: the incident is critical (e.g., death or serious injury or illness), the device fails to meet its specifications or labeling statements, or the incident represents a repeat occurrence. Priority should be based on a risk assessment of safety, performance, reliability, regulatory compliance, and cost issues.

Even minor problems arising from customer dissatisfaction, if repeated frequently, can adversely affect a company's bottom line. Such complaints may also indicate underlying regulatory issues. For example, when a returned product is found to meet product specifications or when the problem cannot be duplicated, the complaint may be classified as “user error.” It implies that the customer misused the product or did not follow the instructions, either intentionally or unintentionally.

Although a manufacturer does not usually have control over how a customer uses a device once it is distributed, redundant complaints deemed user error should trigger an investigation. To prevent similar complaints and problems from recurring, several steps can be taken:

•Add a warning statement to the label.
•Clarify the instructions for use.
•Repeat or improve in-service training.
•Modify the manufacturing or the inspection processes.
•Redesign the device. 

In addition to reviewing the specific complaint, other data should be analyzed. Complaint files should be reviewed for similar incidents, and service and repair records should be examined for nonroutine maintenance issues. It is also important to review device history records for test failures and rework. Other data to examine might include component and supplier history records, environmental monitoring records, and sterility reports, as well as nonconforming-material reports, discrepancy logs, and material review board records. 

If the investigation reveals that the device failed to meet its predetermined specifications, further investigation is needed. During this investigation, certain questions should be addressed, including:

•Why did the device pass the initial final inspection?
•Did the final inspection process analyze for the specific nonconformance of the device?
•Are the inspection criteria sufficiently robust?
•Did the inspector follow the procedure?
•If the nonconformance was not addressed in the final inspection, could it have been identified at an earlier in-process inspection step during manufacturing? 

It is essential to determine the root cause of the nonconformance. Although there may be other contributing factors, most production errors are due to deficiencies in one of the following areas: the process, the equipment, a component, or an operator. 

Processing errors may be traced to lack of clarity of the manufacturing or inspection procedures or inadequate validation. If the process cannot be inspected, it must be validated, and the validation must be documented. Equipment malfunctions may result from poor design, incomplete qualification, or calibration or maintenance issues.

Component problems may be due to inadequate specifications, unacceptable suppliers, or insufficient inspection of materials. If a nonconforming component initially passed incoming inspection, it is critical to evaluate acceptance criteria, technician training, and inspection or test equipment records. If the problem is traced back to the supplier, the company's purchasing controls and supplier quality program must be reviewed. Operator error usually results from insufficient supervision, ineffective training, poorly written procedures, or the inability of the person to perform the activity.

A thorough investigation keeps asking why, until the root cause is found. It must go beyond the obvious solution, which might remedy the immediate problem but fail to prevent it from happening again. Each answer should lead to a new question until there are no more questions to ask.

Once the root cause is determined, a corrective and preventive action (CAPA) plan must be established and implemented. A weak CAPA program results when plans are not submitted, implemented, or sufficiently evaluated to verify that the action will permanently resolve the issue. For example, a frequent corrective action for operator error is to retrain the operator. Assuming the operator was trained originally, it is critical to determine why that training was ineffective. The initial training may have been inadequate or difficult to understand. If this is the case, the next step it to establish how the retraining will be conducted differently to ensure that it is effective. 

Perhaps the training itself is not the issue. The instructions may be unclear, or the procedure may require a special ability, such as manual dexterity, that a particular operator did not have. In these cases, no amount of additional training would improve performance.

The corrective action may involve modifying or redesigning the device. The company must have a procedure to link device changes to the design control program. Major modifications must go back through the design process and include a new risk assessment. Postproduction risk management integrates both product (e.g., design control) and procedural (e.g., CAPA program) improvement. 

Submission of a corrective action plan should not constitute closure. A closed-loop CAPA program includes verifying that the corrective action was completed and effective. For example, if the proposed correction involves a change to a specification or procedure, the nonconformance report or complaint should not be closed until the document change order has been issued or released. Frequently, incidents are closed based on a plan submitted to change a document, but the change is never implemented.

Several methods can be used to evaluate the effectiveness of the corrective action. Trending for recurrence of the nonconformance or complaint issue is one way to determine effectiveness. Another method is to measure an increase in yield or product quality or a decrease in the number of rejects, rework, scrap, or errors. A manufacturer can also evaluate the corrective action through subsequent compliance inspections and audits. In addition, the elimination of product recalls can be a measure of the corrective action's effectiveness. Operator training can be assessed by skill demonstration, certification, on-the-job observation, and periodic personnel evaluations. An effective risk management program involves not only identifying and resolving problems, but also taking steps to prevent problems before they occur.

Conclusion

Along with design control and risk management, customer satisfaction and continuous improvement should be included in any comprehensive quality management system. 

Risk management enables the development of safe and effective devices—whether new or modified—that meet customer needs, requirements, and expectations by designing and manufacturing them in compliance with the QSR. Moreover, risk management includes procedures and processes for evaluating all aspects of existing and potential quality issues. Risk management should be designed so that it incorporates customer feedback.

An effective quality management program can combine both customer satisfaction and risk management into a single system. By thoroughly investigating and permanently correcting the reasons for customer discontent, manufacturers can improve product quality and customer satisfaction. Customer satisfaction should be the result of an effective quality management system, and satisfied customers are generally repeat customers. 

References

1.ISO 9001:2000, “Quality Management System—Requirements,” (Geneva: International Organization for Standardization, 2000). 
2.ISO 13485:2003, “Medical Devices—Quality Management System—Requirements for Regulatory Purposes,” (Geneva: International Organization for Standardization, 2003).
3.ISO 14971:2000, “Medical Devices: Application of Risk Management to Medical Devices,” (Geneva: International Organization for Standardization, 2000).
4.Code of Federal Regulations, 21 CFR 820, Quality System Regulation, October 1996.
5.Guidance for FDA Reviewers and Industry Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices (Rockville, MD: Food and Drug Administration, 1998).
6.Guidance for Industry and FDA Pre-
market and Design Control Reviewers—
Medical Device User Safety: Incorporating Human Factor Engineering into Risk Management Document Issues, (Rock-
ville, MD: Food and Drug Administration, 2000).
7.Craig Cochran, “Customer Satisfaction: The Elusive Quarry,” Quality Digest 21, no. 11 (2001): 45–50. n

Copyright ©2004 Medical Device & Diagnostic Industry