Medical Device & Diagnostic Industry
Magazine [3] |
An MD&DI October 1996 Feature
When used appropriately, statistical tools can make a significant contribution to the improvement of quality and productivity in medical device manufacturing. In the design phases of a product’s life cycle, for instance, such tools as risk analysis can be used to evaluate potential problems with a particular design approach, saving vast amounts of time and effort by eliminating faulty approaches from further consideration. In the preproduction phase, design of experiments (DOE) can be used to refine processing specifications, thus speeding the transition to full-scale production. And in the postmarket phases of a product’s life, trend analysis can enable manufacturers to rapidly identify adverse events that must be reported to FDA and to determine appropriate corrective actions.
In recognition of this important role for statistical tools, their use is strongly recommended in both the ISO 9000 family of quality systems standards compiled by the International Organization for Standardization (ISO) and FDA’s proposed revision of its good manufacturing practices (GMP) regulation.1,2 Nevertheless, very few device companies presently operate quality systems programs into which statistical tools have been integrated to achieve the greatest possible efficiency and productivity. In our experience, when the performance of biomedical companies is evaluated against the 20 elements of ISO 9000, it is in the section on statistical techniques that they consistently score lowest.
Where device companies do make use of statistical tools, the function is commonly limited to basic training in statistical process control (SPC) for manufacturing employees and to the use of control charts and inspection sampling programs in key manufacturing processes. Such limited strategies rarely result in widespread or effective use of statistical tools. To make the best use of such tools, the developers of quality systems should give greater attention to integrating them into those systems.
Unfortunately for those quality systems developers, very little literature is available to assist them in creating an effective statistical program. Subpart O–statistical techniques in FDA’s revised GMP regulation provides an outline for integrating statistical tools into the quality improvement system, but few details. ISO 9000 section 4.20 offers nearly the same advice, in the same depth. And although ISO is currently working to identify statistical tools that are appropriate for quality improvement, this effort has not yet resulted in a document that could be adopted as an ISO standard.3
The task for device manufacturers is thus a daunting one. And even after an ISO standard on the selection of appropriate statistical tools is issued, device manufacturers will still be faced with the work of integrating them into their quality systems, so that they can become widely and effectively used throughout the organization. This article describes the strategy and results of just such an effort conducted at nine facilities belonging to Medtronic, Inc. (Minneapolis), [4] a manufacturer of medical devices worldwide. The effort to create a successful statistical program comprised three distinct elements:
- Identification of appropriate statistical tools.
- Development of procedures to assign responsibility for the implementation, control, and review of statistical tools.
- Assessment of the quality system for use of good statistical practices.
IDENTIFYING STATISTICAL TOOLS
Both the revised GMP regulation and ISO 9000 require that each manufacturer identify valid statistical tools for establishing, controlling, and verifying the characteristics of their products and the capabilities of their manufacturing processes. Although the process of identifying such tools can be helpful, the act of documenting the results is often more helpful.
The first step in identifying the tools that are to become part of the quality system is to compile a list of all candidate techniques and divide them according to the phases of the product life cycle (see Figure 1). Accomplishing this step will provide the manufacturer with a ready-made list of alternative techniques that can be applied to those phases whenever appropriate to answer specific questions.
Once the list has been compiled in the first step, the second step is to refine it, stage by stage, and determine how the company’s quality systems manual will reflect the use of each statistical tool (see Table I). This step takes the manufacturer from the many possible techniques to the few that it intends to consider for application. Depending upon the type of data generated by a particular statistical tool, the rationale for requiring its use may include regulatory compliance, improvement of product design, refinement of manufacturing processes, or other reasons.
As part of this step, the manufacturer should also determine when it plans to apply a particular tool. In general, companies will gain the greatest benefits from powerful tools such as DOE if they are used as early in the product life cycle as possible. For such tools, then, pushing the use “upstream” should be a guiding principle. This principle is illustrated in Table I, where the design control stage includes the use of DOE, while corrective action consists mostly of the seven basic tools.
Figure 1. Statistical tools appropriate to the phases of a product life cycle.
One way to identify and assign applications to statistical tools is to survey the company’s managers, engineers, and technicians to discover which tools they are already using or believe they should be using. For instance, DOE can be used to improve quality during several phases of the product life cycle, so company personnel may recommend that its use be required or considered at a number of points in the quality systems manual. Another useful way to identify the tools to be used is to review the results of previous internal and FDA audits. For example, FDA auditors often focus on a manufacturer’s use of statistical tools for process validation, so it would make sense to include qualification and validation techniques as part of the design control and process control activities.
The process of considering which statistical tools to use can be reinforced by establishing procedures that remind company personnel of what tools to use and when to use them. As part of the design review process, for instance, the company’s standard operating procedures could include a requirement that the review team consider the use of DOE or design failure mode and effects analysis (FMEA). Even if use of a particular tool is not a firm requirement, such procedures should ensure that employees will consider using it at appropriate times.
DEVELOPING PROCEDURES FOR STATISTICAL TOOLS
FDA’s proposed revision of the GMP regulation and ISO 9000 both require that manufacturers develop and maintain procedures to establish, control, and verify the acceptability of product characteristics and process capability. A procedure for a statistical tool can accomplish any or all of the following:
- Identification: ensuring that appropriate tools are considered for the application.
- Implementation: ensuring that the tools are implemented at the right time and place.
- Control: verifying that the tools are being applied appropriately.
- Review: evaluating the use of statistical tools on a regular basis to be certain they are appropriate and are being used optimally.
Whenever statistical tools are being used to satisfy one or more of these purposes, quality systems developers should write procedures that define such use. Writing effective procedures is a key step toward integrating statistical tools into a quality improvement system. A procedure for use of a statistical tool should work to achieve the highest possible level of quality and productivity, encourage creative use of the tool, and add value to the process. A procedure that employees view as a burden that does not add value to their work will inevitably be disregarded.
The length and complexity of procedures can vary greatly, depending upon what is needed to describe the proper use of the tool in question. A procedure can consist of a short and simple statement, such as a single line in a process development checklist requiring that the process engineering team consider the use of DOE. A procedure may also be a lengthy and complex document, such as an acceptance sampling procedure that defines responsibilities for implementation, control, and review of sampling plans; includes a guideline for the rationale to be used in selecting the appropriate sampling plan; and provides work instructions for the operators who will use the procedures to perform the acceptance sampling.
Procedures may be written either as separate, stand-alone documents or as subsections of a larger protocol or process. In either case, it is important that the manufacturer keep track of the location of the procedure and be certain to place it under change control. This ensures that the proper version of the document is always available, and that earlier versions are discarded.
Stand-alone procedures are most useful when the statistical tool in question is an essential part of several processes and is used repetitively throughout the product life cycle. Such tools might include acceptance sampling, process capability studies, process qualification and validation studies, repeatability and reproducibility studies, and SPC. Including procedures for the use of a statistical tool within a larger process is a good choice when the tool is used only in conjunction with that process, or when the procedures may change from use to use. For example, it would be difficult to write a stand-alone procedure for DOE, because its implementation varies considerably each time it is used. Thus, the best way to integrate DOE into the process development phase might be to write a protocol specifying that part of the process development review will include consideration of DOE.
To write a good procedure for the use of statistical tools, quality systems developers should ensure that each of the following elements is addressed.
- What tool is to be applied.
- Who will apply the tool.
- Where the tool is to be applied.
- When (or how often) the tool is to be applied.
- How the tool is to be applied.
The second element-who will apply the tool-is arguably the most important aspect of any procedure relating to statistical tools. The manufacturer should ensure that every procedure clearly assigns responsibility for every step that must be taken with regard to the use of the tool. For example, a procedure might need to specify who will be responsible for reviewing inspection stations used for inspection sampling, who is responsible for taking action when an out-of-control condition is indicated by a control chart (and what action they should take), or who is responsible for reviewing existing sampling plans and control charts to ensure that they are appropriate and are being used optimally.
In general, it is most effective to assign responsibility for statistical tools to the level of production that is closest to their actual use, and to train personnel accordingly. Assigning responsibility for all such uses to the quality assurance department-a practice that is all too often the first instinct of quality systems developers-usually proves unproductive.
ASSESSING STATISTICAL PRACTICES
A manufacturer’s assessment of its use of statistical tools should consist of a formal, documented examination of current statistical practices and procedures, and an evaluation of future plans for improvement of the company’s quality system. To be useful, this assessment should go beyond the compliance-oriented approach that is commonly seen in quality audits. Following are some key objectives that manufacturers should consider for their assessment of good statistical practices:
- Determine the company’s current state of compliance with the GMP regulation and ISO 9000.
- Determine impediments to compliance with the GMP regulation and ISO 9000.
- Raise awareness of the GMP regulation and ISO 9000.
- Measure improvement over time.
- Discover the best statistical practices in use throughout the company and share them with the rest of the company.
- Provide advice on incorporating statistical tools into the quality improvement system.
To accomplish these objectives for their company, the authors developed a systematic approach to conducting a statistical practices assessment. The following sections describe the basic steps that were included in that approach, with some suggestions that others may use to implement such a program in their own companies.
Participate in Formal Company Audits. Formal audits of the company’s quality system are required by both the GMP regulation and ISO 9000. By participating in these audits, and making the statistical assessment a part of them, assessors can help to alleviate undue stress that can result when a department has to undergo a separate evaluation. In addition, since audits and assessments consume valuable resources, combining the two enables the company to conserve resources and improve its efficiency.
Focus on Key Areas. For most medical device companies, the key production areas in which statistical tools are used are design control, process control, incoming inspection, testing and measurement, and corrective action. Sometimes assessment of practices cannot be accomplished for all of these areas in the same tour. For example, assessment of process control and incoming inspection can take as much as three days all by themselves. Thus, assessors should begin with the most important areas and let the others follow as time and resources permit.
Follow a Procedure. Adopting and following clearly defined procedures can help assessors to ensure that their assessments are consistent, even when they are carried out at a variety of sites and perhaps over a long time. Consistency is important if the company is attempting to compare the practices of several divisions or production areas, in part because it ensures that the company will have a clear understanding of the best practices available within it. Being consistent can also help to allay the natural anxiety that can arise when the practices of one department are being compared to those of another; an inconsistent assessor will soon find his or her results called into question from many sides. A written assessment procedure that spells out what areas are to be inspected and what questions will be asked can help to resolve these problems. Figure 2 shows samples of the authors’ assessment forms related to process control.
Apply a Metric. Nothing gets management’s attention like a meaningful metric, and nothing is quite so good at conveying the differences among various practices. By using a metric to indicate the sophistication or quality level of a department’s statistical practices, assessors can make the job of comparing sites and communicating results much easier. Here again, consistency is a key element, so assessors should make sure to follow a procedure that will ensure regularity in their scores. Table II shows the authors’ scorecard for assessing the use of statistical practices for process control.
Provide Advice and Assistance. Auditors almost never give advice. But during statistical assessment interviews, assessors should feel free to go beyond the normal limits of an audit and offer whatever observations and advice they can. Company personnel often misunderstand the appropriate and optimal use and application of statistical tools. Most interviewees thirst for this information, and assessors should give it to them whenever possible.
Document Results. Assessors should prepare a formal report of their work and give copies to the assessed departments. This will aid company personnel in understanding the important and often difficult points involved in applying statistical tools. The report can also be used to establish benchmarks against which future performance can be compared.
Identify Best Statistical Practices. As assessors gather information from a number of divisions or functional areas, they should take the opportunity to compile a record of the best practices they discover and share them throughout the company. Circulation of information about a company’s best practices should not be limited by departmental boundaries; sometimes practices used in one department can be usefully adopted by others. A procedure for acceptance sampling, for instance, may work as well for in-process and final inspection as it does for incoming inspection. Our assessments have revealed some best practices that have been well worth sharing, including an excellent procedure for gage repeatability and reproducibility studies, a quality manual with a very good section on the identification of statistical tools, and the report of a project to identify appropriate inspection stations for acceptance sampling.
CONCLUSION
In most medical device companies, there is a need for greater integration of statistical tools into the quality system. Although some departments and facilities may argue that their existing procedures minimally comply with the GMP regulation and ISO 9000, and are therefore adequate, there is no doubt that a well-structured statistical program can improve quality and productivity and reduce costs beyond anything that might be obtained by a minimally compliant procedure.
Such improvements often come as a surprise to personnel in assessed departments. At one facility visited by the authors, staff were amazed to learn that use of a variables sampling plan could reduce sample sizes by an order of magnitude from previous levels. With such improvements in sight, the authors intend to continue conducting such assessments in conjunction with the company’s series of ongoing internal audits.
The role of the statistician in developing the use of statistical tools includes determining which ones are applicable and which ones are best, developing easy instructions for use of the tool, and determining whether it is necessary and is being applied optimally. These are challenging tasks for statisticians, and they are even more difficult for nonstatisticians. In fact, the difficulty involved in writing procedures for the use of statistical tools is one of the main reasons they often don’t exist where they are most needed. To help personnel cope with these challenges, assessors should consider providing templates of procedures that can be modified for use by all of a company’s facilities.
In general, the inclusion of statisticians as members of an internal audit team is likely to be well received by the departments being assessed. With their training in GMPs, ISO standards, and quality auditing, these statistician-assessors can offer valuable advice of a sort that company auditors are usually unable to provide. On the other hand, statistician-assessors should not assume that they always know best. Each assessment tour should provide new examples of a company’s best statistical practices, and assessors should learn as much as possible with every inspection. The assessment process is dynamic, requiring flexibility and an ability to adapt as well as consistency. Assessors should remember that their ultimate objective is continuous quality and productivity improvement, and that the application of statistical tools is but one means to that end.
REFERENCES
1. “Working Draft of the Current Good Manufacturing Practice (CGMP) Final Rule,” Rockville, MD, FDA, Center for Devices and Radiological Health, Office of Compliance, July 1995.
2. “Quality Systems—Model for Quality Assurance in Design, Development, Production, Installation and Servicing,” ISO 9001-1994, Geneva, International Organization for Standardization, 1994.
3. Wadsworth HM, “Standards for Tools and Techniques,” in Proceedings of the 48th Annual Quality Congress, Milwaukee, American Society for Quality Control, 1994.