Medical Device & Diagnostic Industry Magazine
MDDI Article Index

Originally Published MDDI September 2005

Process Verification

Regulating manufacturing processes in the medical device industry is challenging. But process signature verification can help manufacturers get a handle on problems before they get out of control.


Laura Dierker and Nathan Sheaff

Most medical device manufacturers still produce and test in batches, counting on their process guidelines to ensure product quality. But unless all the variables can be controlled—from operator hand strength to ambient temperature to component alignment—it is impossible to guarantee that a process step is truly consistent.

It is possible to develop this consistency, but doing so hinges on understanding the physical effects of a manufacturing process. Characterize those well, and for most manufacturing processes, it will be clear whether a step has been successful or whether potential defects have been introduced.

For years, the automotive industry has used a technique of evaluating the physical signatures during an in-line manufacturing process to determine the success or failure of a particular process. This technique could also work well for medical device manufacturing.

The Link to Regulation

Before reviewing the process itself, it is worth drawing a link to the new regulatory environment that has come about with the acceptance of ISO 14971:2000. This standard extends the concept of risk management to “all stages in a medical device life cycle,” including manufacturing. Risk management techniques now include the concept of detectability (the ease with which a fault can be detected) in evaluating risk.

The ability to detect possible faults or problems is intrinsic to removing them from a product. Perhaps most important, manufacturing is the only safe place in which to monitor detectability. According to Mike W. Schmidt of Strategic Device Compliance Services (Cincinnati, OH):

Process FMEA [failure mode and effects analysis] introduces a third term into the calculation. During manufacture, when a defect that could result in harm is detected, action can be taken to either repair the defect immediately or impound the product until it is repaired. In these circumstances, the use of detectability to figure the RPN [risk priority number] is completely appropriate. The time lag between detection during manufacture and the actual use, where the harm typically occurs, is substantial. 

However, detection of a hazard during use of the device may not assure that the harm will be avoided. An example of how detection can be virtually irrelevant to preventing harm would be as follows: The pin is pulled from a hand grenade with a 10-second fuse. After waiting eight seconds, the grenade is tossed into the room. It is detected, and then everyone in the room is dead. Detection in fact was irrelevant to the prevention of harm.

While the example is extreme, it shows that considering detectability as equivalent to severity and probability in determining the base RPN value is inappropriate when use [of the device] is involved.¹

In other words, a problem must be detected while there is still time to do something about it, and the best place for that is during the manufacturing process.

Process Signature Verification

Process Signature Verification is an in-process monitoring method in which the signature for each part undergoing a manufacturing process is used to provide an objective evaluation of the success of that process. It can verify that the process adheres to specifications, and it can determine the presence or absence of identified failure modes or defects.
Process Signature Verification is based on the fact that physical variables change during a process, and that those variables can indicate the success or failure of that process.

Temperature, pressure in a cavity, physical dimension, angle of a part position, force used to couple parts, flow rates of liquids, change in electrical characteristics, and many other variables help determine whether a process has been completely and successfully reproduced (see Table I).

Process Signature Verification is itself a series of three steps:

• Collect a detailed data set while monitoring a process. This data set is the process signature.
• Apply a detailed mathematical algorithm (predeveloped during an experiment) to identify correct behavior versus out-of-spec behavior.
• Provide the results (pass/fail/
observed values as required) and store the information for records or future analysis.

What Is a Process Signature? Every person has a unique signature. This signature is made up of lines, curves, and squiggles and represents one person's name. No two people have the same signature, but one person's signature may vary over time. A process also has a signature. It is made up of the changes in physical characteristics that occur during that process. The data recorded as that process proceeds in time are known as a waveform. One process for a particular type of part will generate a unique waveform, which is the process signature.²

You can tell a lot from people's signatures. Are they happy? Are they sick? Have there been any major disruptions in their life? You can learn even more from a manufacturing process signature. Was the process successful? Did it vary? What went wrong? The signature tells a story. For instance, in Figure 1, the quick dip in the downward curve over the blue area shows that two pieces coming together were misaligned and had to slip back together. This may cause problems downstream.

How Does Signature Analysis Work? True signature analysis software systematically decomposes a signature curve into identifiable characteristics, typically through analysis of separate portions of the curve.

The characteristics analyzed are those deemed to be most appropriate by scientific and process engineering staff, but are limited only by the availability of appropriate sensing equipment.

For example, each colored section of the curve in Figure 2 identifies a separately analyzed portion of a standard leak test. Each portion of the curve gives different information. In addition, each triangle identifies specific points whose placement and dimensions add to the evaluation of the process success. For example, leak-test studies have revealed the presence of rotating and damaged O-ring seals, bent or folded tubing, and part defects through proper analysis of a leak-test signature.

With appropriate analysis, even a simple test can yield significant process and component information. In implementation, a detailed data set is collected, and specifically determined features of the waveform are used to characterize the process.
Figure 3 shows an example of a classic press-fitting curve in which more than 60 significant characteristics and behaviors can be used to aid in analysis.

Getting Value Out of Process Signatures. There are several ways to derive value from implementing Process Signature Verification in medical device manufacturing:

• It can provide objective evidence of process compliance to specifications. A well-behaved process will be repeatable and reproducible. The signature of this process will be identical within measurement accuracy. The signature can be used as the specification of expected process results or behavior.
• It can characterize and document the detectability of manufacturing-induced failures or defects. The best signature analysis packages currently available enable analysis of hundreds of curves as a single data set. This powerful analysis helps identify concise, repeatable indicators for even very subtle process effects. It frequently happens that with detailed review, there will be a potential for unknown defects within a process. Although this can be dis-concerting, it is better for you to find these failures than for your customers to find them in use.
• In manufacturing, the clear advantage is that failure modes can be caught during the manufacturing process rather than escaping outside. In one case, the move from end-of-line to in-process test resulted in doubling the possible known failure modes. While that sounds bad, it means that those potential problems (which always existed before) were removed from any downstream effect.
• Finally, process validation can be done quickly. Signature analysis delivers detailed data on the consistency of a process. When a process is conforming to specifications, multiple curves run on multiple parts will overlay themselves with a very high repeatability. Using a set of curves like those in Figure 4 to prove process compliance is a fast and scientifically supportable way to validate a manufacturing process.

Confidence in the Numbers. For a manufacturing operation, the goal is to define a monitoring system that provides both the highest confidence in detectability of possible failure modes and a near-zero rate of false positives. This requires adequate information. A simple diagram can make this clear.

Measuring a limited set of points does not provide the detailed behavior of the process. More scientifically, to attain the highest confidence requires a level of data that satisfies the Nyquist sampling theorem.³ Simply put, in this theorem, the sample rate must always be twice the rate of the maximum frequency (Fmax) found in the data. Fmax is related to the rise time of the data, so if the data show a very quick slope change, as in Figure 5, the sampling rate must be twice as fast as the change in slope to see the effect properly.

However, even with the best possible data, it may be difficult to isolate a unique feature to characterize a behavior. In this case, using mathematics to interpret the physics is vital; a transform of two related variables may give an answer where no visible information is apparent, as in Figure 6. In the engine crankshaft torque and turn curve in Figure 6, abnormal behavior is hidden within the first set of curves (i.e., the bright red curve is in the middle of the first set, but it is clearly visible in the second set after mathematical processing).

Analysis Development

Generating a process signature for a manufacturing process and creating the algorithms to provide process verification requires two steps, both involving experimentation.

In the first step, scientists familiar with the roster of possible failure modes must determine the variables and the features of those variables that will indicate with confidence the presence or absence of a particular failure mode. A signature feature is defined as any specific behavior, such as an abnormal rise in a slope, that indicates uniquely the presence of the effect that needs to be characterized. In the second step, the experiment performed in the first step is repeated, generally with more samples, to provide documentation on the level of detectability using the features and waveform specification found earlier.

The following example of these steps analyzes the press-fit of one component to another. The analysis examined the tightness of the fit needed between the two pieces so that they won't work free from each other once in use.

To conduct such a test, samples can be collected or created. Some samples of the product should have the failure mode identified; other samples should be known to be free of that failure.

The failure in the example was easy to determine by using four good samples and two known-faulty samples. To demonstrate the procedure, the example used a simple plate with a hole and small inserts (see Figure 7). The plate with the insert and the press used are shown in Figure 8. The load sensor on the press is shown in Figure 9.

One or several variables can be monitored independently of each other. For example, it is known that a loose fit will lead to the insert moving quickly through the plate. So the force and distance can be used to monitor for this fault. A small load cell was used to measure the applied force, and an LTDV linear sensor was used for the distance measurements (see Figure 10). The force and distance were measured against an internal time for consistency. It was combined into the force-distance curve shown in Figure 11.

The manufacturing process was monitored at set times. Both acceptable and defective parts were monitored. Monitoring frequency can vary depending on the severity of the defect. The pressing operation was monitored for six parts, four acceptable and two with the known defect. Figure 11 shows the screen after the data capture. The behavior is clearly different for acceptable processes compared with those that are too loose and therefore prone to failure. The well-aligned results for the acceptable parts can be seen within the green band. The two parts that were defective fall below this band.

Next, the data set (signature curve) for each variable was reviewed using signature analysis tools. Data sets may be reviewed alone or in combination to identify one or several features that indicate the presence or absence of the failure mode.

In this case, the feature or characteristic that indicated process success most accurately and easily was the maximum insertion force, the force required to make the two parts fit together. When this characteristic is isolated, looking at one good part and one faulty part provides a clear indication of success or failure. So the algorithm in this simple case is one of reviewing the curve to find the peak insertion force, and then comparing it with the predetermined peak force band for a good insertion process.

Once the signatures of conformant processes were known and the features that would identify failure modes or nonconformant processes had been identified, a second set of experiments was run to document the confidence levels for the verification.
Finally, a process signature analysis algorithm was brought to the monitoring equipment in the manufacturing plant. The graphs in Figure 12 show pass and fail results for two parts, one acceptable and one defective, as determined by measuring peak insertion force.

Implementation

The signature analysis algorithm code is transported to the manufacturing systems for implementation. In the best systems, the identical code is ported between the analysis system and the monitoring systems, so there is absolutely no break between what the science says and what the monitoring systems provide.

It is essential that the data sensitivity of laboratory measurements is carried over to the systems in the manufacturing plant. Detectability in the lab is meaningless if the process monitoring equipment cannot apply the same level of sensitivity, resolution, and analysis within a working manufacturing environment.

The following list sets out the basis for an effective in-process signature verification system for a manufacturing plant:

• A high cycle rate for capturing the maximum data points, at least 30 kHz per channel.
• A high level of sensitivity; look for 16-bit data.
• Sufficient processing capacity to use the full derived process signature analysis within a manufacturing environment.
• The ability to provide a simple operator environment customized to the needs of each individual manufacturing environment.
• Near-zero effort to move new and updated analysis to the manufacturing floor in the case of corrective action/preventive action alterations or change of parts under manufacture. This includes using the same code so that no additional typing or setup is necessary on the manufacturing line. This move allows all the trials and validation of the algorithms to happen off-line, rather than disrupting manufacturing.

Conclusion

FDA is starting to recognize the value of a detailed review of process parameters. This is most visible within the guidance documents of the Center for Drug Evaluation and Research (CDER), but discussions with the agency also indicate interest in the medical device arena.

In its description of process understanding, the CDER process analytical technology (PAT) guidance states, “A process is generally considered well understood when (1) all critical sources of variability are identified and explained; (2) variability is managed by the process; and (3) product quality attributes can be reliably predicted over the design space established for materials used, process parameters, manufacturing, environmental, and other conditions. The ability to predict reflects a high degree of process understanding. Although retrospective process capability data are indicative of a state of control, these alone may be insufficient to gauge or communicate process understanding.”⁴

Process signatures are about process understanding. Process Signature Verification gives manufacturers an ability to see inside their processes and to detect and identify the root cause of failures that occur during manufacturing operations. Even subtle variability can be identified. Process understanding allows for increased confidence in product verification, parametric release methods, and faster validation cycles.

Given the increasingly challenging environment for medical device manufacturing—faster product introduction and turnaround coupled with higher costs and tighter profits—it makes sense to investigate methods that can offer both improved quality and increased yield in the manufacturing plant.

References
1. Mike W Schmidt, “The Use and Misuse of FMEA in Risk Analysis,” MD&DI 26, no. 3 (2004): 56–61.
2. Steve McMahon, “Signature Analysis System,” White Paper [online] (Ottawa, ON, Canada: Sciemetric Instruments Inc., 1998); available from Internet: www.sciemetric.
com/uploads/signatureanalysissystem.pdf.
3. “Sampling Theorem” [online] (Sunnyvale, CA: eFunda Inc.); available from Internet: www.efunda.com/designstandards/
sensors/methods/dsp_nyquist.cfm.
4. “PAT—A Framework for Innovative Pharmaceutical Development, Manufacturing, and Quality Analysis” (Rockville, MD: FDA, Center for Drug Evaluation and Research, 2004).

Laura Dierker is market segment manager for life sciences for Sciemetric Instruments Inc. (Ottawa, ON, Canada). Nathan Sheaff is CEO and CTO for the company.

Copyright ©2005 Medical Device & Diagnostic Industry