Validate Your Process Using Design of Experiments

Design of experiments enables engineers to demonstrate or understand a process while providing information required for achieving regulatory compliance.

A valuable method for predicting process variability, design of experiments (DOE) allows medical device engineers to validate their processes in order to improve product quality. On February 12 from 10:45 to 11:15 a.m., Robert Launsby, president of Launsby Consulting, will present a workshop at MD&M West exploring the advantages of the DOE approach. In the following guest blog, he explains the advantages of the DOE strategy over other experimental methods and highlights the ability of this method to predict whether a process is likely to meet engineering specifications.

*      *       *      *      *

Formally developed in the early 1900s, experimental design is now more than 100 years old. But only during the last 15 years has it been applied worldwide by masses of engineers, scientists, and technicians. The primary reasons for this shift is the availability of powerful computers, the prevalence of easy-to-use software, and the accessibility of teaching approaches that provide a practical and pragmatic understanding of experimental design.

In a graduate course on statistical experimental design many decades ago, my classmates and I learned to invert matrices, transform matrices, crunch lots of numbers, and generate ANOVA and regression tables using HP and TI calculators. We became skillful in crunching the numbers--I got A's in the class--but had no understanding of how to tackle a typical industrial problem using design of experiments tools. Fortunately, many of the classes taught today in industry are much more practical and applications oriented.

Experimental design involves systematic and controlled changes to the input parameters in a process in order to mathematically estimate the impact of the process's key output parameters. Each input parameter under consideration is varied at two or more setpoints (or levels). Based on the number of variables involved in the design, the experimenter uses software to exploit and evaluate a balanced family of trials. Relationships between input factors and output variables (responses) can be readily visualized using simple graphs. And by applying proper experimental design approaches, statistically and practically significant data can also be assessed.

The DOE approach is not the only strategy available for conducting design experiments. Another method, known as one-factor-at-a-time experimentation, relies on an easier to understand set of tests and offers "pick the winner" analysis. Especially before the advent of the computer/software revolution in the last few decades, one-factor-at-a-time experimentation was a popular technique for conducting experiments. Nevertheless, this technique has several weaknesses. It cannot detect interactions (synergism) between factors and responses; it becomes convoluted if two or more response outputs must be traded off; it cannot mathematically determine the individual contribution of each input factor to changes in the response; and it cannot perform simple graphical analysis using such aids as main effects plots, interaction plots, contour plots, and Pareto charts showing factor effects for response.

Yet another experimental method can be dubbed "run a bunch of tests based on what we think will provide useful results." However, this method tends to be much less efficient than DOE, does not lend itself to simple analysis approaches, and does not allow for precision in estimating input factor effects.

Superior to other experimental methods, design of experiments enables us to demonstrate or further a fundamental understanding of a process while providing key information required to achieve regulatory compliance. Focusing on drug and biological products, an FDA Guidance Document titled "Process Validation: General Principles and Practices" (January 2011) discusses design of experiments as a valuable tool for understanding and characterizing processes. Hopefully, the language in this document will eventually become part of the guidance for medical device process validation as well. Used during process characterization or as part of the operational qualification phase of process validation--in other words, before the formal process validation stage--design of experiments helps us to understand the key input variables and the optimal setpoint for each input variable.

Many DOE practitioners ask, how much input variation can we allow while ensuring that the process delivers response variation that meets engineering specifications? To ascertain the expected amount of variation in a response given known variation in key input variables, a properly designed experiment can generate a mathematical model that can then be used to perform sensitivity analysis using simulation tools such as Monte Carlo analysis. As part of characterization or the operational qualification phase of process validation, such an analysis allows us to predict whether the process has the potential to meet engineering specifications during the performance qualification stage and during ongoing manufacturing.