Medical Device & Diagnostic Industry Magazine
MDDI Article Index

An MD&DI October 1999 Column

Developed in concert with industry groups, FDA's new inspection process emphasizes management responsibility for establishing effective quality systems.

Ever since the reengineering efforts at FDA's Center for Devices and Radiological Health (CDRH) began in 1997, at least one inspection-related team has been in place. The early reengineering activities, however, targeted only the issues of pre- and postinspection efficiencies. CDRH was, in fact, nibbling around the edges of the real issue: how to change the inspection process itself. Certainly, the thought of reengineering a major inspection program represented a daunting task.

The need for change was substantial. Numerous and significant issues needed to be addressed. The process for medical device inspections had not undergone a major change in years. One reason to look at reengineering the inspection process was the worsening of FDA's coverage of the device industry. Instead of inspecting device firms every two years, the agency found itself approaching the possibility of inspections every seven years. Indeed, whereas the "inventory" of registered device firms was increasing, the available number of investigator hours to accomplish the inspections was decreasing. In order to inspect the larger inventory with fewer investigators, the agency needed a shorter inspection. At the same time, the process called for quality inspections, with the agency focusing on the key areas of the industry's quality systems.

Another significant stimulus for reengineering was the need for FDA to consider the harmonization of inspections with other government entities. The mutual recognition agreement (MRA) signed by President Clinton in December 1998 stipulates that the U.S. government will normally endorse audit reports from conformity-assessment bodies in the European Union by December 2001.

Finally, the agency realized that the existing process for inspecting firms was not in alignment with the quality system (QS) regulation. FDA was using an inspection technique built around the premise that firms probably had device problems and that it was the job of FDA to find those problems. Indeed, over time there have been problems involving good manufacturing practices (GMPs). For several years, quality system inspection conclusions have shown an increase in "official action indicated" notations. Such findings usually result in warning letters or other actions taken by the agency.

While the responsibility for demonstrating compliance has always been on the manufacturers, things changed with the advent of the QS regulation. The regulation, which becamse effective in June 1997, is very specfic in holding management responsible for quality system performance.The imposition of ISO-type requirements that came into place with the QSR meant that management of device firms now have to ensure that adequate and effective quality systems are established and maintained. As firms gear up to meet the regulation, FDA has concluded that it needs an inspection process that mirrors this new reality.

ACCOMPLISHING CHANGE

The reengineering process involved a number of key stages including the formation of teams, the solicitation and analysis of design inputs from diverse sources, the preparation of a working model, and the final elaboration of a new approach to quality system inspections.

Team Formation. FDA's reengineering team was formed toward the end of 1997. Using national, district, and CDRH device experts, the team brought significant expertise to the table. Adopting the reengineering procedures that had been tested and proven in Michael Hammer's book Reengineering the Corporation enabled FDA to institute a flexible and open process.

Whereas traditionally the agency would sit down and rewrite its own inspection process, with reengineering, extensive stakeholder input, benchmarking, and design controls were employed in developing a new process. Proposals were put before the public and the industry early and throughout the project. An ad hoc group was formed comprising device manufacturers, FDA, and other industry experts. The group met several times at informal meetings hosted by FDLI in Washington, DC, to discuss how a quality system inspection could and should be done.

Design Inputs. The first input into the process was the suggestion by industry that, during quality system inspections, FDA needed to emphasize the importance of management in establishing an effective quality system. Another input came from the experiences of firms that are audited by notified bodies and wanted FDA to use some of the same procedures. Finally, FDA and industry realized that, under the existing inspection process, it was not uncommon for firms to address quality issues brought to their attention through FDA inspections. However, the companies would not necessarily take responsibility for those problems, or look at them from a systems viewpoint.

The Model. Working with the ad hoc group, FDA developed a new inspection process that appeared to meet all user needs. After several meetings with the group, FDA unveiled the process at an open public meeting in Rockville, MD, in May 1998. The new inspection process—called the quality system inspection technique, or QSIT—was released to the public via the reengineering Web site in September 1998 in a document called the QSIT Handbook, the title of which was later changed to Guide to Inspections of Quality Systems.

A New Inspection Approach. The QSIT process encompasses three new approaches that are different from the traditional quality system inspection. The first is the concept of a "top-down" process. With traditional inspections—and for those that will continue in the future under the "for-cause" approach—the investigator begins the process by looking at raw data or records that relate to device problems and production issues. These "bottom-up" inspections work their way up the system, with the goal of reaching the root cause of the problem(s) and working out a solution with the firm. With QSIT, the inspection begins with review of top systemwide procedures and policies. As more information is needed, the investigator bores down farther into the records, taking a top-down approach.

The second new approach is the concept of record sampling. Sampling allows the investigator to look at a selection of records by using statistical tables as a guide for determining how many documents to examine. This helps to keep the process moving, and to cover more ground during the inspection.

The final new approach is the concept of preinspection record review. With QSIT, investigators are encouraged to ask for records before the inspection actually begins. While firms are not required to send any records to FDA prior to the actual start of the inspection, data from the QSIT study indicate that this practice saved time and enhanced the efficiency of the inspection.

QSIT PRINCIPLES

Two aspects of QSIT distinguish it as a unique inspection process: the principles of management responsibility and of corrective and preventive actions (CAPA). The principle of management responsibility means that QSIT places a major emphasis on evaluating a firm's management and how well it is doing in establishing an effective quality system. QSIT inspections both start and finish with a review of the firm's management controls. The QSIT approach actually advises the investigator to consider the fact that product, design, and CAPA problems found during the inspection can be used to tie back to the inability of the firm's management to establish an effective quality system.

The principle of CAPA involves a new way for the investigator to look at the firm's systems for handling CAPA information. The investigator is aware that, under QSIT, it is possible for product and process nonconformances to occur, just as they would under any system. But what is more important is the firm's process for handling nonconformance data: the investigators are told that the system for handling quality data is more important than the actual events that reside in the data. Because the investigators are checking the system, QSIT will result in more conclusions being made about the system that will heavily influence FDA inspection decisions. It thus becomes especially beneficial for companies to establish and maintain good systems prior to inspection. A benefit of the QSIT approach is that it should result in the dialogues between the agency and the firms being on systems issues, rather than on issues of product and process problems.

Quality Subsystems. QSIT is an inspection process based on the subsystems of the quality system. It has been suggested that a quality system can be broken down into seven subsystems (Figure 1). To improve the efficiency of the inspection process, the QSIT approach focuses on the four primary subsystems: management controls, design controls, corrective and preventive actions, and production and process controls. The other three subsystems—material controls; records, documents, and change controls; and equipment and facilities controls—are addressed by QSIT through linkages rather than through specific coverage. As indicated in the guide, linkages allow the investigator to leave a subsystem in order to cover another aspect of the quality system. They are used because subsystem probes can not be performed exclusive of other aspects of the system.

Figure 1. This schematic of quality system subsystems emphasizes the central importance placed by QSIT on management controls.

As mentioned above, QSIT places management controls at the heart of the quality system. However, the concept of CAPA—as the location for all quality data—is also very important.

Subsystem Inspection. The Guide to Inspections of Quality Systems directs the QSIT inspection process and sets forth a uniform methodology for inspecting each of the subsystems. For each subsystem, the guidebook contains four common sections: purpose and importance, inspection objectives, flowcharts for guiding the inspection process, and a narrative, providing how-to advice on inspecting each subsystem. Sampling plans and instructions are also provided in the guide, which can be found in its entirety (in PDF) on the QSIT Web site.

The QSIT "Establish" Test. The QSIT approach uses what has been called the "establish" test in each of the subsystem probes. What this means is that each of the four subsystem inspections proceeds according to the three compenents of the regulation's definition of establish: define, document, and implement (21 CFR 820.3(k)).

The inspections start with an analysis of the documents that the firm uses to define and document the subsystem. These documents are the firm's standard operating procedures (SOPs) and policies that relate to the subsystem. For example, during the inspection of the design control subsystem, the very first QSIT objective is to look at the company's procedures for design controls.

The second part of the QSIT establish test is the evaluation of what the firm is doing to keep the subsystem in place. This evaluation is actually two separate reviews. The first determines whether the firm is doing what its procedures say it is supposed to be doing. The second evaluates whether the processes being performed comply with the regulation. Samples of records will be used to verify compliance in several aspects of the review.

The third part of the QSIT establish test is the evaluation of whether the firm is carrying out the process for that subsystem adequately. Various records will be reviewed to determine adequacy. For example, under CAPA, the investigator will determine if the CAP subsystem is capturing and acting on quality data. If it fails to do so, the subsystem could be deemed inadequate.

As in any inspection, QSIT challenges a system by probing records and comparing the findings against procedures and regulations. This aspect of the inspection is the area in which QSIT is closest to the old inspection procedures. In this regard, FDA has demonstrated considerable expertise in evaluating the processes of manufacturers. For example, during a bottom-up inspection, the investigator will check raw data (records) and compare the information in those records against the firm's procedures and the regulations. Although the agency has had great success in inspecting via this process, the details were extremely time-consuming and investigators had the difficult chore of deciding when they had enough data to indicate a firm was or was not in compliance. The use of sampling tables with QSIT makes this task much easier.

Sampling Tables. One of the strengths of QSIT is the sampling tables. These tables allow an investigator to cover more ground during a QSIT inspection than what might have been covered traditionally. The sampling tables allow the investigator to document the existence of problems (or their nonexistence) through statistically valid sampling.

In a traditional inspection, each investigator had to look at records and decide how many examples of compliance or noncompliance were needed to satisfy his or her supervisor and compliance officer. Was one record enough? Fifty? Five hundred? Generally, the investigator had to make sure that enough records had been examined, given the time and effort involved in returning to the firm for additional documents. Obtaining the right records was another problem. Did the investigator document the worst or best case? How and why did he or she choose those records? With the QSIT approach and the use of sampling tables, the investigator is led to the right records and is provided guidance on how many records to look at to determine and document a violation or nonviolation.

Table I. Example of sampling plans and upper confidence levels (95% confidence limit).

The tables are used to tell the agency the likelihood that a violation will occur. For example, Table I shows that if you look at 35 records and find no examples of the violation you are looking for, you can be 95% confident that the population of records from which you drew the sample has that violation in no greater than 10% of the records. An important point about the sampling statistics is that the tables can be used to indicate both the probability of the violation occurring in the population and the probability of the violation not occurring.

Order of Systems Inspections. QSIT guides the investigator through the four major subsystems in a prescribed manner. The inspection starts with management controls and proceeds to design controls, CAPA, and production and process controls, finishing with a return to management controls. While these probes into the quality system may appear to be discreet evaluations of specific areas, in reality it is neither practical nor likely that the inspections will follow the process so closely. In fact, the dynamic process of establishing quality systems also results in a dynamic process for inspections.

As the investigator goes down one of the four probes, linkages that tie back to the other subsystems will lead the inspection out of the designated subsystem and into other sub-systems as needed. For example, during the probe of production and process controls, the investigator is asked to sample training records of production employees, and training is technically covered under the management section of the regulation. It is thus not possible to conduct the four probes in total isolation from one another.

During the standard baseline QSIT inspection, the amount of time spent in each subsystem varies from a half day to nearly two days. There is no established time limit for each subsystem review; rather, the number of records reviewed will determine the length of time each requires. An average of one day per subsystem is suggested in the guide, but some (CAPA, for example) are known to take longer and others (such as management) shorter. The probes will take longer if the investigator selects larger sample sizes from the sampling tables. Since higher-risk devices will often result in larger sample sizes, firms with such devices can expect slightly longer QSIT inspections. While the extent of sampling will determine the time it takes to do the probes, sampling also ensures that the investigator keeps moving through the inspection process. Unless the inspection was initiated as a for-cause action (e.g., a follow-up to device-related death or injury reports), it is unlikely that the investigator will spend a large amount of time in any one area of record review.

It is important for those involved in a QSIT inspection to understand why the process follows a prescribed order. Industry was clear in telling FDA that management was the glue that holds the quality system together, and the agency responded by placing management controls at the beginning of the QSIT process. The examination of design controls is done next in order to get the necessary coverage of the regulation at the beginning of the inspection. However, CAPA is where the inspection takes on a more traditional appearance. In fact, the coverage of production and process controls that follows CAPA is a throwback to the traditional inspection process, since these two areas are often where device-related problems reside in the quality system and their scrutiny requires the investigator to be highly skilled in rooting out system weaknesses.

In spite of the systems review of CAPA and production and process controls, FDA will be using real examples of processes and actual problem data for evaluation during the probes. These data are considered by the investigator to be examples of all of the firm's other CAPA and production data. In other words, under the systems evaluation, FDA will not look at all problem data or production records. But the agency will use examples from those two areas to reach conclusions regarding the subsystem in question—and, by extension, the firm's entire quality system.

An Emphasis on Management. The key tenet of the four probes is found in item seven of management controls, which advises inspectors to "evaluate whether management with executive responsibility ensures that an adequate and effective quality system has been established and maintained." Perhaps the most important aspect of QSIT, this holds management accountable for the weaknesses found during the inspection.

Under QSIT, the investigator is advised to complete items one through six in management and then to move on to the other three subsystems. But he or she is told to return to management at the conclusion of the inspection. This is where item seven is covered; it instructs the investigator to think about the firm's quality system that has just been reviewed. Was there any weakness found that could be considered a major nonconformity to the QSR? If so, the investigator can write an FDA-483 observation stating that management did not ensure the adequacy and effectiveness of the quality system.

QSIT VALIDATION

At the end of 1998 and early 1999, FDA conducted a study of the QSIT approach involving three districts with four specially QSIT-trained investigators and a compliance officer. In all, 42 QSIT inspections were done under the study. A report (in PDF) on the findings was placed on the QSIT Web site. (A future MD&DI article will provide detailed information about the study and the validation FDA performed.)

Both industry and the agency were pleased with the results of the study. The QSIT approach reduced the time required for comprehensive inspections and appeared to focus the inspections on the key areas of company quality systems.

FUTURE DIRECTIONS

Will FDA's new quality system inspection technique bring great improvements to the medical device inspection program? Based on stakeholder efforts and findings from the QSIT study, it appears that the answer is yes.

As the agency takes comments on a compliance program related to QSIT(Federal Register announcement August 12, 1999), theimplementation moves forward. Training is being implemented and regionalindustry workshops on QSIT are being scheduled (dates, locations, andcontact information can be found on the CDRHWeb site). The agency hopes to widely adopt QSIT in the fall of 1999.

BIBLIOGRAPHY

Guide to Inspections of Quality Systems Washington, DC: Food and Drug Administration, August 1999.

Hammer, Michael, and Champy, James. Reengineering the Corporation. New York: Harper Collins, 1993.

Quality System Regulation, Code of Federal Regulations, 21 CFR 820, 1996.

Timothy R. Wells is presently team leader of FDA's (Medical Device) Quality System Inspections Reengineering team. During his 23-year career with the agency he has worked as a field investigator in the Chicago district and as a small business representative in the Atlanta regional office. Since 1989, he has been branch chief in the Office of Compliance at the Center for Devices and Radiological Health.