Originally published November 1995Director, Office of Device Evaluation, CDRH, Rockville, MD
In the two years since her appointment as director of the Office of Device Evaluation (ODE) at FDA's Center for Devices and Radiological Health, Susan Alpert has become a well-known figure to the medical device industry. Ubiquitous at meetings and conferences and tireless in meeting with industry representatives, Alpert has displayed an openness that has met with widespread industry approval.
The primary challenges of her office in the past few years have been to bring down review times for product submissions and to eliminate the massive backlog of 510(k)s that threatened to bring industry to a grinding halt. While she admits that these tasks are not yet fully accomplished, progress over the past year has been dramatic, and industry seems now more concerned with the wide-ranging issues involved in FDA reform.
A number of these have also come to rest in Alpert's office - including industry proposals for third-party review of product applications, questions about the formulation and use of guidances, and complaints about the attitudes displayed by FDA reviewers toward industry. In this interview with MD&DI, Alpert discusses these and other issues that are now being considered at ODE.
FDA has recently released a revision of its guidance on 510(k) submissions for device modifications, which has been in the works since April 1994. What was the process for creating the revised guidance?
There was industry participation and sounding as we developed the first straw man, the draft that was released for comment in April 1994. During the announced comment period, and afterward, we received even more comments from industry, and these revealed concerns in certain areas, particularly in vitro diagnostics. Other issues included what to do about labeling changes, and how to integrate working with the matrix and flowchart that were the basis of the first draft.
Once these comments had come in, [ODE deputy director] Phil Phillips and [Office of Science and Technology acting deputy director] Harvey Rudolph gathered together a group of staff to sit down and pay really careful attention to the comments. This group also contacted members of industry and worked back and forth on some of the issues.
So when members of the group were considering a comment, they might call the person who'd made it?
Yes, in some cases, if they had a question about the comment. Or, if the comment came from a certain sector of industry, they might try to address it and then go back to representatives of that sector and ask, "Are we getting there?"
When the first draft of the device modifications guidance was released, the agency said that it was intended to be "submissions neutral" - that is, it would not stimulate a greater number of 510(k)submissions or result in fewer. Has that intent been carried over into the revised version?
Well, the intent is the same, but what the actual impact will be is really not clear at this time.
This guidance provides industry with information on the issues that need to be considered along the path to determining whether to file a 510(k) or merely to document the changes and handle them as good manufacturing practices (GMP) records. The guidance carries two strong messages. First, that record keeping is critical, and that manufacturers must record why a change was made and what its impact was. And second, that manufacturers need to maintain documentation to demonstrate the continued safety and effectiveness of their product after a change is made. For instance, if a labeling change expands the use of the product into new populations or provides new indications for use, the manufacturer needs to document the safety and effectiveness of the product under those new conditions.
With regard to manufacturing changes, we're not expecting manufacturers to file 510(k)s on every change order, and we don't want to see all of them. But again, it's a question of how many changes are being made and whether they have a significant impact on the safety and effectiveness of the device. Each company will need to evaluate its changes and determine when they cross the threshold for updating the 510(k).
One major difference between the April 1994 guidance and its new version is the latter's reliance on GMPs to keep track of the manufacturing changes.
Right. If all goes well with the July draft of the revised GMP regulation, all of this will become much easier for industry and for all of us. Because the design control requirements of the GMP regulation dovetail with the modifications guidance, the new regulation should go a long way toward ensuring that manufacturers collect and record the kinds of data that inspectors need to have on hand. By having reviewers rely more on the GMP documentation that companies have, the issue of manufacturing changes can be more often eliminated from consideration during the decision-making process about whether to file a 510(k) for a device modification. This is a simpler and easier way to handle the issue than the alternative of requiring 510(k) documentation.
How does the process used to develop this guidance differ from the one used for other guidances issued by FDA?
This process illustrates what we're trying to do in the area of guidance development. We're working out a more regularized process that is distinct from the formal notice-and-comment rule making used to produce big guidances.
I think we need to recognize that there really are different levels of guidances. At the low end, there are simple checklists that tell manufacturers the things they need to have in their 510(k) submissions. For the most part, these are compilations of information that has been developed over many, many years, and the checklists just put that information into a form that's easy to deal with. This type of information has already received an industry critique as part of the back-and-forth connected with the 510(k) review process, and compiling it into a checklist really requires minimal input from outside the review group; the division knows pretty well what the problem areas are. For guidances at this level, what we are attempting to do is compile the issue in-house. After all, guidances are a result of experience with a lot of submissions, or with a cluster of submissions.
What other levels of guidance would you distinguish, and how should they be compiled?
One level has to be devoted to new technologies - areas in which enough companies are developing products so that the agency needs to be giving them a consistent message. The goal of these guidances is to identify the issues that we know exist for the technology, and to spell out the areas FDA expects manufacturers to address in their submissions.
Guidances of this type are still one step back from the big guidances, and in making them we wouldn't expect to follow formal, 90-day, notice-and-comment rule making. Instead, we'd begin by brainstorming the issues in-house, then send the guidance out to industry for a short comment period. Probably, our notice to industry would indicate that the document is an active draft that is being revised, and that we intend to implement it in several months. So industry will be given a limited period to respond, and several months later everyone will be using the guidance.
And then there are the really high-impact documents, those like the device modifications guidance, that have potential for affecting industry on a broad scale. In those cases, we're trying to get industry and the clinical community involved early on, when we're just beginning to formulate the guidance. In this way, the agency's ideas can be sounded with industry as part of the preparation of a draft guidance. The draft can then be sent out for a formal critique, the comments can be discussed and incorporated, and a revised version can be sounded a final time. At this level of guidance there would be a lot more input from outside.
So the idea is to get industry input even for documents that the agency considers to be below the threshold for notice-and-comment rule making?
That's correct. If we were to try to do formal notice-and-comment rule making for everything we are trying to provide, the time frames for their appearance would become excessively long. Providing a framework for people to work in shouldn't take six months, or a year, or two years. Often, we need something right away, because if we wait, manufacturers don't know what they should be doing with their submissions in the interim.
ODE is really committed to getting out for comment any guidances it's developing on a new technology, or where it's making a major change in a guidance. One way we're doing this is by using our panel meetings; we're trying to make the documents available at the same time that panel meetings are announced in the Federal Register, so that anyone who is concerned can read the document and come to the panel meeting to comment on it. We're also making sure that appropriate trade associations and practice associations know that the documents are available, and we're making them accessible via the electronic docket and flash fax systems at the Division of Small Manufacturers Assistance.
In some cases, we're also doing workshops. One that was quite successful was on the reuse of dialysis membranes, where we issued a draft document and then worked back and forth with the affected industry to develop guidance for reuse labeling. At another workshop, sponsored by the North American Society for Pacing and Electrophysiology, the American Heart Association, and the American College of Cardiology, we looked at a draft guidance on clinical trials that they had created. In the course of discussing that document, the agency was able to talk about its regular target requirements and what sorts of things reviewers need to see in regulatory submissions. That interplay helped to make the draft more mature; it was later sent in for agency comment and panel review.
Do you encourage organizations to draft voluntary guidances that the agency can use as straw men for soliciting further input?
Absolutely. We've told industry and the trade groups that if they recognize an area where guidance is needed, they should feel free to put together a draft that can be sent in for agency comment. The agency can then add in the necessary regulatory bells and whistles, and send it back for further comment or adoption.
We're doing the same thing with the clinical community. In the area of orthopedics, we recently participated in a meeting sponsored by the American Orthopaedic Association, the American College of Orthopedic Surgery, and the Orthopedic Surgical Manufacturers Association, to discuss how to oversee and control new technologies, and how to develop them correctly. The meeting included representatives invited from manufacturers, the clinical community, the academic community, the research community, and the regulatory community - the National Institutes of Health, the National Institute of Standards and Technology, and FDA. It resulted in identifying some key areas - particularly for orthopedic implants - where there is a need for guidances and for a consensus on what data are needed to support submissions and how to go about getting those data. Now the sponsors have an agenda for drafting additional guidances and points-to-consider documents, and for creating a reasoned approach to the development of new technologies and biomaterials testing. These issues are really important, and they affect all of us. There shouldn't be a unilateral decision on anybody's part.
How does this approach to guidances respond to the criticisms put forward in the citizen petition submitted by the Indiana Medical Device Manufacturers Council?
Essentially, the IMDMC petition asks that FDA refrain from all informal rule making. In effect, the group is proposing that all of these types of documents be developed with notice-and-comment rule making.
Congressman David McIntosh [R-IN] has announced that his panel on National Economic Growth, Natural Resources and Regulatory Affairs will be holding hearings on the IMDMC petition in September, and it will be interesting to hear the views presented there.
Is the guidance development the same as rule making?
We don't believe it is. We believe guidances and points-to-consider documents fill different needs of the community. But the petition's point is that when FDA generates these documents, essentially they are handled as though they were rules and regulations. That's the issue the group is trying to address.
Another issue that the petition points out is an important one, namely, that guidances are not intended to be rules. Guidances are signals from FDA to industry indicating the easiest path to market. It's all right if manufacturers choose to take another path - they know best how to develop their products - but doing so may impose additional requirements on them to validate and explain to the agency what they've done.
Guidances embody the issues that FDA has learned are important in a particular device area. So if a manufacturer has a different view of what the real questions are, or a different set of tests to answer the agency's questions, those will need to be explained to the reviewers. It's a bit more work, but it's perfectly acceptable.
We have made it very clear to staff that guidances are not regulations or requirements. We cannot set thresholds or absolute criteria for anything in a guidance document.
How does the agency train staff not to enforce guidances or points-to-consider documents as though they were regulations?
We at CDRH are working closely with frontline staff to help clarify the true intent and proper use of guidances, and the impact they should have. We have a lot of new staff, and guidances are the sort of tools that a reviewer can ignore, use, overuse, or abuse - and only one choice is correct. Luckily, overuse and abuse have not been pervasive at ODE. But we have been made aware of some situations in which staff members told companies that if they didn't follow the guidance, the office would not review their submissions. That's not appropriate, and we've let staff know it.
Another issue that we watch very closely is the protection of proprietary information - making sure that reviewers do not use the information in one submission to fill in the blanks of another. In the scientific community, everyone reads everyone else's papers, and the thoughts are combined to move the field forward. But reviewers are precluded from doing that. So we have to teach our staff to deal with each submission on its face. They can't take data from one and have them support another; each submission has to have its own data.
To train staff, we're conducting essentially three levels of training. First, there's general training in the process. No one comes to FDA understanding the FD&C Act, IDEs, PMAs, 510(k)s, GMPs, GLPs, and all the alphabet soup of any regulatory agency. So we have basic training.
Second, we have division-level training in each of the divisions covering certain approaches and specific issues that the division faces. Clinical laboratory devices, implants, orthopedics, and ophthalmics are each very different, so each division has its own internal training.
Third, new employees participate in a mentoring program, where they are assigned one or two senior individuals in the division with whom they have a student-teacher relationship. Review work is not taught anywhere, so new employees need guidance from someone who has been through the process. The mentors can show new employees what a good review is, and what pitfalls to avoid. They can also teach new employees what their role is vis-à-vis industry, how to communicate with sponsors, and how to document.
In addition, the CDRH Staff College in the Office of Health and Industry Programs, which is overseen by Jerry Jenson and Marvin Rosenstein, is developing core training programs. The courses in the college are for both new and seasoned reviewers. The college recently offered a course on clinical trials taught by many staff members, including [CDRH director Bruce] Burlington, me, and staff of the Office of Biostatistics and Surveillance. It covered FDA's role in the oversight and development of clinical trials, what device trials are intended to do, and the models that exist for conducting a device trial.
We also have other training programs, and they are not restricted to ODE staff. We are trying to provide an educational environment for staff - to train them to understand the GMP and medical device reporting regulations, for instance. Mostly, this training is teaching by example, by way of the senior managers who set the goals and performance characteristics for their units. And I can't think of a better way for new employees to learn.
When does a new technology reach the point at which it is possible to write a guidance, and how do you get staff up to speed on new technologies?
Two years ago I would have said that we needed to have some in-house experience with a few submissions before we could write a guidance, but that's no longer true. Industry has been tremendously willing to come in early - at the pre-IDE stage or even earlier - and to help teach our reviewers about emerging technologies. This simplifies matters considerably, because the presenters are not talking about a particular product submission, and the reviewers can instead focus on learning about the science. Then our staff can figure out where they need more training, locate appropriate courses, read up on the subject, share information, and get some self-training. And once the IDE is up and running, staff can continue to stay on top of the changing technology.
These are the kinds of areas where industry can take a lead in helping us develop guidances. It can help us identify the areas where there will be a need for guidance, explain the latest trends, and share with us the latest publications and minutes of international meetings. Industry can initiate the process by suggesting a meeting where some early decisions about the review requirements can be made.
We've seen the industry be very willing to take these steps, and we welcome that involvement.
How is the medical device industry involved with ODE now?
Industry is playing a key role in helping us bring review staff up to speed on what they do. It's very important that the review staff understand the industry, the actual technology they're dealing with, and the devices they encounter. And hands-on experience is the best teacher.
We're conducting three types of training programs that involve industry. The first is to send reviewers on industry site visits. We don't have a lot of funding for that, but we have been able to use some of our operating budget to send groups to see manufacturing sites. Many of our staff members are new to either industry or FDA. They are well-educated academicians who may not have seen the actual implementation and use of certain technologies, and may never have seen a manufacturing plant. So educational plant visits are really valuable.
A second program involves sending members of the product review team on real preapproval plant inspections. They understand the technology, and they can help the field and compliance staff understand what they are seeing. They can also make the inspection more productive and reasonable for industry.
The third activity is a vendor day program, where manufacturers bring their products into CDRH, set up displays, and talk about the basic technologies involved in their devices. This is literally hands-on experience for our staff, and it goes a long way toward improving their understanding of device technologies. We had a vendor day on defibrillators and pacers, and another on genetic amplification technologies for the clinical lab group, and we're looking forward to more. We really owe a lot of thanks to the companies that have been willing to participate; without their cooperation it couldn't happen.
Many in the device industry are now recommending that FDA adopt the European model of using third-party reviewers for both product approvals and compliance activities. How might the agency go about ensuring that those reviewers are qualified?
Well, that is going to be a challenge, and it's one of the questions we're wrestling with. We expect that one piece of the model will involve a system for accrediting those bodies, and that would include demonstrating that their staff have adequate engineering and scientific backgrounds for performing the kind of evaluation that is expected.
We will clearly have to provide some training and guidance, and we are gearing up to do that. We will need to provide the third parties with our own guidance documents, and we will clearly have to be accessible to answer questions - particularly at the beginning. Beyond that, our job would be to perform a secondary review - to make sure that the third parties have looked at the aspects that the agency considers important in comparing a device to its predicate, to review their documentation, and to determine that we agree with their recommendation.
These issues have a lot to do with how we selected the types of devices that are included in the pilot program for third-party review. The products we chose really do need oversight; someone needs to see that they actually match their performance characteristics, and that they are substantially equivalent to what is currently on the market. We intentionally selected devices with which we have sufficient experience, and for which good and complete guidances exist. We also looked for devices that could include consensus standards as part of the review, or for which the testing methods are well accepted by the manufacturing community.
Is the primary goal of the third-party pilot program to get products reviewed or to test the viability of such a program?
The pilot program will get products reviewed, but the main idea is to determine whether or not outside groups can do a 510(k) review. We're starting with simple products, and we're hoping that this is the right threshold. We know that it would be a bit much to expect a testing company to come up with a very large and sophisticated organization just for a pilot program. So we are trying to do things in a very straightforward manner, with a lot of guidance, so that existing groups can fill in the pieces they don't have and do the review.
My personal expectation is that there are groups that can perform a straightforward 510(k) review. We have been told by some outside groups that they already do a lot of similar work for both government and industry, and that they are qualified to do it.
So the shape that's been given to the pilot program may not be the eventual shape that a third-party review system would take on. When will the agency begin to evaluate the results of the program?
I think we have to have an ongoing measure of how the program is working, and that has been designed into the program. We need to be able to make midcourse corrections, to consider how the program could be developed and expanded beyond the pilot stage, and to begin preparing for that next stage. I don't know any other way to go into this type of program.
So, assuming that everything succeeds, the pilot program would grow incrementally into a full-blown program?
Assuming that things work well and that we get positive results from the program, I don't see why not. But right now, what we're committed to is a two-year trial. We are developing the model, and we are going to see how it flies and evaluate it while it's flying. The whole reason for conducting any kind of pilot program is because you are not really sure whether it's going to fly or crash.
Given that third-party-review systems are in use in Europe and elsewhere, would FDA adoption of such a model improve the accessibility of foreign markets to U.S. manufacturers?
Well, I don't know whether access to foreign markets necessitates a third-party system. That question is probably better addressed by the development of consensus standards that cover an increasing number of device development and evaluation procedures. Eventually, there would be enough standards in place that simple assessment of compliance with them would constitute the review, with only labeling and regional differences needing to be evaluated separately.
What we see as the advantage of the third-party model is that it enables the agency to face this period of shrinking resources, when there are fewer people and financial resources to perform a large volume of work. We need to continue a program that ensures that all of the products on the market are safe and effective, and one way to do so is to have more than just FDA participating in the evaluation of products.
New technologies can significantly affect the nature of health care, but they require a lot of upfront attention if they are to be given a streamlined route to market. FDA would still be seeing the newest of the new, but there are other groups that can facilitate getting and maintaining other products in the marketplace.
We don't think old products ought to suffer for new, or that new products ought to suffer for old. They all ought to get attention, and that's going to require some managing on our part.
Are you concerned that implementation of a third-party system for product approvals would put your office out of business?
I'm not at all worried. However, that is a very-often-discussed concern at certain levels within the staff.
If you look at other jurisdictions around the world, you can imagine why. In the United Kingdom, only medical device reporting and related areas remain as government functions; otherwise, the system is almost totally privatized. For device evaluation, the UK system is funded totally outside the government. In some of the other European countries, it's all notified bodies with minimal government involvement.
But our process, issues, population, and health-care needs are quite different from some of those other jurisdictions. Not better, not worse - just different. The U.S. system has developed differently and has different approaches to what is expected in the delivery of any technology for health care.
I am not worried that we are building our own demise. I think there is plenty of work. The challenges of brand-new technologies, of developing guidances and evaluation procedures for them, and of getting them to market convince me that we are not putting ourselves out of business.
Do you think that the focus of your office might shift toward reviewing the reviewers?
There is concern that we would be regulating at a distance - regulating the people who are actually regulating. Anything is possible, and I think that there may be some of that in our future. We will be smaller in the future, and I think we will be focusing our attention differently on how we regulate technology and new clinical advances. And while my crystal ball doesn't work well enough to really say for sure, we will probably be doing some of our work through other parties.
(This article originally appeared in the November, 1995, issue of Medical Device & Diagnostic Industry. © 1995 CanonCommunications, Inc. All rights reserved.)