How to Submit Preclinical Data to ODE

FDA’s reviewers see device sponsors make common errors when submitting preclinical test data. A bit of forethought can help firms avoid these mistakes.

+2
Elisa Harvey, Semih Oktayand 2 more

July 1, 2008

16 Min Read
How to Submit Preclinical Data to ODE

REGULATORY OUTLOOK


Harvey

Companies wishing to submit investigational device exemption (IDE) applications or marketing applications to FDA must provide clear, concise documents that include a description of the preclinical evaluation of the device. Guidance is available on FDA's Web site (www.fda.gov/cdrh/devadvice/). Even with such resources, companies tend to make common mistakes that cause FDA to find the documents lacking. Luckily, the such errors made in submissions can be avoided.

Oktay

Preclinical Evaluations

Preclinical evaluation includes all nonclinical (i.e., not performed in humans) testing done to support an IDE or marketing application to FDA. It may include bench, animal, biocompatibility, and sterility testing.

Chwirut

Use Clinically Relevant Acceptance Criteria. It is common to see submissions with preclinical test descriptions that include pass-fail or acceptance criteria, but do not include justification or rationale for how those criteria were chosen. One of the most important pieces of information that FDA considers in determining whether testing provided is sufficient to support the application is the relevance of tests to the proposed intended use. There must be an explicit connection between a test's pass-fail criteria and the expected exposure of people to the device given its intended use. FDA's first question is: How do such criteria make sense for this device and its intended use?

Danielson

Without a demonstration that the acceptance criteria are logical and based on scientific rationale, FDA cannot conclude that the testing is useful, even if all devices pass the stated criteria. Criteria must be clinically meaningful. There are many consensus standards and FDA guidances that address and identify appropriate pass-fail criteria. For some novel technologies or intended uses, this information may not be so readily available and may need to be developed de novo by the sponsor. In such cases, it is advisable to check with FDA on appropriate pass-fail criteria.

Test in Appropriate Environments. It may seem obvious that device testing should occur in a way that represents actual anatomical and physiological conditions, but FDA commonly receives data from tests that were not done in the most appropriate environment.

A device that will be exposed to the bloodstream should be bench tested at body temperature and in fluid. Even then, depending on the test, a solution such as 0.9% saline may be adequate, or a blood analog such as glycerol with viscosity closer to that of blood may be required. Testing in room-temperature air would likely not be appropriate.

With respect to biocompatibility testing, some articles may need testing in both polar and nonpolar extracts depending on the type of materials and expected exposure. Carefully consider the expected environment during planning and designing the preclinical test.

In addition, if the device has an implantable component, the agency will request separate testing of the implanted device and delivery tool.

Do Homework Early. Often, FDA sees tests that were rushed in an attempt to meet internal company deadlines. Companies must adequately research similar devices and appropriate standards and guidance documents. Companies should also develop comprehensive risk analysis, and, if needed, check with FDA to ensure the analysis is sufficient. Waiting until verification and validation are complete to perform these tasks is likely to result in gaps in the testing that FDA will easily find. It is best to delay submission testing until it is determined that the test design and methods are the most appropriate to demonstrate the safety, efficacy, and performance of the device.

Sidebar:

Avoiding Submission Errors

Justify Deviations from Guidance and Precedent. Standards, FDA guidance documents, and regulatory precedents can be useful to sponsors developing a preclinical test plan. But frequently such resources are only partially relevant. Sponsors often must modify the test methodology to accommodate unique features of a device. As long as these deviations are accompanied by good justification, it is reasonable to assume that FDA will accept the sponsor's position.

However, companies might either not indicate where they have deviated from a standard protocol, or not describe why the deviation was necessary and appropriate. In such a situation, FDA has difficulty accepting the data. The agency may conclude that the testing is irrelevant, or worse, it may conclude that the deviations were made to skew results in the sponsor's favor.

The onus is on the sponsor to articulate any deviations and the rationale for them. However, the sponsor should never assume that justifications are accepted or understood by FDA without prior discussion.

Select Adequate Models. Selecting an appropriate test model is absolutely critical to the success of an FDA submission. Failure to choose the best animal model, for instance, can waste huge amounts of money and time, especially if the model is a large animal species and the testing includes long-term (several months) follow-up. Although it is becoming common for companies to check with the appropriate branch and reviewers before embarking on such complex testing, some companies still submit animal testing for which FDA has never had the opportunity to provide feedback. Something as simple as failure to perform needed histopathological analysis of device explants or tissues, even if the rest of the animal study was adequate, can result in the need to repeat the entire study.

Avoid Poor Study Design. A poorly designed preclinical study, no matter how well conducted, will be of little use to a sponsor or FDA. All studies take significant time and money to accomplish; therefore, spend time up front determining the exact objective. The only way to be certain that a particular study can meet the scientific objective and the practical objective (i.e., be acceptable to FDA in supporting the application) is to rely on FDA's guidance.

If written guidance documents, FDA-recognized standards, or clear precedents are not available, check with the branch or review team to determine whether the investment is worthwhile. It is the sponsor's duty to check whether a study design will be considered acceptable to FDA.

Write Test Reports for an FDA Audience. During the course of device development, many companies conduct preliminary tests for purposes other than submission to FDA. Frequently the agency receives applications containing documents that were never intended for an FDA audience. Such testing may be used in the submission, but firms should tailor them first.

For example, statistics may be inappropriate for newly stated hypotheses. Reports that were originally written for a different audience or with another purpose in mind need to be restructured to provide a level of detail that enables the FDA reviewer to fully understand the purpose, methods, and results of the test.

If the reports are vague or lack sufficient detail, the FDA reviewer may struggle. It is important to remember that anything the sponsor can do to make the reviewer's job easier will ultimately be an advantage. Failure to present data appropriately can result in FDA responding with deficiencies. Such deficiencies require a resubmission. The best practice is to have technical staff, whenever possible, write all laboratory test reports as if they will be submitted for publication to a peer-reviewed journal or for submission to FDA.

Identify the Test Lab. Something as simple as failure to identify the test lab that has conducted the tests can generate deficiencies from FDA. Not identifying a contract test lab may leave the mistaken impression that the work was done in-house. In addition, many contract test laboratories have excellent reputations and are familiar to FDA. Detailed contact information for all test laboratories that have conducted any of a sponsor's testing should be provided in the test reports.

Avoid Vague Descriptions. Providing sufficient detail on preclinical work is paramount. FDA reviewers are detail-oriented individuals who must be able to reconstruct and understand the entire test in their own minds. Nebulous references to test materials or procedures are unacceptable. Further, they hinder the application from moving forward until details have been clarified. Companies often find they need to take an extra review cycle to submit information that could have easily been provided in the first place. It's a waste of time and resources. A good idea is to have a person or two not directly involved with the project read the reports for clarity and level of detail.

Justify Sample Sizes. One of ODE's most commonly noted deficiencies relates to a sponsor's failure to justify the sample sizes chosen for preclinical testing. The number of bench test articles chosen or the number of animals used in a preclinical study should be based on a logical justification, statistical or otherwise, and not be arbitrary. Test sample sizes must have a rational basis in the same way that acceptance, or pass-fail criteria, do. Before conducting a test, the number should be chosen depending on the objective of the test, available guidance from FDA, and standards as to what is considered acceptable. Choosing a sample size because it seems to be enough, or because someone else did the test that way, might not be adequate rationale.

Not all preclinical tests need to be based on statistical hypotheses. Some can be based on the expected forces and stresses a device may encounter in use. For example, if a firm expects to label its cardiovascular catheter to make no more than two passes in normal clinical use, then a test that evaluates device performance with 10 passes may provide an adequate safety margin. Regardless of the parameters, the sponsor should include a clear statement indicating why the sample size was chosen for every test.

Include Raw Data. Another common error is to omit raw data from all preclinical tests. Sponsors often provide only the summary reports written by themselves or the contract laboratory. Providing only summary data almost always results in a significant deficiency. As noted earlier, reviewers want as much information as possible so that they can come to the independent conclusion that the data are valid and acceptable for supporting the application. Reviewers do not simply accept conclusions made by either the test laboratory or the sponsor without being able to verify them through raw data.

If there is a great deal of raw data, there may be a certain format for presenting this information that will make it easier for the FDA reviewer. Sponsors should check with the agency to see whether there is a preferred way of presenting such information.

Submission Quality

A submission that appears of poor quality can leave the impression that the science and entire operation is not of sufficient quality, even if all the actual science provided in the submission is good. Grammatical and typographical errors can be easily avoided by implementing several proofreading steps.

Refine Intended Use and Indications for Use. It is common for submissions to have either no statement of intended use, or a statement of intended use that is too vague. The statement of intended use is the basic starting point (from FDA's perspective) that determines the regulatory pathway, risk-benefit analysis, required testing, and need for clinical data. It should be a clear and concise statement of exactly what the device is intended to do and the specific patient population for whom the indication is intended. FDA must have a proper context in which to frame the device, what it does, and where. Sponsors frequently need to craft intended use or indication much more carefully than originally anticipated. Every word in the intended use is important. Leaving in or taking out particular words may greatly influence the needed clinical data or preclinical testing. Examine the intended use statements in available regulatory precedents such as other 510(k)s, premarket approvals, or IDEs. In addition, speak with FDA about your proposed intended use.

Intended use is the first decision-making point in FDA's 510(k) Substantial Equivalence Decision-Making Flow Chart (after determining whether the product is in fact a device). Therefore, carefully refining the statement of intended use and the proposed indications for use should be among the first decisions a sponsor makes.

Fix Poor Grammar. Poor grammar is common in submissions, especially those from sponsors outside the United States who may not use English as a first language. But even U.S. sponsors submit documents that contain grammatical and spelling errors. There simply is no reason for this—build proofreading into the development of the application. It is not worth rushing an application to meet an internal deadline if the submission is full of errors that frustrate a reviewer and cause suspicion of overall quality. The application is a product that is as worthy of quality control as the device it represents.

Provide Adequate Description of Data. Inadequate data description forces the reviewer to issue deficiencies and request more complete descriptions. Although it may be impossible to anticipate every single item a reviewer may want to see, there are certain procedures a sponsor can employ to anticipate requests. All raw data and test reports should be submitted. Ensure that the submission is in a comprehensive and clear format suitable for publication in a peer-reviewed journal. Place more emphasis on presentation of the results than on the summary and conclusions. Check FDA's Web site for information regarding appropriate content for every kind of application.

Sidebar:

Top 10 Errors in Preclinical Testing Submissions

Report Prior Investigations. Sponsors that are focused on providing the specific information for their device often overlook prior investigations. However, prior investigations data are particularly critical for IDEs and premarket approval applications. Such data provide a framework and context for the sponsor's device application. Sponsors often provide insufficient information or do not provide a full report of all relevant information (e.g., unfavorable information that could affect how FDA reviews the application).

FDA is well aware of this common oversight. Reviewers often ask sponsors to list specific search criteria used in a PubMed search, for example, to ensure that the report of prior investigations is unbiased. As difficult as it might be, it is important to provide information (even unpublished or unfavorable data) for the fullest picture of the device. Usually such information comes to light eventually anyway, and at that point it may cause more problems for a sponsor than it might have were it provided earlier.

Another area of confusion is deciding which testing must be submitted if there have been multiple iterations of a device, or a great deal of preliminary research and development testing. Usually FDA only needs to see the testing specifically done on the version of the device that is intended for market; it is not necessary to submit earlier R&D testing. However, sometimes a sponsor wants to use testing results from an earlier version to support an application for a later version of the device. This is possible if the sponsor provides a link between the testing from the earlier device and the version that is intended for market. For example, include some type of bridging data and justify why the testing is representative of what would be obtained if the testing were done on the final version of the device.

Provide an Adequate Investigation Plan. This element refers primarily to IDE applications. An overall investigation plan includes the specific purpose of the proposed study, detailed protocol, risk analysis, device description, and monitoring procedures. Specific areas that are often lacking include risk analysis, device description, and monitoring procedures.

All such elements must be detailed so that the reviewer can fully understand the plan and, more critically, be convinced that the sponsor understands the plan. Without sufficient evidence that the proposed study is scientifically sound, well conducted, and well monitored, the result could be deficiencies or even disapproval of the application.

Provide Design and Manufacturing Information. Although IDEs have somewhat reduced requirements for providing manufacturing information, data on design controls and manufacturability for the study are required. Such information assures FDA that fundamental safety has been established. At the least, describe the methods, facilities, and controls used for the manufacture, processing, packing, storage, and installation of the device.

In addition, FDA says there is often “inadequate characterization or description of the device and its operation due to inadequate or omitted design and engineering drawings of device, rationale for device design, device and performance specifications, description of materials (including biocompatibility information), description of function (i.e., how the device and its components or subsystems work together to achieve desired function), or validation testing for the subsystems and main system.” With specific reference to manufacturing, companies must provide description of the design controls that ensure devices are produced consistently and as designed.

Consider Risk Analysis. Risk analysis is another overlooked (or at least insufficiently addressed) element of submissions. FDA considers risk analysis a fundamental part of deciding whether testing addresses possible failure modes. Please note: a checklist of tests in a guidance document or standard does not obviate the need for a detailed risk analysis. The test plan can only be deemed adequate if both the sponsor and FDA have a complete understanding and description of all the risks associated with the device for the intended use in the indicated patient population.

In addition, the sponsor should provide corresponding description of how each risk is mitigated or minimized via the device design, instructions for use, selection of patient population, and justification for the sample size. Unless the testing somehow addresses all known anticipated risks, it is unlikely that FDA will come to the conclusion that the device is sufficiently safe for human use in the context of an IDE.

Risk analysis is often an afterthought, but it shouldn't be. It should come before any testing has been conceived. Properly thought out and comprehensive risk analysis informs much of the device development plan, including the specific parameters for all of the preclinical and clinical testing.

Conclusion

Details are important when developing a preclinical test plan in anticipation of submitting to FDA. Consequently, mistakes can be very costly in terms of time, money, and lasting impressions left with the agency. It pays to attend these details up front to maximize success with submissions and preclinical testing the first time around. Fortunately, there are abundant resources available for sponsors wishing to navigate the regulatory waters of FDA.

Elisa Harvey and Judy Danielson are senior regulatory consultants at CardioMed Device Consultants LLC (Sandy Spring, MD). Daniel Chwirut is vice president of engineering, and Semih Oktay is president and founder of the company. The authors can be contacted at [email protected], [email protected], [email protected], and [email protected], respectively.

Copyright ©2008 Medical Device & Diagnostic Industry

Sign up for the QMED & MD+DI Daily newsletter.

You May Also Like