Abbott Defends the Accuracy of Its Rapid COVID-19 Test

This micro-podcast episode of Medtech in a Minute gives you the details you need most on this developing story. Scroll down to read the full story.

What's going on?

Several independent research teams have called into question the accuracy of Abbott's ID NOW COVID-19 test, a rapid point-of-care molecular diagnostic test designed to produce results in five minutes. FDA granted Abbott an emergency use authorization of the test in late March.

The accuracy of the test was first questioned by researchers at NYU Langone Health. The NYU researchers conducted a study involving 101 patients receiving emergency COVID-19 care through the system's Tisch Hospital. The study, which was pre-published on bioRxiv and has not been peer reviewed, suggested that Abbott's test produced false negative results for almost half of the positive samples.

Conducted by researchers at NYU Langone Health, involving 101 patients receiving emergency COVID-19 care through the system’s Tisch Hospital, the study was pre-published on bioRxiv, but it has not been peer-reviewed and the journal cautions that the study should not be regarded as conclusive or be reported in news media as established information.

So is the test unreliable, or the study results?

It's more complicated than that. See, each patient in the NYU study had samples collected twice at the same time, but not at the point of care like the test was designed for. Instead, the researchers transported the samples to the hospital's laboratory within one or two hours and then tested using both Abbott's ID NOW machine and Cepheid's Xpert Xpress test.

"While we understand no test is perfect, test outcomes depend on a number of factors including patient selection, specimen type, collection, handling, storage, transport and conformity to the way the test was designed to be run. ID NOW is intended to be used near the patient with a direct swab test method," Abbott said in a statement released Thursday.

Since the NYU study results emerged in the media, other research teams, including a team from Cleveland Clinic, have conducted similar studies. The Cleveland Clinic study, which also has not been peer-reviewed yet, found the test correctly detected the virus in 85.2% of the 239 samples tested. That implies a false negative rate of about 15%. But once again, it's unclear if the samples in the study were inserted directly into the machine using a dry swab as instructed, or if the samples were first stored in transport solution.

The company added that when a direct swab test method is used, the test performs as expected.

"We're seeing studies being conducted to understand the role of ID NOW in ways that it was not designed to be used. In particular, the NYU study results are not consistent with other studies," Abbott said. "While we've seen a few studies with sensitivity performance percentages in the 80s, we've also seen other studies with sensitivity at or above 90%, and one as high as 94%."

What is Abbott doing about the allegations of false negatives?

Recognizing a responsibility to provide healthcare professionals and the public with accurate information, Abbott said it will further clarify its product information to provide better guidance to healthcare providers that negative results should be considered in the context of a patient's recent exposures, history, and the presence of clinical signs and symptoms consistent with COVID-19.

"Negative results should be presumed negative, but if inconsistent with clinical signs and symptoms or necessary for patient management, should be tested with an alternative molecular assay. We are also reinforcing proper sample collection and handling instructions. We are communicating this to our customers," the company said.

Abbott also noted that it is continuing to optimize the test as the world learns more about the virus, and will incorporate those learnings into the test as it does with all of its diagnostics tests.

And finally, Abbott has agreed to conduct post-market studies for the ID NOW device that each will include at least 150 COVID-19 positive patietns in a variety of clinical settings.

What does FDA say about the test's accuracy?

FDA issued a public alert late in the day Thursday addressing the issue. The agency acknowledged the early data that suggests potential inaccurate results from using the Abbott ID NOW point-of-care test to diagnose the virus that causes COVID-19. Independent studies aside, FDA also disclosed that it has received 15 adverse event reports about the ID NOW device that suggests some users are receiving inaccurate negative results.

“We are still evaluating the information about inaccurate results and are in direct communications with Abbott about this important issue," said Tim Stenzel, MD, director of the Office of In Vitro Diagnostics and Radiological Health in the agency's Center for Devices and Radiological Health. "We will continue to study the data available and are working with the company to create additional mechanisms for studying the test. This test can still be used and can correctly identify many positive cases in minutes. Negative results may need to be confirmed with a high-sensitivity authorized molecular test."

FDA also emphasized the fact that no diagnostic test is 100% accurate due to performance characteristics, specimen handling, or user error, which is why it is important to study patterns and identify the cause of suspected false results so any significant issues can be addressed quickly. The type of swabs used in the studies or the type of viral transport media used (especially given that the test is designed to be used with a direct swab-to-machine method) are all factors that may have influenced the study findings, FDA said.

"While there is important information to gather from these studies, it should be noted these studies have limitations, including small sample size, potential design biases, or tests that may not have been executed according to the manufacturer’s instructions for use, an important part of scientific research. This is why external scientific studies are one part of the FDA’s overall evaluation of a diagnostic performance," the agency said.

As for the adverse event reports FDA has received, the agency said such reports can be incomplete, inaccurate, or unferified, so agency staff must "meticulously comb through the reports to identify crucial data to support any signals or patterns about device use."

FDA said it will also reveiw interim data from Abbott's post-market studies on an ongoing bases to inform any additional actions the company or the agency should take.

Even if the independent study results are unreliable, they may reflect how the ID NOW is actually being used in a real-world setting. After all, if the NYU and Cleveland Clinic researchers didn't handle the samples correctly, it's possible that some frontline clinicians are making similar mistakes.

A few things to keep in mind about EUAs

First and foremost, it is important to remember that an emergency use authorization (EUA) is not equivelent to an FDA approval or FDA clearance. In fact, it's suprisingly easy to get an EUA for a SARS-CoV-2 test.

FDA can issue an EUA for a COVID-19 diagnostic test if the agency concludes:

The type and quantity of information needed to support an EUA differs from the data required to initiate clinical trials or to approve or clear a product. Also, FDA issued an EUA for Abbott's ID NOW COVID-19 test (and others) within 24 hours of receiving the request form and supporting data.

It's also important to keep in mind that all of the COVID-19 tests that are on the market under an EUA were developed insanely fast. In March, Norman Moore, director of scientific affairs and infectious diseases at Abbott, told MD+DI that developing new molecular tests usually takes year, but Abbott's scientists worked around the clock to compress the development timeframe down to just a few weeks.

FDA drew criticism early in the pandemic for having too much red tape and beurocratic hurdles that delayed the EUA process for many test developers. In recent weeks, however, critics have accused the agency for swinging too far in the opposite direction and rubber stamping tests in an effort to get them to the COVID-19 frontlines as quickly as possible. But too many people took advantage of the agency's flexibility and a rash of fraudlent antibody tests have turned up on the market as a result. To combat this trend, FDA recently decided to  revise its policy concerning the regulation of COVID-19 antibody tests, saying that "flexibility never meant we would allow fraud".