Artificial Intelligence (AI) and Machine Learning (ML) are poised to revolutionize the field of healthcare. Researchers are leveraging deep learning methods to find new ways to efficiently diagnose and treat diseases.
Although lacking a well-articulated AI strategy, the United States invested an estimated $2 billion on research and development for AI-based technologies in 2017. Since that time, the Department of Defense has also committed to providing up to an additional $2 billion per year in spending for AI technology and infrastructure over the next five years.
Despite such investments, bringing an AI or similar software-based product to market is often delayed because of the hurdles involved in FDA regulatory pathways. In recognition of this inefficiency, FDA established the Software Precertification Program, which is intended to streamline the FDA approval process for AI and ML-based medical technologies. Taking this a step further, the agency said earlier this month that it will consider a new regulatory framework for reviewing medical devices that use advanced artificial intelligence algorithms.
The conventional FDA approval process for marketing new medical devices is an arduous and conservative pathway, driven by policies and procedures that are intended for hardware-based medical devices. Overall, the entire FDA medical device approval process takes an average of 3 years to 7 years. FDA device review can take anywhere from 3 months to 12 months or longer, dependent upon the medical device category and data supporting safety and effectiveness. The process is also expensive, with typical costs to bring a device for FDA review ranging between $10 million and $20 million
This FDA review process is intended for all medical devices requiring FDA approval, including those involving AI and ML technologies. Thus, FDA oversight of AI and ML is far-reaching and even applies to software as a medical device (SaMD). Such devices include most software and mobile apps intended to treat, diagnose, cure, mitigate, or prevent disease or other conditions as medical devices. This rigid framework is ill-equipped to deal with AI-based software technology that changes in near real-time based on responses to real-world performance.
AI creates a further unique problem under the current regulatory scheme because there is often lack of a tangible device. Instead, the FDA regulatory framework requires the evaluation of software code to assess the accuracy, reliability, and safety of AI-based healthcare. However, such code typically does not directly address the specific FDA metrics (e.g., safety, efficacy) required for FDA approval. Furthermore, a significant advantage of AI and ML medical devices is that they are frequently updated, in some cases in near-real-time, based on real-world data. However, the current FDA approval process is designed for devices that may be updated quarterly, annually, or even less frequently.
Software Precertification Program
The Software Precertification Program is a voluntary pathway that embodies a regulatory model tailored to assess the safety and effectiveness of AI-based software technologies without inhibiting patent access to the technologies. The foundation of the program is the identification of medical device manufacturers that have demonstrated a robust culture of quality and organizational excellence and are committed to monitoring the real-world performance of their AI-based technologies.
The program launched in 2017 as part of the Digital Health Innovation Action Plan, and was limited to FDA-regulated SaMD, defined as software intended to be used for one or more medical purposes that perform these purposes without being part of a hardware medical device. Instead of the traditional FDA approval process of focusing on the eventual product, the program focuses on the software or digital health technology developer. The developer/company-first approach should, in theory, remove traditional regulatory hurdles and permit trusted companies to harness the advantages of AI to quickly and effectively address safety concerns and respond to adverse events when they arise.
In response to a notice seeking voluntary participation, over one hundred companies applied to be included in the program. FDA used selective metrics such as product quality, patient safety, clinical responsibility, clinical responsibility, cybersecurity responsibility, and proactive culture to identify companies that exhibited organizational excellence. Apple, Fitbit, Johnson & Johnson, Pear Therapeutics, Phosphorus, Roche, Samsung, Tidepool, and Verily were selected as the initial nine trusted SaMD manufacturers to participate in the program. Voluntary participation required the companies “to provide access to measures they currently use to develop, test, and maintain their software products,” including methods for post-market data collection, and allowance for FDA site visits.
Earlier this year, FDA released the three key initiatives that outline the next phase of the program. First, FDA released guidance intended to explain the framework for the program under the agency's current regulatory authorities. Specifically, the program will be implemented under the de novo pathway, traditionally used for approval of lower risk medical devices. Under the de novo classification process, new types of low to moderate risk devices can obtain market authorization as a class I or class II device rather than being designated as a class III device, which requires premarket approval.
Pilot participants with a SaMD product eligible for the de novo classification may have the opportunity to participate in an excellence appraisal wherein FDA intends to evaluate the excellence principals of the organization that correspond to de novo request content. The results of the excellence appraisal and the records supporting the appraisal will be collated in a device master file to support the de novo request and to support a future premarket submission. The excellence-appraised manufacturer could then participate in a streamlined pre-cert de novo request in which the manufacturer submits any additional information not already in the master file to support that those general, or general and special controls can provide a reasonable assurance of safety and effectiveness.
After substantive review of the master file and additional information FDA would classify the device by written order and for a class II device establish special controls such as post-market data collection, changes in appraisal data, or post-market real-world performance data necessary to assure safety and effectiveness of the device type. The program is in its infancy, but its goal is to “determine the contours of a possible regulatory model that provides efficient regulatory oversight of certain software-based medical devices from manufacturers who have demonstrated a robust culture of quality and organizational excellence (CQOE) and are committed to monitoring real-world performance while assuring that these devices are safe and effective.”
Second, FDA released a pre-cert test plan for 2019 that outlines testing related to refinement and implementation of the program. The goal of the test plan is to determine how the program can ensure safe and effective products. FDA will compare parallel submissions made to the de novo route and to the traditional route in an effort to determine whether the program might provide efficiencies over the conventional FDA medical device approval process. FDA proposes the use of both retrospective and prospective tests under the pre-cert test plan. The retrospective arm will be used to evaluate elements applied systematically across all SaMD products at the organizational level during the excellence appraisal, whereas product-specific elements will be reviewed in the premarket submission.
The elements of a previously-reviewed complete submission package will be reviewed retrospectively to iteratively refine the excellence appraisal. Initially, FDA intends to test program components of the model using a mock standard review package. Sponsors who opt-in to the prospective approach will submit traditional and excellence appraisal submission requirements. FDA’s Center for Devices and Radiological Health (CDRH) will assemble and review a mock streamlined review package to determine if sufficient information exists in the package to assure reasonable safety and effectiveness. However, an official regulatory decision would be based on traditional regulatory submission. A finding that excellence at the organizational level is correlated with excellence in design, developing and testing a SaMD product would support the use of the excellence appraisal approach as part of the product premarket authorization.
Third, FDA launched an updated working model that incorporates the regulatory framework and test plan. The working model describes the proposed implementation approach and future vision for the program.
The program appears to be a concerted effort to streamline the regulatory process for software-based medical devices. However, many questions remain. For instance, it is not immediately clear where the bar will be set for companies to achieve the necessary CQOE required to participate in the program. The initial slate of companies includes well-known global businesses with substantial product portfolios and a wealth of data to address the criteria for inclusion. Thus, it remains an open question whether, and how, startups and other less established companies might become certified. In addition, it is unclear whether FDA will have authority to force a recall on companies and/or their products in the program. Also, FDA will need to consider situations where companies should have implemented a recall but failed to do so. Furthermore, the protection of private information will be an important concern for AI and ML innovations that utilize large datasets that involve numerous patients and their sensitive personal information. Such privacy issues will likely represent substantial hurdles that the agency must address moving forward.
Importantly, the United States currently lacks a well-defined AI strategy to address the rapid rise of big data and its impact on health technologies. As a result, full adoption of the Software Precertification Program appears to be years off, with only traditional FDA approval methods currently available for AI-based health technologies. Other FDA AI-based qualification programs, such as the Medical Device Development Tools program, remain in the nascent stages and have not been widely utilized by AI and ML developers. Thus, today’s AI-based medical technologies continue to be subjected to lengthy regulatory timelines and risk being outdated by the time they are approved for commercialization.