Medical Device Software Standards: Vision and Status

Originally Published in MDDI May 2001

SOFTWARE

For manufacturers of software-containing devices, a clear understanding of all relevant standards is crucial. This article takes a close look at the past, present, and future state of those standards.

Sherman Eagles and John Murray

In a time when software products can be rendered obsolete in six months, long regulatory review periods can slow the delivery of new medical devices to the market. At the same time, with software controlling more and more critical medical device operations, the consequences of software failure in a medical device can be severe. The problem for the medical device manufacturing industry and its regulators, then, is twofold: it must deliver new software-containing products to the market in a timely manner, while at the same time protecting the public from software-related device failures.

SOFTWARE ENGINEERING

Understanding the role standards can play in producing safe and effective software-containing medical devices begins with a simple but fundamental premise: safe medical device software depends on sound software engineering practice. While sound software engineering alone is not sufficient to ensure safety, without it no claim to safety can be made.

Supporting this premise requires an examination of precisely how software differs from hardware. One difference is that almost all software problems are the result of errors made during software development. Software failures are systematic, not random, in nature. Software does not wear out and fail; it contains any and all errors the day it is first installed. It follows that software quality is necessarily a preproduction (design) issue, and generally not a manufacturing concern.

A second difference is the complexity of software. Because software executes different paths based on differing inputs, it is not practical—even in simple programs—to fully test all possible execution paths. This makes it nearly impossible to evaluate whether software is error free. Thus, the quality of the software cannot be determined by software testing alone.

A third difference between software and hardware is flexibility; software can be more easily changed late in the development cycle. This provides a great deal of flexibility for working around hardware challenges or other problems discovered during development. The ease with which software can be altered, however, can lead to the assumption that software errors are easily correctable. But changes to one component of software, even those that seem inconsequential, can create unexpected problems elsewhere.

Designing sound software, implementing it, and ensuring that changes are complete and correct requires a disciplined, methodical approach. Such an approach is what is meant by the term software engineering.

Establishing that software has been developed using good software engineering requires a common understanding between FDA and industry of precisely what the term means. Once that understanding is achieved, manufacturers can more easily demonstrate that these practices are in place and that risk management has been adequately performed.

If a manufacturer can establish that a medical device's software was developed using good software engineering practices, the premarket review of that device can concentrate on the risks associated with the device and how best to manage them. Eliminating questions about software development allows a quicker and more consistent premarket review.

THE ROLE OF SOFTWARE STANDARDS

The role of software standards is to precisely document and define what constitutes good software engineering practices. Figure 1 illustrates this role by connecting safe software development to regulation. Beyond this role, software standards have two additional benefits: they reduce or eliminate premarket submission documentation for the software components of medical devices, and they enhance global harmonization.

Eliminating Software Documentation. A long-term goal for software standards use at FDA is to create a regulatory environment that minimizes the need for the premarket submission of most software documentation. This goal would enable FDA to reduce the time and cost of premarket reviews for devices containing software. Industry would benefit as well by having more concise premarket submissions and a quicker and more consistent review process.

Global Harmonization. A second long-term goal is to address the issue of global harmonization. Worldwide software standards that are accepted as evidence of the acceptable engineering of medical device software would provide consistent guidance for manufacturers. If these standards require the creation of appropriate engineering information during the software development process and this information meets regulatory concerns, there should be no need to repackage or restructure software information for regulatory review.

SOFTWARE STANDARDIZATION ACTIVITIES PRE-FDAMA

Software Quality Audits. One of the first efforts to bring standardization to the regulation of medical device software came in September of 1996, when a workshop was held at the National Institutes of Health (NIH) to discuss the formation of a policy for the regulation of medical software devices. As a result of this workshop, an effort was started to develop a software quality audit (SQA) system that could be used in lieu of the traditional premarket review process for software.

The idea was to develop a method for manufacturers to follow when undergoing the auditing process that adhered to a specified, FDA-approved audit process. Manufacturers who followed the audit procedure (and who had no serious, negative findings) would be allowed to market their devices without undergoing the traditional premarket software review.

The SQA effort focused on stand-alone software and did not address the issue of embedded medical device software applications. An important component of the effort was to create an architecture for the SQA document. The architecture called for a "cherry picker" approach to the medical device software issue—many software quality standards would be reviewed and the best elements would be selected for incorporation into an SQA process for medical device software. A task force was established by what was then HIMA—now AdvaMed—and work began to define a suitable audit process and identify the appropriate elements of existing software standards.

IEC 60601-1-4. The International Electrotechnical Commission (IEC) first published "IEC 60601-1-4 Collateral Standard: Programmable Electrical Medical Systems" in May 1996. It is commonly referred to as the "1-4" standard. This standard required that a process be followed by manufacturers that included risk management and development activities. Manufacturers would be required to establish a record of this process to support the safety claims of medical electrical equipment that incorporated programmable electronic subsystems.

In response to this publication, FDA raised the question of whether the 1-4 standard was a software standard or a risk management standard. The agency believed that the 1-4 standard by itself was not sufficient to be used for software and therefore was not willing to use it as part of its regulatory process.

At the same time, however, FDA indicated that the 1-4 standard was useful as a risk management standard for medical device systems and that there would be a benefit in following it. Nevertheless, FDA did have four main concerns: one, IEC 60601-1-4 was written as a risk management standard and was not designed to address effectiveness; two, it only covered the design portion of a product's life cycle and did not address maintenance or retirement; three, the standard addressed system-level risk management concerns but did very little to address the engineering of software; and four, IEC 60601-1-4 did not cover all the medical devices that are regulated by FDA (i.e., stand-alone software).

THE AAMI SOFTWARE COMMITTEE

AAMI's first attempt at software standardization came in response to IEC 60601-1-4. Spurred to action by FDA's concerns with the 1-4 standard, the AAMI software committee attempted to generate a software standard that met agency requirements for safe and effective software. The idea was to take the basic foundation of IEC 60601-1-4 and add to it the software engineering parts necessary to meet FDA's need to ensure safe and effective software.

This effort drew a lot of attention, but was ultimately an attempt to draft a standard that addressed a yet-undefined set of requirements. In 1997 and 1998 there were several large AAMI software committee meetings, but these meetings failed to establish a consensus, and little was achieved. Progress was thwarted for several reasons. First, there was no hook in the regulatory environment; this was pre-FDAMA, when there was no established method for CDRH to use standards in the regulatory process. Second, there was no demonstrated traceability to existing IEEE software standards or IEC/ISO JTC1 software engineering standards. The material contained in the first drafts of AAMI's effort was not well connected with established software engineering standards. There was a definite need to show that the AAMI proposal was actually based on established software engineering standards being applied to the medical device industry. Unfortunately, this was not happening.

Table I: Four-level model of standards.

SOFTWARE STANDARDS ACTIVITIES POST-FDAMA

The CDRH Software Standards Task Group. The signing into law of the FDA Modernization Act (FDAMA) in November of 1997 enabled FDA to use standards in its regulatory process. By December of 1997, CDRH had formed 13 standards technical groups (STGs) to deal with 13 different areas of medical device standardization, one of which was the software STG. The CDRH software STG evaluates software engineering standards, makes recommendations for the recognition of software engineering standards, and provides guidance for the software engineering standards program.

In its search for software standards that met its regulatory needs, the software STG researched and reviewed more than 315 standards, guides, and technical reports related to software engineering. At the end of this research, the task group discovered that a well-established but broad set of software standards did exist that covered all major areas of software engineering. Unfortunately, these standards were written to cover all business sectors, and therefore the level of detail of the standards was not sufficient to meet FDA's concerns for safe and effective software.

To address this problem, the software STG took three actions: it developed an architecture for the future of medical device software standards, it established the relationship between the recognition of software standards and the regulatory process, and it recognized the first CDRH software standard for general use.

The software STG architecture helped explain the various levels of software standards. In early STG discussions, the question was raised of what was needed for inclusion beyond the quality system regulation (QSR) and ISO 9001, which were already incorporated into the architecture. In response, the task group designed the architecture to explain various levels of standards that apply to software. These levels identify components that are not addressed in ISO 9001 and the QSR. Table I identifies the four-level model, with the associated components for each level.

The second action taken by the task group was to establish the relationship between the recognition of software standards and the regulatory process. It was important for the STG to show the link between recognized software standards and the premarket review process. This task was accomplished by clearly linking the requirements from the FDA premarket guidance for software-containing devices directly to the recognition statement. The software STG then recommended which items of documentation can be omitted from a submission when the manufacturer conforms to a standard.

Each recognized standard contains a chart similar to Table II, which is for ISO/IEC 12207. The task group divided the software standard recognition process into three risk categories to match the level of concerns contained in the premarket submission guidance. The left column of Table II contains a list of documentation taken directly from the submission guidance. There are two permissible entries for each document based on level of concern—Don't submit and See guidance. The phrase Don't Submit indicates to the manufacturer that the document is exempt from submission requirements provided the manufacturer declares conformance to the recognized standard. When the words See guidance appear, the manufacturer is directed to submit the documentation required by the premarket submission guidance.

The third action taken by the software STG was to recognize ISO/IEC 12207. This recognition was based on two primary factors. First, ISO/IEC 12207 is an established software engineering standard that was developed using a consensus international process. The 12207 standard is a product of ISO/IEC joint technical committee 1 (JTC1), subcommittee 7 (SC7). JTC1 was established in 1987 by agreement between ISO and IEC; it covers information technology. SC7 represents software engineering. The addition of this international flavor supports the ultimate goal of global harmonization. The second factor involved in the selection was the need to firmly establish a framework with well-defined terms for the engineering of medical device software. ISO/IEC 12207 set the framework to support future work on high-risk medical device software standards.

AAMI SOFTWARE COMMITTEE 1998–2001

After the passage of FDAMA and the review work of the software STG, the AAMI software committee focused on developing a medical device software standard that was acceptable to FDA and that recognized the significant investment made by many companies that had established their own proprietary software development processes. The committee determined that such a medical device standard should meet the following objectives:

The committee decided that a standard based on the framework established in ISO/IEC 12207 ("Information Technology—Software Life Cycle Processes") would best meet these goals. ISO/IEC 12207 is a well-accepted international standard that uses software engineering language and answers several FDA concerns.

But the AAMI committee felt that ISO/IEC 12207 had some flaws. It included requirements for system and management activities that went beyond the software focus requirement of the AAMI committee. Furthermore, the standard did not account for previously existing software; manufacturers with acceptable software development processes already in place would have to make major changes to their existing processes in order to claim conformance with 12207. The AAMI committee established a software executive committee to create a draft standard that was based on ISO/IEC 12207 but that eliminated the identified problems. This was the beginning of "AAMI SW68 Medical Device Software—Software Life Cycle Processes."

The executive committee met several times during 1998. It created a draft standard that maintained the framework and terminology of ISO/IEC 12207 but removed the objectionable sections and added new requirements. The most significant additions were the requirements for software that can contribute to safety hazards and the requirements for off-the-shelf software used in medical devices. This first draft was circulated for review in January of 1999 and received comments from more than 20 companies. The committee then revised the draft standard based on these comments during 1999.

One important concept that was clarified during revision was the relationship between risk management—as it was described in ISO/IEC 14971—and software engineering. It became clear that good software engineering alone was not sufficient for creating safe medical devices, but that some minimum software engineering information was necessary to make good risk management decisions. At this point, the committee decided that it was important to incorporate the framework and terminology of 14971 into the AAMI standard. The implementation of 14971 then became a requirement for conformance to the AAMI standard.

A second draft of the standard, including this clarification, was submitted to the full AAMI software committee for approval at the end of 1999. The committee approved the draft, but also provided additional comments. Once these comments were resolved, the draft standard was circulated for public review and comment in the summer of 2000. After some final editing, it was approved by the AAMI standards board in March of 2001. After more than three years of work, a software standard developed specifically for medical device software was born.

Table II. Documentation submission chart from ISO/IEC 12207, "Information Technology—Software Life Cycle Processes."

THE EVOLUTION FROM AUDITS TO ASSESSMENT

By 1999, the implementation of FDAMA and the recognition of software standards eliminated the need for AdvaMed's software quality audit process. But a new, similar need appeared. If manufacturers wanted to claim compliance with a software standard that had been recognized by FDA, how could they be sure that their individual development processes actually did comply and that their understanding of compliance was the same as the agency's? The AdvaMed SQA task force, with advice from FDA software experts, took on a new task—to develop an assessment tool that manufacturers could use to measure their compliance with the recognized software standard. The first public draft of this document is expected in the summer of 2001.

The Approved AAMI Standard. ANSI/AAMI SW68 follows the structure of ISO/IEC 12207 in that it recognizes primary and supporting processes that consist of task-based activities. SW68 identifies two primary processes—development and maintenance—and five supporting processes: software hazard management, documentation, configuration management, verification, and problem resolution. While SW68 does not specify a particular life cycle model, it does require developers to identify their life cycle model and specify when the activities and tasks of the standard are performed. SW68 also requires the developer to have a quality system that meets the requirements of ISO 13485 or FDA's QSR.

A key element of SW68 requires some of its stipulations to be met even if the software does not contribute to risk or is not used to control risk in a medical device. This is the minimum level of software engineering necessary to establish confidence in the management of device risk. SW68 requires the developer to meet the requirements of ISO/IEC 14971 ("Risk Management for Medical Devices"), and if the risk management identifies software contributions to hazardous situations, or if the software is used to control safety risks, additional software engineering requirements are specified. This two-level approach demonstrates that adequate software engineering practices are in place while at the same time allowing the level of effort to be commensurate with the risk inherent in the software. The processes and activities included in SW68 are shown in Figure 2.

SW68 Conformance Assessment. The AdvaMed SQA task force has nearly completed an assessment process and spreadsheet tool that manufacturers can use to determine if their software development process meets the requirements of SW68.

AAMI Standard Internationalization. The AAMI software committee believes that an international software life cycle process standard would be a step toward the global harmonization of medical device software regulation. The committee has proposed the creation of a new international software standard using AAMI SW68 as the starting point. A "new work item proposal" on the subject is currently being reviewed by the U.S. technical advisory groups for IEC TC 62 and ISO TC 210.

This new work proposal, which will soon be submitted to ISO and IEC, envisions a joint ISO/IEC standard on medical device software life cycle processes. The standard will most likely be developed by a joint working group composed of members of IEC 62A and ISO 210—the same committees that collaborated on the recently approved risk management standard, ISO/IEC 14971. If ISO and IEC accept the proposal, work on the new medical device software international standard will likely start in late 2001, scheduled for completion in 2004.

THE FUTURE OF SOFTWARE STANDARDS

The next step in this process is the recognition by CDRH through its software STG of AAMI SW68. After the American National Standards Institute (ANSI) approves AAMI SW68, the software committee will present it to the CDRH software STG for review and recognition. It is expected that AAMI SW68 will appear as a recognized software standard in FDA's next standards update.

The goal of the AAMI software committee was to produce a standard that would be sufficient for 50–80% of medical device software; the committee believes that this goal can be met by SW 68 and its coverage of low- and moderate-risk software. The next step in the process, then, is to identify or create documents that will aid in improving the premarket review of all remaining software.

The AAMI software committee has approved two activities for this purpose. The first will be the creation of a technical information report (TIR) on techniques for the management of software system safety in medical devices. This document will provide guidance for applying ISO/IEC 14971 and give specific examples of good techniques for controlling software factors that contribute to overall device risk. A task group began meeting in November of 2000 to draft this report.

The second new activity is to create a software verification and validation standard for medical device software. This standard will include the software verification and validation activities required for medical devices in which the failure of software could lead to a serious injury or death—software identified by FDA as "major-level-of-concern" software. Such a standard would build on the consensus achieved by AAMI SW68 to provide additional requirements for software verification and validation for the most critical medical device software. The goal of this activity is to produce a new specific process standard (level 3 in the software STG model) that FDA could recognize as sufficient for medical device software at all levels of concern. A task group is currently being formed to draft this standard as well.

CONCLUSION

Though there is more to be done, a significant amount of work has been completed to establish appropriate software standardization for low- and moderate-risk software devices. The architecture increases the understanding of how all the pieces fit together in the regulatory model. And AAMI SW68 establishes the software engineering framework needed to address the design, development, maintenance, and risk management of medical device software.

The next step, which will occur sometime in the near future, is for the AAMI committee to present SW68 to the CDRH software STG for evaluation and recognition. Once SW68 is recognized, manufacturers will have two tools available to reduce the premarket burden for low- and moderate-risk software: the manufacturer can apply and conform to AAMI SW68 and then validate conformance using the AdvaMed conformance assessment tool.

A declaration of conformance to AAMI SW68 will alleviate much of the required documentation in the premarket review process. And the use of the conformance assessment tool will provide assurance to senior management that the manufacturer's conformance declaration is valid. In addition, the use of the software engineering framework of AAMI SW68 will provide a solid foundation for the design control of software, and will complement the design control requirements of the QSR. The net result of this will be better results from FDA quality systems inspections that address software. Overall, the use of AAMI SW68 by regulators and manufacturers will result in a more consistent and efficient regulatory process related to the safety and effectiveness of medical device software.

The AAMI committee plans to address the issue of high-risk medical device software. An understanding must be developed between the regulator and industry on what is required to design, develop, and maintain high-risk medical device software. AAMI has begun two efforts to address this issue: the creation of an AAMI technical information report on techniques for management of software system safety in medical devices and the creation of an AAMI standard on medical device software verification and validation. The analysis, discussion, and documentation of proven techniques for software risk management and software verification and validation will lead to a better understanding of what is actually required for the design of safe and effective high-risk medical device software.

The long-term goal of these combined efforts is to minimize the need for the premarket submission of much of the software documentation. While there are some naysayers, the AAMI software committee remains optimistic; even if software premarket submissions are reduced in size by only 50%, industry will have come a long way in establishing the least-burdensome method of software regulation.

Sherman Eagles is a technical fellow at Medtronic Inc. (Minneapolis). He is currently working on software quality in the reliability assurance department of the company's cardiac rhythm management division. Eagles is also a co-chair of AAMI's medical device software committee, and a member of the medical device software task force of AdvaMed. John Murray is the team leader for software and intelligent medical devices with FDA (Rockville, MD). He is the primary advisor to the chief of the medical electronics branch in the areas of software engineering, software safety, and software standards. Murray is also a chairperson for FDA's software standards technical group and serves as a co-chair of the AAMI software committee.

Copyright ©2001 Medical Device & Diagnostic Industry