DESIGN : Requirements Management in Medical Device Development

Medical Device & Diagnostic Industry Magazine | MDDI Article Index

Originally published March 1996

Alan Davis and Dean Leffingwell

To managers, developers, quality assurance personnel, and others on the front line of the rapidly evolving medical device industry, the complexity of device technology seems to be growing almost exponentially. Today, medical instruments may make use of specialty analyzers to evaluate the action of chemical reagents, biological reagents, or both; they may incorporate complex electromechanical and robotic apparatus; or they may include electronic subsystems as sophisticated as any modern-day computer. Running such systems requires thousands of lines of software so complex that they would have been unthinkable just five years ago.

At the heart of this complexity, however, lies a fundamental question: what are the real requirements for such a system? After all, if the intended behavior of a system is not fully understood, how can its labeling claims of efficacy and safety be assured? How can the system be designed? How can the system's purpose be determined, or its quality measured? And how can a company predict what level of effort will be required to develop and produce the system?

Part management process and part engineering discipline, requirements management is a technique that can be used effectively to manage this increasing complexity.

REQUIREMENTS MANAGEMENT

Design and development are exacting endeavors. Mechanical tolerances are specified to within a few thousandths of an inch, electrical timing is specified in nanoseconds, and software is specific to every bit in applications containing thousands of lines of code. Despite this precision, many projects suffer a nagging uncertainty: what exactly is this thing supposed to do?

Requirements management is defined as a systematic approach to eliciting, organizing, documenting, and managing both the initial and the changing requirements of a system. A primary result of this effort is the development of one or more requirements specifications that define and document the complete external behavior of the system to be built.

A well-implemented program of requirements management creates a central information repository, which includes requirements, attributes of requirements, requirements-related status information, and other management information related to a company's environment. This repository can be not only a key link in a project's success, but a valuable corporate asset as well. Requirements and their associated attributes can evolve, be adapted, and be reused for subsequent development projects, lowering their cost. This repository not only defines the specifics of what the system will do, it assists management in determining relative priorities of system development activities and assessing the status of the program.

Some project engineers consider requirements management to be an unnecessary exercise.They would prefer to proceed directly to implementation. However, our experience suggests that effective requirements management can offer a variety of benefits that would otherwise be missing from the product development process. These include the following:

*Better definition and management of labeling claims. An up-front investment in requirements management can provide a stronger basis for the claims of efficacy and safety that the manufacturer will ultimately make for the device.

*Better control of the development project. Requirements creep and inadequate knowledge of the intended behavior of the system are commonplace in out-of-control projects. Requirements management can provide the development team with a clearer understanding of what is to be delivered, as well as when and why.

*Improved quality and end-user satisfaction. The fundamental measure of quality is, Does the device do what it is supposed to do? Higher quality can result only when end-users, developers, and test personnel have a common understanding of what must be built and tested.

*Reduced project costs and delays. Research shows that requirements errors are the most pervasive and most expensive errors to fix. Reducing the number of these errors early in the development cycle lowers the total number of errors and cuts project costs and the time to market.

*Improved team communications. Requirements management encourages the involvement of customers, so that the device meets their needs. A central repository provides a common knowledge base of project needs and commitments for the user community, management, analysts, developers, and test personnel.

*Easier compliance with regulations. FDA has a keen understanding of the requirements management problem. The most recent revision of the good manufacturing practices (GMP) regulation, with its proposed incorporation of design controls, provides the agency with a solid basis for auditing the requirements management process.

THE HIGH COST OF REQUIREMENTS ERRORS

There is strong evidence that effective requirements management leads to overall project cost savings. Requirements errors that remain undetected until test or deployment, for example, typically cost well over 10 times more to repair than other errors. Requirements errors also typically constitute over 40% of total errors in a software project. Small reductions in the number of such errors can save money by avoiding rework and scheduling delays.

There are few published data that quantify the cost of requirements errors in medical equipment, but the software industry has been accumulating evidence of such cost for some time. These data should be directly applicable to medical device software development, which is one of the most labor-intensive portions of device development.

Studies performed at GTE, TRW, and IBM measured the costs of errors occurring at various phases of the project life cycle.¹ Although these studies were conducted independently of one another, they all reached roughly the same conclusion: Detecting and repairing an error at the requirements stage costs 5 to 10 times less than doing so at the coding stage, and 20 times less than doing so at the maintenance stage (see Table I). The later in the life cycle an error is discovered, the more likely it is that repair expenses will include both the cost to correct the offending error and the cost to correct additional investments that have been made in the error in later phases of product development. Typical investments include the cost to redesign and replace the code, rewrite documentation, and rework or replace software in the field.

In a study performed at Raytheon, it was reported that, on average, approximately 40% of the total project budget was spent on rework.² Other studies confirm that for the majority of companies, rework amounts to 30­40% of total project costs. Because of the frequency, and the multiplier effects, of requirements errors, finding and fixing them consumes 70­85% of the total rework costs on an average project. Organizations like Raytheon have demonstrated that refining their development processes can lower their rework costs by up to 60%.³ While not all companies will experience such a dramatic improvement, even a small reduction in the number of requirements errors can generate significant savings.

But as high as they are, these estimates of the hard costs of requirements errors tell only part of the story. Intangible costs include features that could have been delivered had the project's resources not been devoted to rework, the increased potential for adverse incidents, and lost market share, along with the associated loss of revenues and profits. The sum of these costs demonstrates that companies cannot afford to ignore the benefits of improved requirements management.

THE VIEW FROM FDA

Under existing regulations, FDA's pre-production quality assurance guidelines provide the clearest insights into its views on requirements management.⁴ The document states, "Prior to the design activity, the design concept should be clearly expressed in terms of its characteristics....Once these characteristics are agreed upon ...they should be translated into written design specifications." Based on these guidelines, inspectors and reviewers today expect requirements documents not only to exist, but also to be accurate, complete, and unambiguous. When it is published later this year, FDA's revised GMP regulation will have a significant impact on medical equipment manufacturers because it will bring design-related activities within the scope of the regulation. These activities, referred to collectively as design control, are processes and procedures for documenting the design activities used to implement the labeling claims of the device. Once the revised regulation is in place, it seems likely that FDA will take a very active role in reviewing and evaluating a company's requirements management activities.

ELICITING REQUIREMENTS

The process of gathering and eliciting requirements for a new device, system, or application is one of the most exciting and important phases of device development. The goal of this phase is to gather all the requirements of the device. To be successful, these requirements should reflect a consensus not only within the company, but with prospective customers and users as well.

There are a variety of potential sources that can be used effectively in this process, including a company's existing documentation for a similar or predecessor device, 510(k) submissions from other companies or competitors marketing similar devices, experts from inside or outside the company who understand the application area, a company's marketing and sales department, and prospective users. Once these sources have been identified, the goal is to develop a systematic process for collecting data from them, synthesizing and collating the data, and then rationalizing and reducing the data. This last step is needed to remove conflicting or redundant requirements, and eliminate or annotate those requirements that are impractical based on current technology, cost constraints, or market factors.

To elicit requirements, device manufacturers can employ several techniques, either alone or in combination:

*Well-structured interviews can be highly effective in collecting requirements from experts, the sales and marketing department, and prospective users.

*Brainstorming is a structured yet creative technique for eliciting ideas, capturing them, and then subjecting them to objective criteria for evaluation.

*The domain where the device will be used can be a rich source of system requirements. Placing engineers and designers in this environment even for a day is a quick way to educate them about potential problems and issues.

*Structured workshops, managed by trained facilitators, are also an effective way to elicit input.

DEFINING REQUIREMENTS

Typically, requirements start as an abstraction, along the lines of, "We feel there is a patient need for a low-cost ambulatory infusion pump." As the process continues, the requirements become more specific, diverge, recombine in often new ways, and eventually emerge as a set of detailed requirements such as, "The pump will weigh less than 12 ounces," or "The pump will be able to operate for up to 24 hours on a single 9-V battery." As they become more detailed, requirements also become less ambiguous.

When the final requirements have been arrived at, the document that contains them is called a requirements specification. This specification is the basis for design, explaining to designers and testers what the system should do. It also must be communicated to and agreed upon by all relevant parties.

Good requirements specifications have the following attributes in common:⁵

*Clarity. If a requirement has multiple interpretations, the resulting product will probably not satisfy needs of users.

*Completeness. It may be impossible to know all of a system's future requirements, but all of the known ones should be specified.

*Consistency. A system that satisfies all requirements cannot be built if two requirements conflict.

*Traceability. The source of each requirement should be identified. The requirement may represent a refinement of another, more abstract requirement, or it may result from input from a meeting with a target user.

As long as requirements address external behaviors--as viewed by users or by other interfacing systems--then they are still requirements, regardless of their level of detail. A requirement becomes design information, however, once it attempts to specify the existence of particular subcomponents or their algorithms.

Most requirements specifications include auxiliary information that is not, by definition, a requirement. Such information may include introductory text, summary statements, tables, and glossaries. Actual requirements should therefore be highlighted by a unique font or some other identifier.

The Role of Hazard Analysis. Hazard analysis plays a unique role in ensuring the safety and efficacy of a medical device. As a technique, it takes into account design- related elements that can mitigate or eliminate hazards to the patient, as well as design elements that create hazards that must be mitigated elsewhere. When reduced to product documentation, the understanding that hazard analysis conveys is qualitatively different from that of a pure requirements document. Because requirements documents focus on what the product should do, rather than on how it should be done, they should not contain design-related information. Hazard analysis, on the other hand, is intended to apply knowledge of good design practices to the mitigation of potential hazards. In doing so, it necessarily creates new system requirements, and the documents that contain these requirements should therefore be considered to be at the highest level of the document hierarchy (see Figure 1). In this way, new requirements developed through hazard analysis can be traced into implementation and testing in the same way as other product requirements.

ORGANIZING DOCUMENTS

After feedback on requirements has been elicited, the next step is to organize the requirements into a document hierarchy, which will support the development and maintenance phases that follow. No hard-and-fast rules define the number and types of documents that will be necessary to manage the evolution of the product; it will depend in part on the level of sophistication of the project team, the number of subsystems, how critical the device is, the total number of requirements, and other factors.⁶

In many cases, a document hierarchy with no more than three levels will suffice. In the hierarchy shown in Figure 1, the document tree has been divided into three major branches according to the architectural decomposition of the product: software, hardware, and reagents and consumables. This tripartite division provides working groups in each area with specific requirements documents around which they can focus their development efforts. In this hierarchy, the three branches stem from the product requirements document, which is the source of all system requirements. This is typically the highest-level document that defines the claims for the device, and the document upon which the 510(k) or premarket approval is based. At the lowest level of the hierarchy, the documents are divided further into implementation documents, and individual-test documents and specific test protocols to be applied at the subsystem level.

THE SOFTWARE REQUIREMENTS SPECIFICATION

One of the more important requirements documents is the software requirements specification, which is a primary concern of the software developer. This document defines not only the complete external behaviors of the software system to be built, but also its nonbehavioral requirements. A number of standards and guidelines are available to help manufacturers create a requirements specification.⁷ Standards are by no means a cure-all, but they do provide checklists of things to address and can help to shorten the learning curve for new requirements writers and other project team members. The standard chosen should ensure accuracy, encourage consistency, and promote a short learning curve. Then, based on usage, the standard can be modified as necessary and made company-specific.

Documentation writers should attach labels to those portions of text, graphics, or sound objects that must be tested after implementation. Ideally, the actual requirements will be left in situ rather than stored in multiple places. That way, they can be edited and maintained in the project documents even after they have been selected as individual requirements. This makes it easier to update project documentation as requirements change.

ASSIGNING ATTRIBUTES

All system requirements possess attributes. These provide a rich source of management information to help plan, communicate, and record a project's activities through its life cycle. Because the needs of each project are unique, its attributes should be selected accordingly. Specific requirements, for example, will have different implications for efficacy and patient safety. For an infusion pump, the requirement that "overinfusion shall be prevented through redundant safety systems" is more critical than one that states, "The case shall be painted gray."

Companies must therefore set priorities and make trade-offs, particularly in response to scheduling and budgetary constraints. By talking with customers and management, developers can improve their decision making by ranking requirements in order of importance. Status fields should be created to record key decisions and progress. When defining the project baseline, the terms proposed, approved, and incorporated are useful. As a project moves into development, in progress, implemented, and validated can be used to describe important milestones.

Another important attribute is ownership; that is, who is responsible for ensuring that a particular requirement is satisfied? It is also important to create a field for rationale; that is, why does a particular requirement exist? This field can either explain the rationale directly or refer to an explanation somewhere else. The reference, for example, could be to a page and line number of a product requirements specification or to a videotape of an important customer interview. And the dates on which the requirement was created or changed should also be recorded.

Many relationships can exist among requirements. Attribute fields, for example, can record:

*A more abstract requirement from which the present requirement emanated.

*A more detailed requirement that emanated from the present requirement.

*A requirement of which the present requirement is a subset.

*Requirements that are subsets of the present requirement.

In addition to the ones listed above, other common attributes include difficulty, stability, risk, security, version, and functional area. Regardless of the method used to track them, attributes should be easily customizable to adapt to the unique needs of each development team and each application. Manufacturers should pay particular attention to maintaining links between requirements and all products that result from them. These links help to determine the effect of any changes, as well as their development status, which can be recorded as an attribute of those downstream entities.

REQUIREMENTS TRACEABILITY

Requirements begin their existence when first elicited from customers or users. Once they have been captured, links must be maintained backward to their more abstract predecessors and forward to their more detailed successor requirements. Maintaining the ability to trace the path of these changes is a prerequisite to quality assurance and sound requirements management. Use of requirements traceability enables product development teams to attain a level of project control, quality assurance, and product safety that is difficult to achieve by any other means.

Traceability should be a key tool in project management, if only because ensuring full requirements test coverage is virtually impossible without some form of requirements traceability. FDA's "Reviewer Guidance for Computer Controlled Medical Devices Undergoing 510(k) Review" states that "testing requirements should be traceable to the system/software requirements and design."⁸ The proposed revised GMP regulation goes further, stating that "manufacturers must establish a documented program to ensure that design requirements are properly established, verified, and translated into design specifications, and that the design released to production meets the approved design specifications."⁹

In its simplest terms, requirements tracing demonstrates that the system does what it was supposed to do. The key benefits of this process include:

*Verification that all user needs have been implemented and adequately tested.

*Verification that there are no system behaviors that cannot be traced to a user requirement.

*Improved understanding of the impact of changing requirements.

A document hierarchy in the form of a diagram can be a useful tool for managing requirements traceability. Such a diagram graphically displays the interrelationships that exist among a project's documents, and therefore also among the requirements expressed in those documents. The chart shown in Figure 1, for example, indicates that there is a direct relationship between the product requirements document (the source of all product requirements) and the software requirements specification. Hence, all software-related product requirements should be satisfied by one or more software requirements, and, conversely, any software requirement may help to satisfy one or more product requirements. In this example, any product requirement that has no corresponding software, hardware, or reagents and consumables requirements will not be satisfied. By the same token, a software or other requirement that has no related product requirement is extraneous and should be eliminated. The document hierarchy enables project team members to visualize and check the completeness of requirements contained in all levels of a product's documentation.

In addition to using a document hierarchy, manufacturers should employ some system that will enable them to continuously identify relationships among items within the hierarchy. This can be done by embedding unique identifiers and electronic links within the document, or by using a separate spreadsheet or database that manages the links outside the document. The latest generation of requirements management tools automatically maintains traceability links.

CHANGE MANAGEMENT

Requirements traceability provides a methodical and controlled process for managing changes that inevitably occur as a device is developed and deployed to the field. Without traceability, every change would require project team members to review all documents on an ad hoc basis in order to determine what other elements of the project, if any, require updating. Because such a process would make it difficult to establish whether all affected components have been identified over time, changes to the system would tend to decrease its reliability and safety.

With traceability, change can be managed in an orderly fashion by following the relationships of product requirements and specifications through the document hierarchy. When a user requires changes, developers can quickly identify elements that may need to be altered, testers can pinpoint test protocols that may need to be revised, and managers can better determine the potential costs and difficulty of implementing the changes.

REQUIREMENTS REPORTING

A requirements repository gives managers a powerful tool for tracking and reporting project status. Critical milestones can be more easily identified, scheduling problems better quantified, and priorities and ownership kept visible. Querying the repository can help managers quickly retrieve key information such as how many requirements have been established for their device, how many of these might affect patient safety, whether all hazards have been identified and mitigated, the current status and schedule for incorporating requirements into the prototype, and the estimated cost of proposed changes.

High-level reports drawn from the requirements repository can also aid management reviews of product features. Requirements can be prioritized by customer need, by user safety considerations, or by how difficult or costly they will be. Such specialized reports can help managers allocate resources by focusing their attention on key project issues.

CONCLUSION

Many device manufacturers carry the scars from projects that failed to meet expectations. It is common for projects to over-shoot their schedules by half, deliver less than originally promised, or be canceled before release. To keep pace with rising complexity and increased user demands, companies must become more sophisticated in the ways they develop, test, and manage their projects. Improved requirements management offers a first step in this direction.

A requirements management repository that includes a product's requirements, specifications, and attributes--and the links among them--can help manufacturers determine precisely what the product is supposed to do. It can be used to manage and control the device development process, improve team communications, improve quality, and, ultimately, ensure that the final product fully satisfies the needs of the customer.

REFERENCES

1.Davis A, Software Requirements, Objects, Functions, and States, Englewood Cliffs, NJ, Prentice Hall, 1993.

2.Dion R, "Process Improvement and the Corporate Balance Sheet," IEEE Software, July, pp 28­35, 1993.

3.Paulk M, Curtis B, et al., "Capability Maturity Model for Software, Version 1.1," SEI-93-TR-025, Pittsburgh, Software Engineering Institute, 1993.

4."Preproduction Quality Assurance Planning: Recommendations for Medical Device Manufacturers," HHS Publication FDA 90-4236, Rockville, MD, FDA, Center for Devices and Radiological Health (CDRH), 1989.

5.Davis A, et al., "Identifying and Measuring Quality in Software Requirements Specifications," presented to the IEEE International Software Metrics Symposium, Baltimore, May 1993.

6.Wood B, and Pelnik T, "Managing Development Documentation," Med Dev Diag Indust, 17(5):107­114, 1995.

7."IEEE Recommended Practice for Software Requirements Specifications," IEEE Std 830-1993, Piscataway, NJ, Institute of Electrical and Electronics Engineers, 1993.

8."Reviewer Guidance for Computer Controlled Medical Devices Undergoing 510(k) Review," Rockville, MD, FDA, CDRH, 1989.

9."Working Draft of the Current Good Manufacturing Practice (GMP) Final Rule," Rockville. MD, FDA, CDRH, Office of Compliance, July 1995.

Alan Davis is a professor of computer science and El Pomar chair of software engineering at the University of Colorado, Colorado Springs. Dean Leffingwell is president and CEO of Requisite, Inc. (Boulder, CO), a company specializing in requirements management software for medical devices.