PRODUCT DEVELOPMENT INSIGHT

Many medical device companies develop software using a traditional waterfall methodology in which each step is taken in sequence: requirements, design, implementation, verification, and validation (see Figure 1). At each step, documents are written, reviewed, revised, and approved.

Often, this style of software development is tied to the organization's phased, gated product life cycle approach, driven by management approval and budget allocation decision points. Although this has been a widely accepted approach for years, many organizations are now seeking approaches that yield demonstrable value more quickly and that are more adaptable to changing business needs.

These approaches exist. Commercial software development best practices have embraced agile software development methods. Commercial software development clearly demonstrated how agile development improves pace and efficiency while providing greater flexibility. It is possible to adapt such approaches to medical device software development and still maintain the quality and design controls demanded by various regulatory bodies, including FDA.

This article describes how medical device manufacturers can increase the efficiency and pace of software development by adopting—or adapting—agile development methods while preserving quality and meeting design control demands.

Agile Development Principles

Agile development has been loosely interpreted by some as any attempt at an iterative development approach. True agile development methodologies all exhibit at least some of the “Principles behind the Agile Manifesto.”¹ In summary, agile principles advocate:

Most medical device manufacturers heartily endorse the first three principles and strongly believe that such dialogue and commitment are hallmarks of their own history and process.

The remaining principles, however, represent a dramatic shift in philosophy that challenges medical device manufacturers' established quality management procedures and budget approval process. For example, the frequent delivery of working software requires organizations to execute review and approval cycles more quickly and more frequently. This adjustment is difficult for many organizations that are used to waterfall-style review cycles.

These agile principles can be so drastically different compared with the organizational thinking and behavior of some medical device manufacturers that, without proper preparation, one of the following negative reactions can occur: rejection, force fitting, or abandonment.

Rejection. The organization rejects these concepts outright. Some medical development organizations have spent decades refining their gated waterfall processes, document templates, and step-by-step procedures. They are unwilling to abandon these seemingly successful programs.

Force Fit. The organization tries to embrace agile principles by force fitting them into its existing process. These organizations adopt the agile tag in name only and iterate solely within the confines of their serial, sequential waterfall phases. Dividing requirements specifications into several iterations does not embrace the agile principle of producing working software in each iteration.

Abandonment. The organization initially embraces the concepts but abandons them within months because the misplaced expectation of faster development is not realized. The unprepared organization chafes under new direction, constantly comparing the new system to the previous waterfall process to which it is accustomed.

Agile development requires careful planning and preparation, and a willingness to adjust organizational behavior. Many medical device companies are successful in the marketplace and enjoy a reputation for quality and long-lasting products. For many such companies, organizational change is often costly and difficult.

The use of software in medical devices is growing significantly, and previous development methods no longer suffice. The landscape is littered with failed software and product development efforts, and completed projects are often beset with cost and schedule overruns. The current competitive landscape demands that new products be developed sooner and that development efforts be flexible. The ability to successfully respond to changing business conditions and user needs during the stages of development is critical.

With waterfall processes, years may elapse between the period of defining requirements and actual implementation, compromising any rapid response to requirements. Change management controls, such as an engineering change order (ECO) approach, intend to allow requirement changes, but fail to handle change efficiently and effectively. Too often, medical device companies have modeled these controls after their hardware development processes, and they prove inappropriate for managing software. While controlling change, such an approach can unnecessarily force software development to cycle back through the entire sequential process, resulting in increased costs and schedule challenges.

This can result in more-costly products that are late to market, or products released with reduced features to satisfy a compromised schedule. Once organizations overcome concerns with the new development approach, they can begin to reap the benefits of agile development and manage the challenges it presents.

Requirements for Medical Device Developers

Agile development, while providing flexibility, also requires rigor and adherence to its process controls. Adopting an agile approach requires thought, planning, and preparation. The following points represent the major challenges for medical device software development when moving to an agile development approach.

Organizational Change. Change in any organizational structure is difficult and requires training and commitment by senior management. Members of business and engineering staff at medical device companies have worked only with the waterfall approach for their entire careers. They are often reluctant to change from what has historically been successful. Change is enabled by taking the following steps:

A Committed, Focused Team. Key personnel are often pulled from new development to address customer issues, debug a critical issue, or implement additional features. Shuffling business and technical resources to other projects weakens new development performance.

Agile development proves successful if a team of focused and dedicated resources is allowed to formulate and drive development. Avoid the temptation to leverage staff across multiple projects.

Patience with the Process. Implementing an agile program is not an instant panacea—the change in methodology does not immediately address all development ills. Establishing a fresh development process involves multiple adjustments that take time to learn and adhere. Current waterfall processes did not materialize overnight and neither will agile processes.

If your organization is using an agile methodology for the first time, it is important to consider the following:

Documentation. Agile development favors conversation over documentation. Medical device software development requires a certain amount of documentation as evidence of process compliance. For medical device agile development, the idea is to produce only the documentation that is necessary. Some guidelines include:

Strategy versus Specification. Most waterfall approaches emphasize the complete exposition of user needs, requirements, design, and tests. With agile development, it is important to establish a solid strategic foundation for the product, the architecture, and the technology without overspecifying the solution. Requirements, design, implementation, and testing all occur within each iteration.

For every product, there is a vision and a goal. End-user needs must be captured at the beginning of a project. These may change during the course of development, but it is unlikely that radical or wholesale changes will occur. Certain aspects can be imprecise at the beginning, since changes will be made during the agile approach.

Strategy differs from specification. Strategy establishes the system's organization, operation, growth, and variability principles. Specification details the ways in which these principles are applied. Once the strategy is set, the specification can be created during the process, with revisions to meet changing requirements. Both business and development sides should understand the strategy. Reinforce the principles throughout the overall development effort, especially as requirements and specifications change.

Advantages of Agile Development

There are four key advantages to implementing an agile development process: continuous quality, visible progress, improved risk management, and a successful product.

Continuous Quality. Working software is produced frequently throughout the development of the product. Verification is part of agile development and is realized through unit tests, frequent builds, iterative manual verification, and automated regression testing that results in high-quality software. System integration time is minimized due to frequent, smaller integration efforts.

Better Progress Visibility. All stakeholders—marketing, product management, executive sponsors, quality, regulatory, etc.—on both the business and development sides frequently view progress as software functions are demonstrated during each iteration. This provides tangible proof of progress and promotes early review and feedback. This also ensures that the build and release processes and mechanisms are functional, as they are exercised throughout demonstrations. Demos are installed and run in a deployment context and are specifically not run in a development environment, protecting against software functions that perform on only certain machines. Such behavior commonly plagues late-stage development in a traditional waterfall approach.

Improved Risk Management. The iterative nature of agile development also provides an easy way to consistently identify and update risks. Taking mitigating steps as minor course corrections eases the stress of concerns that could otherwise accumulate at the end of the development process.

Right Product. The increased visibility and willingness to accept requirement changes during the development process all but guarantees the construction of a successful product. The end result will meet user needs and incorporate changing business conditions during the course of the product's development.

Some agile advocates tout faster development as a key aspect of agile development, claiming that development time can be reduced by half. However, compelling evidence does not exist to support such claims for medical device software development, given the rigor of design controls that must be maintained. A more-significant contribution of agile development is a stronger product upon initial release.

As shown in Figure 2, each iteration has requirements, design, development, and testing activities. These activities are not necessarily evenly distributed: early iterations may have more requirements activity, while later iterations have more testing. At the end of each iteration, requirements, along with other changes resulting from verification and validation activities and fluctuating market needs, are considered. The strategies are updated as warranted and applied to the next iteration.

Regulatory Concerns

Medical device manufacturers must comply with regulatory guidelines. Many assume the waterfall process to be the best way of managing design artifacts, reviews, and approvals. The waterfall approach packages documents and reviews in neat bundles, but in doing so, places emphasis on the organizational quality system process compliance rather than the construction of a quality product that addresses user needs.

Contrary to popular assumption, FDA and other regulatory bodies are not partial to waterfall methodology. In fact, FDA remains neutral in this regard and states in General Principles of Software Validation; Final Guidance for Industry and FDA Staff the following:

This guidance does not recommend the use of any specific software life cycle model. Software developers should establish a software life cycle model that is appropriate for their product and organization. The software life cycle model that is selected should cover the software from its birth to its retirement. . . . A life cycle model organizes … software development activities in various ways and provides a framework for monitoring and controlling the software development project.²

An agile development approach aligns well with this guidance. The key point is to monitor and control the project. In many ways, an agile approach provides more visible, tangible evidence of accomplishment more quickly and more frequently than the traditional waterfall approach.

The framework for controlling and monitoring an agile project includes:

Conclusion

Medical device manufacturers can take advantage of commercial software development best practices to improve the pace, efficiency, and flexibility of software development. Agile methodologies can be adapted in a way that meets the rigorous quality demands for regulatory submission. There are challenges, especially for an organization comfortable with a waterfall approach, but these challenges can be met by managing organizational change to enable and empower an agile approach,

Focus on up-front planning and strategy and dedicate resources for the duration of a project. Define the documents and other evidence of regulatory compliance needed to control and monitor the process and to support regulatory submission. It is important to have patience with the process change, allowing ample time to adjust.

The advantages of an agile approach to development include improved risk management; continuous quality management through frequent testing, deployment, and integration; better and earlier visibility of demonstrable progress; and most importantly, ensuring that organizations are releasing the right product that fulfills current user and business needs.

According to the 2006 Standish Chaos report, while nearly 65% of projects don't meet requirements, or suffer budget or schedule overruns, all software development project success rates are increasing.³ Jim Johnson, chairman of the Standish Group, an organization with more than 10 years of experience in software development studies, cites one reason for this improvement: the adoption of agile development methodologies.

Tim Bosch is chief architect in the medical division of Foliage (Burlington, MA). He can be reached at [email protected] or 781/993-5500.

References

1. Ken Beck, et al., “Principles behind the Agile Manifesto,” [online] Agile Alliance, 2001 [cited 22 August 2007]; available from Internet: http://agilemanifesto.org/principles.html.

2. Food and Drug Administration, Center for Devices and Radiological Health, [online] “General Principles of Software Validation; Final Guidance for Industry and FDA Staff” (Rockville, MD: FDA, CDRH, January 2002); available from Internet: www.fda.gov/cdrh/comp/guidance/938.html.

3. David Rubinstein, “Standish Group Report: There's Less Development Chaos Today,” [online] SD Times, March 1, 2007; available from Internet: www.sdtimes.com/article/story-20070301-01.html.

Copyright ©2007 Medical Device & Diagnostic Industry