The hard work of making medical device software better begins at the top.
 |
|
Bosch
|
 |
|
Bowe
|
In terms of meeting deadlines, budgets, and functionality requirements, software development projects are more likely to fail than succeed. After showing some improvement in predictability and quality since the mid-90s, the success rate for development projects has declined in the most recent study conducted by an IT market research firm.
An April 2009 report by the research company The Standish Group International shows a woeful 32% rate of success, down from 35% in 2006.1 The results “are a low point in the last five study periods,” says Jim Crear, the company's CIO. “This year's results represent the highest failure rate in over a decade.”
The medical device industry is particularly prone to this problem. Recalls of medical devices caused by software-related issues increased from 10% in 1996 to 21% in 2006.2 One study suggests that 25% of recalls from 1995 to 2004 can be traced to software issues.3 And executive mismanagement of quality control at all organizational levels is among the top 10 reasons for FDA-483 citations.4
It is clear that chief executives of medical device manufacturers need to accept responsibility for the current state of software development at their companies. It is equally obvious that they must take steps to ensure a more predictable, high-quality outcome from their software development departments. The medical device executive need not be a software expert but must have some understanding of how to oversee the development process. Strong development steps involve properly aligning business, product, and technology strategy as well as establishing quality processes appropriate for modern software development. These steps to set up proper guidelines and direction for product development need to be taken long before the first line of software code is written.
The techniques discussed in this article will enable executives to improve practices and productivity throughout the software developmental process. Clear communication of corporate goals for new products—much more so than the feature set definition—enables executive management to exert a steady and meaningful influence over remote engineering trade-offs. This approach ensures that the product under development meets its established revenue and quality goals and avoids the stigma—and cost—associated with a recall.
‘Cozy, Insular World'
Despite the recently reported setback, advances in project management and development methodologies account for the previous minor improvements in the success of software development. Sadly, the medical device industry has been very slow to adopt these methodologies, compromising the ability of medical device companies to improve their success rates for project software development.
Tim Gee, an expert in medical device development and connectology, says the root of the problem is the nature of the device sector. “The cozy, insular world of medical devices has caused the industry to sorely lag similar industries with regards to product development methodologies and resulting embedded software quality,” Gee writes.”2
Most medical device manufacturers have extensive and detailed standard operating procedures and quality-management practices. However, the software complexity in the medical device industry has increased dramatically over the past decade. As a result, these operating processes and practices have not been updated to address the specific needs of software development, and they often do not identify timely and effective review and approval milestones with the kind of information that is easily consumed by executive management.
In order to address the growing significance of software in their products, most executives have been content to simply add controls for software into their existing hardware-based operating procedures and quality-management systems. Overall product development practices have largely remained the same for the past 20 years or more, often following a waterfall development methodology, and more recently within a formal phase-gate approach. The problem is that software development is fundamentally different from hardware development, and it requires different controls to ensure quality.
The FDA guidance document on software validation itself notes “for these and other reasons, software engineering needs an even greater level of managerial scrutiny and control than does hardware engineering.”5 Many executives are content that documented engineering processes and practices exist. And they are satisfied when software engineering staff members assert that they are following those processes and practices to a T, as represented by the checks on a checklist. In practice, however, shortcuts often undermine the intent of these documented practices, while required steps are more likely performed out of sequence.
New Software Volatility
This state of affairs has the double effect of providing little of the hoped-for project control value and none of the productivity of proper development efforts. Two steps—a formal review of requirements after development and a software code review following system testing—can spot problems early enough to resolve them before they become much more expensive to fix further down the line. These steps must be conducted in the proper sequence after the development process, however. The chief executive's role here is to ensure that the processes are performed appropriately and meaningfully, providing real value for the business and especially for the software engineering aspects of product development.
According to A Survey of Software Engineering Techniques in Medical Device Development from the Fraunhofer Institute,6 the most significant areas of challenge in software development for medical devices are requirements management and software architecture specification. Issues in either of these areas can cause large and costly problems in the later stages of product development. In modern software development, requirements management is the process of determining, documenting, analyzing, prioritizing, and approving requirements and then controlling change and communicating to relevant stakeholders. It is, or should be, a continuous process throughout a project.
One reason requirements management has become such a problem is that the inevitable volatility of new software does not fit well into a waterfall approach to product definition. It is difficult if not impossible to define and approve all software requirements before designing or coding, especially for aspects of medical devices like user interfaces and algorithms that control precise product performance. This problem is exacerbated because requirements management discipline is quite often lax throughout the project, leading to poorly documented requirements.
Lax requirements management discipline also leads to scope creep and scope reduction. The first term defines new requirements that are discovered after the initial ones have been approved. The latter refers to software-engineering trade-offs on features and functions with little executive guidance in order to meet scheduling and budgeting commitments. The sometimes arbitrary nature of scope reduction further compounds the problem.
The other primary area of trouble is poor software architecture. Poor software architecture can cause enormous pain during the product development process, bloating schedules and budgets because of forced rework to address design constraints. This often results in seriously degraded performance at product launch. Simply stated, good software architecture should present the following:
• A description of the software architecture, including major software components and their interactions.
• A common understanding of the architectural principles used during design and implementation.
• A description of the hardware and software platforms on which the system is built and deployed.
• Perhaps most important, explicit justification of how the architecture meets its nonfunctional requirements. Nonfunctional requirements include performance, reliability, safety, extensibility, supportability, and other behaviors that stakeholders expect of the system.
A CEO doesn't have to be a software engineer to understand that a product's software architecture must meet defined goals such as configurability, reliability, and scalability. Product design, which includes software architecture, must directly support the business and product goals before development begins in earnest.
A chief executive officer should not delegate this responsibility to an engineering manager or someone else lower on the organizational chart. Executive management's failure to ensure quality at all levels of the organization is one of the top reasons FDA sends observations or warning letters. Software has become an indispensable and increasingly complex part of the development of next-generation products.
To achieve the intended results in a cost-effective manner, device manufacturers must use efficient and effective software development methodologies, tools, and best practices. There is an untapped potential for optimization of your software development activities. The chief executive needs to lead the way in exploiting this potential in order to produce a safe, reliable, and innovative medical device with significantly higher efficiency than previous devices.
Common Executive Mistakes
A chief executive may directly or inadvertently contribute to the failure of a product in a number of ways. These missteps can be inaction, inconsistent emphasis of corporate goals, or a lack of consistent attention to companywide execution for achieving business and product goals. Some common mistakes are described here.
Lack of Clear Business and Product Vision. When you fail to clearly state where the business is heading and directly show how the product fits within that vision, especially when that vision changes, you're not providing the information required to guide your subordinates in making decisions. This applies in particular to organizations that already have one or more successful products, because the business and product goals change as a product moves through its life cycle. At first, you may want to be first to market, or capture a specific percentage of market share, or be viewed as best in class based on customer perception of features and support. As a product matures, you may shift focus to balance initial goals with improved customer satisfaction or next generation development. Whatever the case may be, just assuming that everyone understands or shares the same vision and its implications is a recipe for failure.
Poor Communication of Product Vision and Goals. Even when there is a clear and unambiguous product vision, another mistake is to not consistently and continually communicate it. A product road map allows the entire organization to understand the big picture and not get lost in short-term trade-offs. This message needs to be reinforced so that each person understands how his or her efforts contribute to the realization of the vision and goals. This is particularly important for each component of the organization as it encounters major decisions. Product management needs to reinforce the product goals when deciding the product's trade-offs. Engineering needs the reinforcement when refining requirements, defining architecture, or choosing technology. And maintenance needs reinforcement when prioritizing its work and the level or degree of support to provide.
One medical device manufacturer determined that the business would shift to a product-line approach in order to reduce the number and variety of parts and to promote greater reuse of software components. In doing so, the business would move from a focus on lowest possible cost of goods for each product to a model that allowed a somewhat higher cost of goods while significantly reducing support costs. The problem that arose was that executive management did not clearly communicate this shift to all levels of the organization. As a result, procurement and engineering staff continued to make new product decisions based on lowest possible cost of goods as they always had, inadvertently sabotaging planned reductions in support costs.
No Defined Product Technology Goals. Failure to clearly articulate specific technology constraints or goals will result in decisions being made based on factors other than market needs or a product's intended life cycle. Such decisions often result in expensive downstream rework; they can also jeopardize a product's launch or its long-term viability. For example, because technology goals were not initially defined, engineers at one device company assumed its new endoscopy product should be portable so that it could run on multiple computer platforms, and key software and technology decisions were made accordingly. They had to start from scratch when they realized that the prototype's usability and consistency with the existing product's platform was essential, and that this was known by product management but not articulated at the outset of the project.
Overemphasis on Customer-Specific Requirements. Sometimes executives can overemphasize the needs of an important (or loud) customer to the detriment of the product. Creating situations where you as an executive demand specific features or solutions can warp the product to the point where its long-term growth or support is unsustainable. This puts the developmental group in the untenable position of deciding when and how to support potentially contradictory requests from you and from product management.
Overemphasis on Time-to-Market Goals versus Quality. Most medical devices must have sufficient quality to ensure patient, operator, and caregiver safety. Because most medical device software development follows a waterfall approach, verification often only happens at the end of the development cycle. Overemphasis on time-to-market goals can end up pressuring verification to a point where quality is sacrificed in order push product out the door. Likewise, such overemphasis can tempt others working on upstream software development activities, like design and coding, to take shortcuts that adversely affect quality in the interest of hitting schedules.
No Processes to Support and Reinforce Quality. The lack of support and reinforcement for quality activities throughout the product development process will result in shoddy work. It is not okay to establish a quality slogan upfront without backing it up with appropriate resources and processes to ensure the quality throughout development, thereby avoiding surprises at the end. This mistake is best remedied by investing in a true internal audit regimen that is influential and respected by all within the organization.
Maps, Metrics for the Road Ahead
While simply avoiding mistakes is one approach, there are five steps a chief executive can take before the software project begins in order to improve the chances that the next-generation medical device is a success.
1. Create or Update Your Business Vision, Directives, and Initiatives. The business vision must be stated in clear, unambiguous, noncontradictory terms. “Maintain current customer satisfaction ratings as measured by industry rankings while reducing support costs by 20%” is clear. This statement helps software development engineers by making it clear that these two expectations must be factored into any decisions or trade-offs the engineering staff makes.
2. Create Product and Technology Road Maps. Your organization needs to understand not only the current set of goals; it also must have an appreciation for the road ahead. The intended long-term direction can influence short-term decisions. Clearly establish needs and priorities—don't defer these to low-level managers or developers. Provide a rationale for these decisions in order to demonstrate that there is a well-thought-out sense of direction and urgency.
3. Establish a Communication Plan. Continually broadcast your business goals, product road map, and technology road map. Stating them at your annual all-hands meeting is a start, but the sense of urgency and direction will fade without constant reinforcement. Periodically (each month, each quarter) show how actions and decisions align with the goals and road maps. Use reviews and progress status reports as opportunities to remind staff of goals. Situations will arise that dictate a change of plans, and you should communicate these “course corrections” and their impacts so that lower level decisions and directions can be adjusted accordingly.
4. Establish a Formal Mechanism to Review and Report on Alignment. During the course of product development, there are key points that provide the opportunity to ensure that requirements, architecture, design, trade-offs, and risk management activities are all aligned with each other and with the prevailing business and product goals and road maps. Establish explicit milestones for this alignment review in a way that allows time and opportunity for corrective actions. Planning for this step will require your focus and direction, but the payoff will be the improved ability to ensure key decisions are aligned with your goals.
5. Align Performance Metrics with Stated Goals. The surest way to derail corporate goals is to have performance metrics that are contrary to the stated goals. For example, if a corporate goal is to favor reduced support cost over the cost of goods, don't measure product managers' performance based on how low they can drive the costs of goods. Make sure metrics used for department, team, and personal performance reviews are consistent with each other and with the goals stated above.
The Executive's Ultimate Responsibility
The medical device company chief executive is ultimately responsible for ensuring quality throughout the organization, including the software aspect of product development. To this end, the executive must take steps to ensure a predictable, high-quality outcome from the medical device's software development process. The reward for moving forward will be better quality products developed at a lower cost.
The steps detailed here are designed to foster an ability to achieve a proper alignment of business, product, and technology strategy and establish quality processes appropriate for modern software development. They should help to ensure that both product and business meet revenue and quality goals for long-term growth and success.
References
1. CHAOS Summary 2009, [online] (Boston: The Standish Group, April 23, 2009 [cited 19 July 2009]); available from Internet: www1.standishgroup.com/newsroom/chaos_2009.php [3].
2. Tim Gee, “Technical Session 2: High Confidence Medical Devices, Software, and Systems,” [online], (Beaverton, OR: Medical Connectivity, June 2007 [cited 6 July 2007]; available from Internet: www.medicalconnectivity.com/2007/07/06/technical-session-2-high-confidence-medical-devices-software-and-systems/ [4].
3. N Pallikarakis, “e-Health and Patient Safety” [online], Future Health; available from Internet: www.futurehealth.fi/inc/downl_file.asp?f=106284&m=25100&a=2358&c=534&x=0 [5].
4. “Ten Most Common Reasons for FDA-483 Observations and Warning Letter Citations” [online] (white paper [cited 23 July 2009]), (Salt Lake City: Master Control, 2007); available from Internet: www.mastercontrol.com/white_papers/fda-483/white_paper_ten_most_common_reasons_fda_483s.html [6].
5. General Principles of Software Validation—Final Guidance for Industry and FDA Staff (Rockville, MD: FDA, January 11, 2002; available from Internet: www.fda.com/gov/cdrh/com/guidance/923.html [7], accessed July 19, 2009.
6. RL Feldmann et al., “A Survey of Software Engineering Techniques in Medical Device Development” [online] (paper presented at High-Confidence Medical Devices, Software, and Systems and Medical Device Plug-and-Play Interoperability Joint Workshop, Boston, June 25–27, 2007 [cited 19 July 2009]); available from Internet: http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=438163 [8].
Tim Bosch is the chief architect at Foliage. He directs consulting and development efforts with a focus on medical device and medical information system products. Timothy Bowe is CEO and a cofounder of Foliage (Burlington, MA), a software product development company. He oversees the development and evolution of the company's consulting services.
Return to MX: Issues Update. [9]
© 2009 Canon Communications LLC