Originally Published MDDI April 2005

Product Development Insight

Good Products from Good Processes

With disciplined project-management processes, device manufacturers can achieve high-quality, cost-efficient, and accelerated development projects.

David A. Vogel
Intertech Engineering Associates Inc.

The complexities of developing a high-tech medical device are so daunting to the inexperienced that it may seem that luck has more to do with success than anything else. Why do companies with similar products sometimes take very different amounts of time to launch new products, and why do they have very different success rates? The performance differences are so consistent that it suggests that maybe luck has little to do with it. Successful managers and companies adopt disciplined engineering and project-management processes and constantly strive to improve their processes and techniques.

Better, Faster, Cheaper

Quality, speed, and cost-efficiency are the measures of success for any engineering project. But maximizing all three qualities at the same time is next to impossible. Maximizing speed or quality often leads to a less cost-efficient product. Conversely, minimizing cost is usually accomplished at the expense of quality or speed, or both.

The relationship of quality, speed, and cost-efficiency can be visualized as a three-seated seesaw, where each seat represents one of the project-success attributes. To raise any one attribute to its maximum height, the other two seats or attributes must be lowered. The successful project manager must determine the right balance and communicate those priorities to the entire project team.

However, there is more to product development than those three factors. In the medical device industry, regulatory compliance trumps everything else. If the regulatory body doesn't accept the product or the processes used to develop it, then speed, quality, and cost-efficiency count for naught.

Quality Has Many Dimensions

Every project engineer wants to design and develop a high-quality product, but quality means different things to different people. For example, regulatory agencies judge quality in terms of demonstrable evidence that a device is safe and effective. Users think the device is high quality if it meets their needs in terms of functionality, performance, and ease of use. Sales departments think a high-quality device has all those attributes at a competitive price. Service departments recognize quality in a device that requires little service or telephone support and that has a small number of field-replaceable units. Engineers think a high-quality product is one that needs no maintenance releases, or if it does, it is so clearly documented that anyone in the department could work on it. Finally, regulatory affairs departments think a project is high quality if it stands up well to an audit or inspection. The project manager must consider all these dimensions of quality and arbitrate the difficult trade-offs when different quality measures are in conflict.

Speed Kills

The concept of speeding development along the schedule seems an obvious preference. But is it? A shorter development schedule is better than a longer one only if all other things are equal. To be sure, development speed can be improved in a nonlinear way by adding resources. Perhaps the best and most-consistent long-term way to shorten a schedule is by improving processes and streamlining communications among the project team's members.

Unfortunately, schedules are often compressed to meet a mandated end date. Such compression is achieved by dropping tasks, making optimistic assumptions, and taking shortcuts in the development process. But speeding the process along by meeting a deadline may mean that a series of maintenance releases will be needed to finish the project later. In addition, the service department might be saddled with problems for months after release. Taking shortcuts actually postpones work until after the original schedule, when doing it will usually take longer than if it had been done properly the first time. It becomes a shell game of moving tasks out of development and into service or maintenance engineering.

In addition, demanding and unrealistic goals can have a destructive effect on a project schedule. When a project team widely believes that a schedule is unrealistic, its members stop trying to meet that schedule. Even worse, when an unrealistic schedule is ignored, no reasonable schedule takes its place. If a team no longer works toward common specific schedule goals, the demand for an abbreviated schedule can actually delay a project.

Pay Now or Pay More Later

Cost behaves much like speed in a development project. Often, development projects are squeezed into an existing budget or a budget set by someone outside the development group. A project manager often responds by cutting corners when possible. Some managers attempt to cut development costs by reducing the project team to a core group of experts, by eliminating so-called unnecessary documentation, or by letting team members bypass certain development phases.

Unfortunately, while such theoretical cost reductions seem feasible on paper, they are seldom realized. Cutting corners usually means additional costs to maintain and service the resulting product. Hence, cutting corners does not really reduce development cost so much as shift it into the maintenance phase. If the corner cutting results in a poor-quality product, the product's sales are affected. And, in the long-term view, the company's reputation can be adversely affected. Both effects lead to reduced sales and decreased profitability.

Once an unrealistic budget is recognized as such by the development team, the budget is often ignored. Since there is rarely a reasonable backup budget, the team continues on without recognized budget goals. An unreasonably small development cost can therefore actually contribute to the cost's running out of control.

Regulatory Acceptance—The Trump Card

All medical devices are subject to regulation by FDA. The very presence of a product on the market depends on the agency's acceptance of the device's safety and efficacy. In addition, the agency must accept the processes and controls that were in place when the device was designed and developed.

Experienced and successful project managers take regulatory compliance into consideration right from the start of the project. They follow the design control requirements of FDA rather than attempt to control the design after the fact.

FDA's design control and validation requirements are nothing more than requirements for a good engineering process. Implementing the controls and processes as recommended by the agency can actually lead to better-quality, lower-cost development. Retrodocumenting design control is tedious and expensive. It delivers none of the advantages of the controls, because the process has not really been followed. In the context of the seesaw analogy, regulatory compliance is like the fulcrum of the seesaw. If it is missing, none of the seats will get off the ground.

The Value of Time

Time is a project engineer's biggest enemy. Every medical device has a finite salable lifetime in the marketplace. Every day that a product is in research and development shortens selling time by one day. In addition, every day a project is in research and development extends by one more day the window of opportunity for requirements to change. Changing requirements leads to longer development times. Likewise, project delays invite requirements changes. This loop leads to unstable situations that can ultimately cause a project's failure when it runs out of time or budget, or when the product is so patched with changes that it becomes unreliable.

A delayed market release can have other consequences as well. New competitors can invade the market space, making it difficult to establish a market foothold. Changes in medical practices and standards can make an instrument obsolete before it is released.

Increased costs also can eradicate the anticipated profits from a new product development. Additional manpower costs for delayed development projects are seldom small and are usually underestimated. Plus, direct manpower charges are only a portion of the total cost of a delay. Administrative and overhead costs are proportional to calendar time, regardless of how many people are late with their contributions to a project. Additionally, test engineering, manufacturing, sales, marketing, and other departments can be idled when projects are delivered later than planned.

Building the Team

Building an engineering development team is difficult. The group of engineers must be able work with each other and have mutual respect for each other. Some team members should be experienced in regulated medical device development.

Many companies have problems building engineering teams that are compatible with their quality, validation, and test teams. A successful manager will cultivate an environment in which all consider it a success when problems are found early or at least before the product is released. It is important to build relationships such that one subteam does not succeed at the expense of another subteam's failure.

Many companies find it efficient to contract some of their development and validation tasks to outside contract service providers. This can be especially effective if the outside resource has relevant technical or regulatory experience. When comparing internal sourcing with outsourcing, the following precautions should be taken.

Make sure that you will own the exclusive rights to any intellectual property resulting from the contract. Specifically, require that all work is being done exclusively for you (i.e., not the result of work done for the contractor's previous clients). Similarly, require that your work product will not be reused for the contractor's future clients. Put this agreement in writing.

Don't depend too much on individual freelance contractors. Contractors are independent and have little or no loyalty to your company. If you need to rebuild the team in the future for maintenance or feature enhancement releases, you may have problems recreating the same team.

Let outsource service providers do their job. If you contract partners to leverage their technical or regulatory expertise, then let them advise you. Too often, the client binds contractors with unnecessary constraints and requirements.

Set early expectations. Based on the service provider's role in the project, determine what documentation that contractor will need to provide you so that you can maintain the product it has designed. Note that you should be looking for the same documentation (no more and no less) as from your in-house developers. Setting a documentation expectation is a complex process.

Validation

FDA has clearly stated that validation activities are key to creating quality medical devices.¹ There still exists a misperception in parts of the industry that validation only refers to testing activities. Validation and verification activities should be taking place at all phases of the development lifecycle for software and hardware.

Early in the development process, validation activities help ensure that the product being developed will satisfy user needs when it is completed. Finding product deficiencies early in the development process helps improve quality, decrease time to market, and minimize development cost. Early identification can be achieved by routine technical evaluations and review activities that fall under the definition of product validation.

Traceability is also a component of validation. This includes tracing or linking higher-level requirements to lower-level requirements and to design elements, or to test designs that verify the proper implementation. Experienced project managers recognize that the value of traceability is much greater than simply satisfying a regulatory requirement for documentation. Traceability is a powerful tool to guide the activities of developers and testers. It can also provide metrics for monitoring project progress. Some uses for traceability data include the following:

• Identifying high-level requirements that have not been further defined, designed, or implemented.
• Identifying design elements that don't trace back to requirements. This can highlight areas of requirement inadequacy, “gold plating,” and invisible functionality that would not otherwise be tested.
• Tracing safety-related requirements through the development process. Design elements that trace back to safety-related requirements are clearly identifiable and less likely to be unwittingly changed or removed in later releases.
• Analyzing trace ratios to uncover inadequate or unnecessary repeat testing.
• Counting unlinked higher-level requirements to provide a metric for completion statistics and schedule progress.

As with other types of design documentation, the value of trace documentation diminishes as it is produced later in the project. Traceability is an effective management tool, but it does no good if it is not available until the end of a project.

Managing Risk

The risk management process described in ISO 14971 can be extended to apply to systems, subsystems, hardware, or software.² There are, however, a number of misconceptions about medical device risk management that should be considered outside the standard.

Risk management is not synonymous with risk analysis, hazard analysis, failure mode and effects analysis (FMEA), or fault tree analysis (FTA). Risk management comprises analysis of risks and identification and design of risk control measures. Methodologies like FMEA, FTA, and others are useful techniques for identifying hazardous situations that may result in risks.

Risk management is not a one-time activity in the course of the development project. Various risk-management activities can and should take place at each phase of the development life cycle. If risk management is only considered early in development, it will not encompass failure modes that could be projected once the design evolved. But if risk management is considered only near the end of a project, risk control measures may be very expensive to retroengineer into the product.
Risk management is a system-level activity. It is not possible to do software risk management without considering the system-level risks and the interactions with the device hardware.

Properly designing risk control measures into a system will reduce the risk of a safety-related incident to an acceptable level. An iterative risk-management process will ensure that risks are addressed at all levels.

A systematic approach to risk management enables project managers to focus engineering and validation activities in functional areas of higher risk. Conversely, low-risk areas of functionality are also identifiable. These areas can benefit from a somewhat reduced level of development and validation rigor. A serious risk-management process can help improve quality and reduce the cost of development.

Management's Contribution to Success

Managers should view their role in a development company as facilitators and guardians of the process. Manpower is probably the largest expense in development. So, managers should calculate the return on investment by comparing labor costs for development and validation engineers with the time and cost savings realized by the work the engineers do. When evaluating cost, managers also need to consider the total cost to the company of a project that is far behind schedule, not just the cost against their own development budgets. In general, the cost of a few extra prototypes or test fixtures is insignificant compared to the cost of the project running even a few weeks longer.

Managers can also contribute to cost and schedule overruns. Perhaps the most common managerial flaw is indecision. Managers need to consider the costs they impose on the development company by slow decision-making. All too often, a manager takes a month to decide about a week's worth of work. Equally disruptive is a decision made quickly, only to be changed a number of times later. The costs of decision problems to the company are difficult to calculate; there are often secondary or tertiary effects on related activities or departments.

Projects that are highly process-driven with a disciplined engineering approach are usually more-successful projects by the measures of quality, speed, and cost-efficiency. Also, it is important not to forget regulatory compliance. Thoughtfully introducing formal processes into a development environment can optimize these measures to meet the desires and goals of the company.

References

1. General Principles of Software Validation: Final Guidance for Industry and FDA Staff [on-line] (Rockville, MD: FDA, CDRH, 2002); available from Internet: www.fda.gov/cdrh/comp/guidance/ 938.html.
2. ISO 14971, “Medical Devices—Application of Risk Management to Medical Devices” (Geneva: International Organization for Standardization, 2000).

Copyright ©2005 Medical Device & Diagnostic Industry