Developing Design Control Strategies to Meet Technology Advances

Medical Device & Diagnostic Industry Magazine
MDDI Article Index

Originally Published September 2000

Cover Story

Dan Olivier

Design controls, as defined by FDA in the quality system regulation (QSR) and by ISO in the new ISO 9001:2000 standard, include classic phases of an engineering design and development process Although this model has been the default design method used by medical device manufacturers to demonstrate compliance with regulatory requirements, the use of these methods alone has not always proven effective in meeting the escalating demands of consumers and emerging technology. Consumers demand products that are increasingly faster, cheaper, and easier to use. These demands, combined with a desire to get products to market more quickly, are forcing manufacturers to reexamine historical design control processes.

It is relatively easy to see technology advances in the cost/performance capabilities of PCs and in the capabilities of appliances such as cellular phones. Even in the medical device industry, the processing capacities of electronic circuitry continue to follow Moore's Law—a tenfold increase in products' memory and processing power becomes available every five years, for the same cost. This equation has proven to be a fairly reliable way to predict technology advances over the past 20 years. Traditional design control processes, however, often fail to integrate the benefits of this technology evolution into the design process.

In its basic form, the design control model as defined by FDA and ISO regulations has been tested by years of use. While the model is fundamentally sound, it requires adaptation by device manufacturers in order to meet new demands from consumers and to leverage the benefits of emerging technologies. This article shares the lessons learned by some companies that have successfully enhanced their design control processes. Revised product design and development practices are discussed under the categories of quality focus, project management and personnel, and leveraging technology (see Table I).

QUALITY FOCUS

Emerging technologies bring with them increased complexity and therefore a greater potential for errors. As the potential for errors increases, so too does the recognition of the importance of quality. Successful companies have learned that a quality focus shortens development schedules by reducing the rework that results from the need to continually fix design failures.

Reviews to Augment Testing. The regulations establish the need for design reviews but cannot prescribe the techniques that will ensure that the reviews are effective. Industry leaders have instituted practices that not only facilitate the conduct of reviews but also measure their effectiveness. These companies have learned that the earlier in the process design flaws can be identified, the less costly they are to correct and the less likely they are to have a negative impact on the schedule. Although some additional effort is required for review activities, these companies recognize that the payback is enhanced quality and a reduced development schedule. Studies have shown that reviews are the most effective method for identifying errors; it can cost up to 100 times more to fix a defect in a released product than it does to find and fix that same defect during reviews in the design-implementation phase. It is also recognized that conducting effective reviews requires that the personnel are knowledgeable, properly trained, and prepared for review meetings. In a study of their implementation at Honeywell-Bull (Zurich), the use of formal review techniques reduced the company's error density (number of errors per 1000 lines of code) from an average of 3.6 errors to 1.3 errors (see Figure 1).

Integrated Testing Strategy. In many medical device manufacturing companies, validation testing of the final product tends to be the most emphasized portion of the product development process. It should not be the only aspect of the process where testing is conducted, however. Testing must be addressed throughout the engineering development activities to ensure that incremental components function properly prior to integration and that integrated subsystems function properly prior to system testing. Often, the most neglected portions of the development process are the unit and integration elements of the test process that should be conducted by developers prior to system testing. Proper functional and interface testing during development and integration is essential for reducing rework during system testing. Most major system errors can be traced to inadequate testing at the unit or integration test level.

Design for Reliability. Initial product designs often focus on the newest features that designers create based on their prediction of upcoming customer needs. These new product designs, however, do not always assess the ability of the newly developed product to stand up to the misuse and abuse conditions of its end-use environment. Techniques such as a failure mode and effects analysis (FMEA), design for reliability methods, and customer usability testing can substantially reduce the likelihood of use-related problems. Subtle design errors are the most difficult to find. Situations that were not considered in the design process or that were discounted because the probability of occurrence was very low are often the very factors that cause problems later in the process. Examples of these types of design reliability errors include the widely publicized Y2K design failures in a variety of products. For an example where the stakes are as high or higher than those of the medical device industry, one need look no further than the military: the design of the Patriot missile accumulated targeting errors because of a loss of precision in a conversion routine. Errors in the range calculation were so great that the Patriot became ineffective in intercepting incoming missiles after a period of only eight hours of continuous use.

Figure 1. Typical effects of reviews on error density.
 

Product and Process Quality Metrics. Medical product design and development processes should advance with the technology of the product. Although the benefits of measuring process improvement are widely recognized, measures to evaluate design processes are not widely employed. Many manufacturers invest significant resources in the measurement of manufacturing processes, but fail to enforce similar measures for their engineering design departments due to uncertainty over which measures to use.

Proven engineering product and process measures include the following:

Through use of measures such as these, the design and development process can be optimized to enhance the productivity of the engineering and manufacturing processes.

Design for Manufacturability. Manufacturability is often neglected in the haste to quickly design a new product. Failure to address manufacturing early in the product design phase, however, can have a significant impact on the cost of manufacturing and the effort required to validate the manufacturing equipment. The release of many new products has been delayed because manufacturing processes could not reliably produce desired quantities, even though the product design validation was completed. Companies that address manufacturability late in the design process not only experience shipping delays but also encounter a high volume of customer complaints about manufacturing-related defects. Manufacturing process yields and the number of manufacturing-related engineering change orders introduced after product release are often an indication of the extent to which the design control process effectively accounts for manufacturing issues.

Figure 2. Pareto analysis of error functional categories.
 

PROJECT MANAGEMENT AND PERSONNEL

Although the need for a quality focus rises with the introduction of more-complex equipment, personnel capabilities continue to be the most significant factor in assuring the successful design and development of a new product. The importance of personnel in a successful development effort was noted by Barry Boehm as early as 1981 in his book Software Engineering Economics, where he stated that a project team with the highest rating in personnel capabilities and experience would be 10.53 times more productive than a project that was staffed with very low-rated personnel. Personnel capabilities are addressed in terms of the skills of the project manager as well as the technical expertise of development engineers.

Project Managers. Although not emphasized in engineering standards and assessment models, an expert project manager is a key to project success. The project manager's ability to motivate the development team and to focus on the appropriate development tasks is more important to a project's success than the technical expertise of the development team, the detail of the development procedures, and the technology used for the development process. The successful manager is responsible for prioritizing tasks and establishing specific process steps that are optimal for the development project. Areas of emphasis are dictated less by an established procedure and more by a continuous evaluation of project risks, as suggested by Boehm in his spiral model of software development (Figure 3). Boehm developed the spiral model based on an evaluation of the practices of successful project managers. The most successful project managers were focused on the continuous identification and resolution of project risks. This is illustrated by the four quadrants of the model. The importance of the project manager is demonstrated by data showing that, for example, more software projects have failed due to managerial causes than technical ones.

Requirement Changes. Perhaps the most significant cause of project schedule delays is a failure to properly manage changes to product requirements. Problems arise from allowing changes to be incorporated into the current product release without considering the impact on the schedule. As the scope of the product expands, schedule slips increase. Up to 40% of the errors experienced during system testing are attributed to changes in requirements. A process must be established whereby requested requirement changes are evaluated based on schedule impact as well as on customer benefit. Demands from the marketing group for new features must be evaluated in light of the effort required, the possible impact on established schedules, and the risk to quality. For many projects, a significant factor in missing planned deadlines is that the final project has become larger in scope than was initially planned. Attempts to reduce time to market can only be effective when any changes to requirements are also controlled.

Figure 3: A spiral mmodel of software design and development.
 

Knowledgeable Team Members. If project management expertise is the first most significant factor in ensuring a successful project, then the expertise of development personnel is second. Obtaining adequate expertise for solving technical problems is essential to project success. To be successful, project personnel must receive appropriate training in all procedures and in the use of required tools. If required skills are not available internally, many successful companies have learned the value of hiring outside experts; critically needed expertise can then be integrated into the team for specialty areas. Using this technique, a company can continue to focus on its core competencies. Building an optimum development team requires appropriate hiring practices as well as a commitment to personnel development. A critical factor is the selection of personnel who are willing to work in a team environment and who collectively strive for project success. Team members who value individual efforts over the results of the team adversely affect productivity. While technical competence is vital to any project, a narrow focus on personnel with more extensive technical expertise can reduce productivity if the ability to work in a team environment is lacking.

LEVERAGING TECHNOLOGY

Personnel capabilities and a quality focus alone do not ensure project success; harnessing the most appropriate technology must also be considered to ensure productivity.

Use of Off-the-Shelf Products. To meet demands for reduced time to market it is essential for medical manufacturers to leverage off-the-shelf and previously developed products. With electronic technology, the classic "make-versus-buy" decision is often best decided in favor of buying off-the-shelf components. Off-the-shelf components such as microcircuits, generic components, operating systems, database-management systems, graphical user-interface packages, and communication utilities provide significant benefits in terms of reduced development effort and a wider range of functionality. In general, off-the-shelf products are often easier to maintain, provide for increased levels of interoperability, and are frequently more reliable than internally developed applications. Use of both hardware and software off-the-shelf components enables development teams to use proven technologies, reduce costs, and shorten schedules. Clearly, unique product features are needed to provide product differentiation, but these features can be implemented in a shortened schedule with reduced cost if existing off-the-shelf products can be leveraged.

Defining New Product Requirements. Technology should be considered not only as a component of a new product but also as part of the definition of new product capabilities. Many new products fail to achieve success on the market because of the limited life of new product features. This is the result of a design process that establishes requirements based on previously developed products or on currently existing competitor products. In most cases these products have been on the market for years and the features that are offered are dated. Interviewing end-users about their needs is essential, but ultimately it is inadequate as a means of effectively predicting the need for future products. Successful manufacturers focus on innovation and consider the capabilities that will be possible based on evolving technology. Although no one can precisely predict the future, truly revolutionary products are the ones that have managed to incorporate innovative new features that define emerging trends. The designer's focus should always be to invent a better way to do work.

Adherence to Established Interface Standards. Historically, some manufacturers have developed devices with proprietary interfaces to restrict access to their products by competitors. This trend has now been reversed. Today, customers in any industry demand interoperability of new products with other systems and related products. Manufacturers that do not support these desires limit their own products' marketability. Successful manufacturers have also learned that supporting standard interfaces provides the opportunity to market these interfaces as product features. For example, the standard proposed by IBM has become the basis for the current PC architecture, while the interoperability of Microsoft products is seen as a significant benefit that has also become an industry standard. Adherence to industry standards for communication protocols and hardware connectors has allowed substantial interoperability across electronic systems. This level of interoperability enables developers to rapidly integrate new technology advances in their own product releases.

CONCLUSION

In the medical device manufacturing industry, the list of successful design and development process techniques must be constantly updated by manufacturers to stay abreast of technology advances. More important than individual practices is a focus by the design team on continuous improvement. Manufacturers must constantly evaluate the effectiveness of their current processes and strive for new optimization methods. As technology continues to evolve at an increasing rate, the need for ongoing evaluation of process effectiveness will become a principal factor in establishing and maintaining market leadership.

REFERENCES

1. Code of Federal Regulations, 21 CFR Part 820.

2. ISO/CD 9001:2000, Quality Management Systems—Requirements, (Geneva: International Organization for Standardization, 2000).

3. P Evans and TS Wurster, Blown to Bits, How the Economics of Information Transforms Strategy (Boston: Harvard Business School Press, 2000), 14.

4. FW Blakely and ME Boles, "A Case Study of Code Inspections," Hewlett Packard Journal (October 1991), 62.

5. RG Ebenau and SH Strauss, Software Inspection Process (New York City: McGraw Hill, 1994), 168.

6. "Patriot Missile Defense, Software Problem Led to System Failure at Dahran, Saudia Arabia," GAO/IMTEC-92-26 (U.S. General Accounting Office 1992), 15.

7. BW Boehm, Software Engineering Economics (Englewood Cliffs, NJ: Prentice-Hall, 1981), 431.

8. BW Boehm, "A Spiral Model of Software Development and Enhancement," in Software Engineering Project Management (New York City: IEEE, 1987), 131.

9. DP Olivier, "Engineering Process Improvement through Error Analysis," Medical Device & Diagnostic Industry 21, no. 3 (1999): 130–149.

10. S Robertson and J Robertson, Mastering the Requirements Process, (New York City: ACM Press Books, 1999), 72.

Dan Olivier is president of Certified Software Solutions Inc. (San Diego), an engineering services company that specializes in support for design controls, verification and validation, independent testing, software development, safety risk analysis, and compliance audits.

Top Photo by Andrew Brookes/The Stock Market

Back to the MDDI September table of contents | Back to the MDDI home page

Copyright ©2000 Medical Device & Diagnostic Industry