Originally Published MDDI September 2001

DEVELOPMENT

When development efforts aren't well managed, appropriately documented, or realistically planned, a company's most innovative product ideas might never make it to market.

Brian Kelleher

By some estimates, less than one-third of medical device development programs that receive funding actually result in a marketable product. Of those that do, nearly all face at least one crisis along the way. A review of dozens of device development programs from the last two decades reveals seven types of problems that often contribute to project failure: launching products prematurely, failing to appoint satisfactory project leadership, misjudging lead times, trying to accomplish goals in parallel with each other, putting off challenges, straying from project requirements, and offering engineers uncontrolled freedom during the creative process. Even projects that are well managed, well funded, and well staffed may be susceptible to these problems. Fortunately, device companies can take steps to increase their chances of avoiding them.

1. LAUNCHING A PROJECT TOO SOON

Projects launched before the proposed product has been rigorously analyzed and evaluated begin their lives on a shaky foundation. As a result, resources and time can be lost when an insurmountable obstacle appears later on in the process. What follows are the most common flaws that device companies frequently overlook at the beginning of a project.

Poorly Defined Specifications. In an effort to launch a project without limiting their strategic options, companies sometimes begin with a vaguely written or overly broad set of product specifications. For instance, an orthopedics company enlisted the help of an electronics firm to develop a stimulator for incorporation into a certain type of brace. The stimulator was specified to produce a broad set of waveforms for pain control, muscle rehabilitation, and bone growth stimulation. Since these three stimulation therapies are vastly different not only from an electronics standpoint but also from a regulatory and user-interface perspective, the project's direction was never clear. The resulting product was expensive and its user interface was complicated. Consequently, it didn't succeed in the marketplace.

To avoid a situation like this one, marketing personnel must explicitly define customer needs, and scientists must quantify performance criteria more objectively. Equally as important, development teams need to make managers aware of specifications that remain unclear or that otherwise necessitate added effort or expense.

Insurmountable Technical Hurdles. Technical problems are frequently responsible for total project failure. For example, a CEO of an early-stage medical device company came up with an idea for treating a certain form of reflux disease with an implantable valve. The idea of a valve was attractive because of its simplicity and potential for healthy profit margins, but questions about long-term biocompatibility made the technology risky. Nevertheless, the company moved ahead with the development of the device, focusing its engineering efforts on the valve form and materials while sidestepping the challenges of the valve's installation and compatibility with surrounding tissue. After the company had devoted nine months and nearly a million dollars to the project, preliminary acute animal tests revealed the approach to be essentially unworkable because of the convulsive nature of the esophagus. Had the initial concept been carefully evaluated by qualified technical personnel, this endeavor might have been a success.

Intellectual Property Problems. Although intellectual property issues are likely to define the competitive landscape for medical device companies in the twenty-first century, a number of companies are still relying on cursory patent searches performed by their attorneys or their attorneys' agents. These types of searches generally turn up much of the relevant prior art, but it is not unusual for important existing patents to go undiscovered. As a result, years of development may need to be redirected once the intellectual property situation is completely revealed. To avoid such unpleasant surprises, development programs should involve a great deal of early interaction between the inventors, the development team, and the patent attorneys in order to devise and execute robust offensive and defensive intellectual property strategies.

Unacceptable Manufacturing Costs. Although most experienced managers know better than to leave manufacturing staff out of the development cycle, few have the luxury of engaging a manufacturing engineer to evaluate critical cost targets and tolerance issues at the start of a project, especially in small companies. Unfortunately, omitting this important step can cause far-reaching consequences. Take, for example, a flexible-endoscope start-up firm that developed a novel disposable sheathing system. The company launched the product with sheaths made using labor-intensive processes and short-run tooling, hoping to significantly reduce production costs by automating the manufacturing process once market demand had been established. Despite the company's eventual investment in hard tooling and automation, however, costs could never be reduced to a profitable level. A focused manufacturing analysis in the early stages of the company's project planning might have led to a business-saving course correction. For smaller companies like this one that lack a staff manufacturing engineer, outside vendors can be recruited to assist with an analysis.

Poor Financial Return. Many projects that face a crisis situation do so because of financial problems. Either development costs run out of control, or revenue falls short of expectations after market launch. Both of these unfortunate circumstances can catapult a company into financial crisis, especially in the case of start-up companies with a single product focus. In these situations, any cost overruns or revenue shortfalls can lead to increased scrutiny from investors and board members, many of whom lack first-hand experience with the challenges of product development. The best way to thwart this risk is to build reasonable worst-case assumptions not only into cost projections but also into development timelines and revenue pro formas.

Assuming worst-case scenarios, however, is especially unappealing to start-up companies, because the resulting projected return on investment might look less attractive to investors. Fortunately, most experienced entrepreneurs have learned that it's much better to accept a lower valuation on the first round of capitalization than to risk launching a plan that's unachievable under real-world conditions.

2. INADEQUATE PROJECT LEADERSHIP

Viable projects are sometimes doomed from the start by a lack of effective project leadership. The following are some typical leadership errors made by project managers and by the companies that appoint them.

Following a Flawed Decision-Making Process. Most medical device development projects consist of a series of investigations, analyses, and decisions. Critical to the success of a project is a decision-making process that is defined at the start and remains consistent throughout the life of the project. Without a consistent and well-defined decision-making process, decisions tend to be made, reversed, and reversed again later in the project, often without any new information as justification. The most successful projects encourage a team-oriented decision-making process but provide for a single, strong leader as the ultimate decision maker. As an additional preventative measure, each decision should be documented in the design-history file—along with a brief rationale—so that if the issue comes up again, it can be quickly and definitively addressed.

Making Frequent Managerial Changes. Few things can undermine a program more profoundly than an ineffective manager. Proper mechanisms should be in place within a company to permit such a problem to be quickly recognized and remedied. Too often, however, companies overreact to poor leadership by changing managers too readily. When afflicted by this unfortunate manager-of-the-month syndrome, projects suffer from the inefficiencies of redefining development strategies and shuffling priorities and resources. As a result, project team members become disenchanted and productivity suffers.

In one recent extreme example of this phenomenon, an orthopedic start-up company appointed seven different managers during the first year of its principal development program before finally settling on a matrix-management structure with a strong central leader. Incredibly, the project survived.

Lacking the Authority to Get the Job Done. While charisma and strength of personality play significant roles in a manager's success, he or she must also possess the authority to request and elicit cooperation from each product development team member and to quickly marshal other in-house and outside resources as needs arise. Furthermore, an authoritative manager must be free to wield influence and break rules (within reason) in order to complete tasks on time. Such authority can only come from upper-level management, such as a CEO or vice president of R&D, and should be visibly granted at the project's inception and reinforced throughout its duration.

3. UNDERESTIMATING LEAD TIMES

It may seem obvious, but the single most devastating mistake made in a time-critical medical device development program is to underestimate necessary lead times. Five examples of lead-time estimation errors—and their consequences—are presented below.

Lead Times for Ordinary Parts. Over the past several years, obtaining seemingly unremarkable components such as capacitors, logic integrated circuits, and even connectors has required lead times of up to a year. In particular, electronic components in heavy demand for use in cellular phones, such as tantalum capacitors, can be nearly impossible for medical device companies to find. Furthermore, because most medical device companies deal with relatively low volumes of products, these firms often find themselves at the end of the line when commodity components are in scarce supply. Accordingly, drawing up a preliminary bill of materials and fleshing out any lead-time issues early in a project is well worth the effort. Finally, early vendor involvement can help developers avoid potential supply problems.

Custom Part Fabrication. Underestimating the time it takes to have a custom component fabricated, whether that component is a molded plastic part or a custom integrated circuit, is a common cause of project delay. Project managers typically insert vendor-quoted lead times into their schedules, when, in reality, obtaining production-grade parts (especially those with unusual features such as tight tolerances or exotic materials) takes 50–100% longer than the vendors' estimates.

For example, a company developing tiny jaws for an endoscopic-biopsy tool chose metal injection molding as the fabrication method. The lead time quoted by the vendor was eight weeks, and the first batch of parts was delivered in about nine. Unfortunately, that batch didn't meet the company's tolerance goals. So, the parts underwent several subsequent design iterations and mold changes, which took an additional 16 weeks to complete.

Safety and EMC Testing. Relying on the standard lead-time quote from a compliance agency can create another problem. For sensitive electronic equipment, electromagnetic compatibility (EMC) testing can be expensive and may take several months longer than anticipated. The best ways to mitigate this risk are to enlist the help of experienced advisors during the initial design phase and to plan for periodic safety reviews and EMC prescans after each critical piece of the system is prototyped.

Product Validation. A carefully constructed project plan should include a healthy budget for verification and validation efforts throughout the life of the project. Even with such a budget, however, some potentially problematic issues must be kept in mind to avoid surprise delays. For example, sterile disposable medical devices must undergo a sterilization validation procedure requiring a significant number of production-quality devices to be tested. Though the time required for testing is minimal, it's often an overlooked step that has to be sandwiched between pilot production and product release. For reusable medical devices, the requirements are much more demanding; often weeks or months of reprocessing and accelerated aging may be required to ensure that devices will maintain their reprocessing capabilities in the field.

Product Transfer. Misjudging the amount of time and effort needed to transfer a product from R&D to manufacturing is another common development pitfall. Tasks such as test-plan development, fixture design, and drafting of inspection procedures and work instructions rarely make their way into a project plan with adequate allocation of time and resources. As a result, project teams often face a few extra months of work after their products are supposedly "finished." To mitigate this risk, companies should encourage the involvement of manufacturing personnel and key vendors as early as possible in the development process.

4. TRYING TO DO EVERYTHING IN PARALLEL

In an effort to accelerate development programs, medical device companies have learned to devise project plans with multiple parallel pathways. This type of planning can get out of hand, however, when the plans encompass everything from basic research and animal testing to software validation and fixture design. Parallel planning is attractive on paper, but it often falls short of expectations in practice. The following problems present challenges to parallel-planning efforts.

Resource Overloading. Most Gantt chart users don't take full advantage of the capabilities of project management software. They typically plug in the task descriptions, dependencies, costs, and resource names, but that's all. The program displays task bars and costs and identifies critical paths, but it doesn't really pass judgment on the practicality of the plan. By contrast, if resources are accurately characterized, and if resources needed for the project are realistically estimated (resource leveled), the software will serve a more useful purpose. For example, it may reveal that either the company needs five times its existing engineers or the project schedule should be five times longer than planned. Managers sometimes turn a blind eye to resource leveling with the hope that their team members will work five times harder if they need to. Unfortunately, the result can be a chronically burnt-out, demoralized project team that's always behind schedule.

Unmanageable Interdependencies. When developing a simple product, such as a bone plate, it's relatively easy for a firm to execute several tasks in parallel because few individuals are involved and few variables exist in the development equation.

On the other hand, when developing a more complicated device, such as an electromedical system, keeping all the different tasks moving along on schedule and tracking the various dependencies poses a significantly more difficult challenge. Control is quickly lost when any significant change in one task affects others, resulting in a domino-like sequence of task adjustments and delays. This often leads to confusion, finger-pointing, and a general breakdown of teamwork.

Premature Filings. For many projects, the tasks of longest duration are those relating to regulatory approvals and patent applications. In an effort to keep these from holding up the rest of the project, there is a real temptation to file 510(k)s, premarket approval (PMA) applications, or patent applications early in the project. There is risk in doing so; for example, filing an incomplete 510(k) can result in an outright rejection by FDA during its initial screening of the application. Such a rejection can waste time and taint the company's reputation with the agency. On the other hand, if the submission passes the initial screen, the design team may be forced to live with the specifications, design architecture, and materials that were disclosed in the early submission—all of which may become obsolete as the project matures.

As for patent filings, submitting an early application (such as a provisional filing) may be a good idea if the technology or concept is well characterized, or if a general methods patent is desired. But designs often take many unexpected twists and turns as projects progress, deviating significantly from what may have been described in an earlier patent filing. Accordingly, early filings may be of little use in protecting later-stage developments.

5. SAVING THE HARDEST TASKS FOR LAST

Often, medical device companies find themselves under significant pressure to show progress in a development project soon after it is launched. Because of this pressure, they sometimes put off dealing with the most difficult or riskiest engineering challenges until much later in the project.

The Trade Show Effect. Major trade shows and scientific conferences are prime venues for new product introductions, but their timing can sometimes lead to undue haste. Because marketing departments are often anxious to generate customer enthusiasm for a new product at such events, they may feel tempted to launch a product prematurely. In one case, a company developed a novel ambulatory pH-monitoring system and chose to launch it at a major conference in the fall, though quite a few significant design problems remained unsolved. To meet a pending trade show deadline, the engineering team stopped working on the more critical problems associated with the product to focus exclusively on making working models for the show. While the models were compelling enough to elicit purchasing interest from early adopters, the company needed an additional six months to work through the tough problems it had sidestepped. Consequently, interest in the product waned, and the company's relaunch at the next major conference the following spring was relatively lackluster.

Management by Milestones. Over the past decade, industry has seen a growing trend toward milestone-based funding by venture capital groups. These schemes are intended to align the interests of investors with those of entrepreneurs by granting incremental funds as certain predetermined milestones are met. Commonly cited milestones include "working prototype complete," "patent application filed," and "successful animal trials performed." Unfortunately, these circumstances can tempt a company to focus entirely on achieving its milestones to qualify for each round of financing, thereby putting off the most difficult technical challenges for last. Take, for example, the start-up firm that was developing a novel eyeglass-lens manufacturing process. The company raised more than $6 million by completing milestones set by its investor group. These milestones included the development of engineering prototypes, the completion of a marketing plan, the establishment of an advisory board, the hiring of a sales force, and the construction of a manufacturing facility. Unfortunately, when the product went through its final validation, the underlying process was deemed inadequate to support the required accuracy and cost targets. In the end, the company folded.

Design First; Ask Questions Later. It is typical for early-stage companies to function in constant money-raising mode, with potential investors parading through their facilities to inspect the technology at any given time. In this type of atmosphere, it's unpopular for company insiders to raise any questions about the viability or marketability of the product or technology, for fear of scaring off investors. At the same time, however, project teams need to show significant continued progress, such as the development of working models or preclinical prototypes. As a result, the development effort may appear to progress smoothly on the surface while, underneath, there may be a growing unspoken anxiety about the underlying technology. Unless a major breakthrough occurs, product development stalls when the shortcomings of the technology surface and investor expectations clash with reality.

6. CHANGING REQUIREMENTS

A medical device development project can be compared to a chain of school kids playing crack the whip on a playground. In the early stages of development, when just a few people are involved, changes in requirements at the front of the chain yield limited influence on those at the end. But later in the project, when 10, 20, or 100 people are involved, small changes in requirements become amplified along the chain of tasks. The unlucky people at the end of the chain are whipped back and forth so hard that they can hardly hang on. As such, it's crucial to the success of a project and to all those involved to carefully develop a set of viable product requirements, and then to adhere to them as closely as possible throughout the project. Unfortunately, doing so isn't always practical, especially when projects are planned with many interdependent tasks happening at the same time. Some of the most common factors that lead to changes in requirements include the following.

Changes in Basic Science. Whenever basic science tasks are planned in parallel with development tasks, the potential for crisis is likely. Take, for example, a company that developed an iontophoretic, transdermal drug-delivery system. A preliminary analysis revealed the desired drug-transfer rate was possible based on certain assumptions about ionic current flow. In conjunction with experiments to confirm the drug-transfer rate, engineering efforts to design the control electronics and electrodes were set in motion. To the disappointment of those involved, the drug experiments revealed that the desired rate of transfer could only be achieved with costly changes in the carrier solution and the drive electronics. Consequently, the product requirements were significantly revised and most of the initial engineering efforts had to be scrapped.

Unexpected Preclinical Test Results. When devices are at last put to use on animal subjects or patients, test results are often disappointing or otherwise surprising. Inevitably, if the project survives in spite of such results, product requirements must be modified. Unless preclinical testing is performed before significant engineering has taken place, changing the requirements at this stage can have wide-ranging consequences.

New Market Information. As a product development project nears completion, control of the project often shifts from engineering to marketing. When this shift occurs, marketing personnel begin bringing models and prototypes into clinical environments for simulated-use evaluation. During this process of disclosure and evaluation, new customer preferences, logistical constraints, and competitive activities may be discovered. These types of market revelations can seriously impact product requirements and can frustrate members of the technical staff, who may rightfully question why such issues weren't uncovered at the beginning of the program. Therefore, early and intimate collaboration between engineers, marketing executives, and customers in an actual clinical environment is crucial to timely project success. After an initial specification is approved by the group, product improvement ideas should be logged in a separate section of the design-history file for incorporation into future versions of the product.

Revising the Company Vision. As if insurmountable technological hurdles and uncontrollable external events aren't challenging enough on their own, some companies further cripple themselves by periodically changing their fundamental corporate direction. This type of change is sometimes a direct consequence of the manager-of-the-month syndrome described earlier, or it can result from an indecisive CEO who is hesitant to pick a single direction and stick to it. One start-up company, for instance, changed its vision three times within its first year of operation. The ramifications of these vacillations are great: projects are started, refocused, and started again. In addition, highly qualified personnel may leave to pursue opportunities with more clearly defined projects and a better chance of bringing products to market quickly.

7. GIVING ENGINEERS FREE REIN (AND CLEANING UP AFTERWARD)

Creative technical people are rarely enamored of the rigors of disciplined project management and document control. Unfortunately, an uncontrolled development phase can result in nonconforming design history files and additional delays and expense on the back end of a project. The following items and tasks are most frequently ignored when the creative process runs unchecked and unsupervised.

Formal Requirements Documents. Neglecting to draft and release formal requirements documents prior to performing design tasks is a commonly taken shortcut. Engineers are often heard saying, "Design it first, then write the spec." To a certain extent, this order of events makes sense, though it violates all basic design control guidelines and leads to an unacceptable design history file. As a compromise, product development programs should plan for a concept development phase in order to settle any design uncertainties and to lay the foundation for a stable requirements document.

Human Factors Analysis. Engineering-driven projects tend to de-emphasize the need for human factors analysis during a project's evolution. Not infrequently, an industrial designer is brought in late in a project, after many decisions affecting human factors and ergonomics have already been finalized.

Take, for instance, a company that developed an ear thermometer under the guidance of two male physicians. Though the thermometer worked well for them, its resemblance to a handgun made mothers and some nurses reluctant to point it at children's ears. As a result, the product failed. A few representative focus groups and more human factors analysis in the early stages of development might have identified this problem and could have saved the otherwise successful project from its fate.

Up-Front Risk and Failure Analysis. Another tendency of engineers is to put off formal risk analysis and failure modes and effects analysis until after a design is completed. Unfortunately, this belief may cause an engineering group to suffer through a redesign effort if significant unmitigated hazards are discovered at the end of the design phase. To conform with design control guidelines, a formal set of analyses should be conducted before design efforts begin. The analyses then function as a roadmap for the engineering team and help team members make critical decisions that affect safety and performance.

Adequate Verification. Failure to conduct timely verification steps or perform design reviews after each design phase not only violates design control guidelines, but also circumvents the check-and-balance benefits that are fundamental to any quality system. A well-planned project breaks up the overall design into manageable chunks, each with its own verification step and design review. Engineers familiar with this type of chunk-by-chunk process learn to recognize that it isn't so different from what they normally do to verify that each design piece actually works. Performing verification tests after each design phase simply formalizes the process and documents the results.

Quality and Manufacturing Involvement. Keeping the quality and manufacturing groups out of the development process seems to be a natural, albeit unfortunate, cultural tendency in most companies. This tendency not only breeds ill will between development and other departments, it can also create a serious delay in the product release schedule. For example, process steps that seem straightforward to the R&D group may actually require significant refinement and validation before they are ready for full-scale production. Furthermore, early quality and manufacturing input on manufacturability, testability, and reliability issues can shorten the transition time from R&D to manufacturing and ultimately result in a better, more profitable product.

CONCLUSION

Most development pitfalls can be avoided by following these simple guidelines:

By following these rules, developers can maintain control of their projects and improve their chances of success. The seven deadly sins of medical device development pose little danger to companies that begin development with a healthy dose of preliminary analysis, planning, intellectual honesty, teamwork, and discipline.

Copyright ©2001 Medical Device & Diagnostic Industry