Return on Investment in Human Factors

FDA's quality system regulation (QSR) requirements essentially mandate the application of human factors processes and design principles in medical device development.¹ As Peter Carstensen, FDA's human factors team leader, says, “If you don't perform human factors analyses in the course of medical device development, you're technically breaking the law.”

Therefore, the days of giving lip service to human factors are numbered. FDA requires that manufacturers conduct comprehensive user research to generate device requirements and assess user interfaces. Market research activities, such as asking trade show attendees to comment on a prototype, do not meet these requirements. Similarly, device styling exercises do not supplant rigorous ergonomic analyses. Instead, companies must assemble a human factors program consistent with the guidance presented in ANSI/AAMI HE74:2001, which FDA recognizes as a proper interpretation of the QSR requirements.²

The payoff for caregivers should be safer devices that are also easier to use. This is an important goal, considering the frequency of patient injury and death owing to improper device operation. Developers should also realize a handsome payoff, which may approach or exceed 10 times the level of investment in human factors. However, the payoff builds over several years (see Figure 1), so company managers must exercise both foresight and patience.

Benefits

Few would question the value of good user interface design. It is something people notice and appreciate in their cellular phones, microwave ovens, automobile navigation systems, and many other utilitarian and recreational items. Similarly, medical professionals notice it in their patient monitors, infusion devices, surgical instruments, scanning equipment, hospital beds, and myriad other devices.

Common denominators of good user interface design include logical display and control layout. It also encompasses comfortable touch points such as handles and body supports, immediate feedback in response to user inputs, and even appropriate aesthetics. These attributes directly contribute to user performance, enabling tasks to proceed swiftly and effectively with little chance of use error. In turn, improved user performance produces several tangible benefits. These benefits translate into dollars made and saved by the manufacturer, as shown in Table I. The analysis of benefits in this article assumes the development of a moderately complex medical device, such as an infusion pump.

Faster Time to Market. Many within the engineering community view a human factors program as an obstacle to bringing a device to market. They consider it one more activity on the development schedule that can add time to the project. However, well-planned and well-executed human factors programs are actually more likely to speed up the design and development process.

Consider the dilemma that a company faces when initial users find a prototype difficult to operate. Often, it's back to the drawing board to revamp the user interface. Or consider the delay resulting from pushback by FDA on a premarket approval application or 510(k) submission. If FDA cites concerns over a device's complexities and doubts about the intended users' ability to operate it safely and effectively, schedules can be ruined.

There is a chance that taking a human factors approach to user interface design early in the development process will extend the development schedule. This is particularly true if the work is performed in isolation. However, it is possible to integrate human factors seamlessly with other engineering activities so that it takes no additional time. In either case, the chance of schedule delays late in development due to usability problems should be greatly diminished, thereby avoiding more-substantial delays.

Calculation: Take as an example a 2–3-year development project. The human factors investment on such a project might save an average of two months. Assuming a cost rate of at least $100,000 per month, a manufacturer could save $200,000. Moreover, the developer might start making money on the new product sooner by getting to market ahead of competitors. One could conservatively assume that this benefit amounts to another $50,000 in profit—the result of increased competitive advantage and earlier profits.

Simpler Learning Tools. Logic suggests it is easiest to create learning tools, such as user manuals, quick-reference guides, and online help, for a device that is easy to use. Usable devices eliminate the need for long and detailed instructions and illustrations on how to perform tasks. Therefore, technical-writing tasks also proceed more swiftly.

Calculation: If learning-tool development normally employs two people for six months, a 25% reduction in effort would save three person-months. If a company's burdened labor cost is $100,000 per year (a low estimate), the one-time savings could be $25,000. Additional savings could accrue from the reduced cost of printing shorter user manuals (e.g., $5 per copy). With unit sales boosted 10% (e.g., from 4000 to 4400 units per year), the annual savings would be $22,000.

Increased Sales. Companies are usually eager to increase sales—better user interface design can make that happen. That said, it is difficult to separate the sales effect of usability from other variables, such as functional capability, reliability, unit price, service plans, and overall marketing prowess. However, many companies make their products' usability a strong selling point. At the same time, usable products sometimes sell themselves, outclassing competitors' products in head-to-head comparisons. Also, clinicians often extol a particular product's ease of use over other offerings. Such positive reviews ultimately translate into more sales.

Calculation: Assume the sales of a predecessor product had reached a plateau of 4000 units per year and that an easier-to-use replacement boosts sales an average of 10%. While the boost may compound over time, a one-time, permanent boost translates into another 400 sales per year. Given a unit price of $3000 and a profit margin of 10%, increased usability could generate another $1.2 million per year in revenue and $120,000 per year in profit.

Reduced Reliance on Customer Support. Customers keep a device manufacturer's telephone number on hand in case of equipment problems, including usability problems. It is common for clinicians to call the customer-support hotline if they are stumped on how to use a particular feature. Such calls may keep a bank of customer-support representatives busy. Increased usability arising from investing in human factors can reduce the number and duration of calls. It might also reduce the time required to train new users. But in-service training sessions also serve other marketing purposes and should not necessarily be targeted for reduction.

Calculation: Suppose a customer-support center had been operating three shifts of six staff members. That amounts to 18 full-time staff members who may take calls about several different products. Reducing calls by 30% because of improved usability might remove the need for one customer representative per shift. Using the same burdened salary cited before ($100,000) for learning-tool developers, the savings might amount to $300,000 per year. However one may want to be conservative with the benefits estimate. Assume a smaller customer-support operation that employs three people during normal business hours. Reducing the customer support need to two people would save $100,000 per year.

Extended Design Life. The classic product life cycle includes the following stages: introduction to market, growth, maturity, decline, and withdrawal of an existing product.3 The introduction of a competing product with a better user interface is liable to hasten decline and withdrawal of an existing product. Conversely, products with excellent user interfaces may stand up to competitive pressures, thereby extending a product's life cycle.

Calculation: Suppose a great user interface extends a product's life just one year, resulting in withdrawal at the end of year six instead of year five. In effect, this reduces design costs. A $2 million design effort spread over five years costs $400,000 per year (disregarding the time value of money for simplicity's sake). Spread over six years, the design effort costs $333,333 per year. In this case, the savings over the course of multiple design cycles amount to $66,667 each year.

Product Liability Protection. Product liability protection is a key variable in the return on investment (ROI) calculation. Particularly in the United States, manufacturers face the possibility of lawsuits arising from user injury, death, and property damage involving their products. One way to reduce the chance of a product liability claim is to reduce the chance of a device use error. The chief goals in any human factors program are to make devices less susceptible to use errors and to make it easier for users to detect and correct errors before harm can occur. These programs can expose a device's vulnerability to errors early in the development process.

Calculation: Out-of-court settlements and jury awards can run into the tens of millions. Damage to a company's reputation can also have adverse consequences, such as declining revenues from lost sales. Presuming a 10% chance each year of a $2 million settlement over a decade of product use, plus one-time legal fees of $200,000, the annual cost amounts to $220,000. Injury and claim prevention through applying human factors can be viewed as saving $220,000 per year. However, realizing such savings may depend on a company's approach to insurance.

Other Benefits. Of course, the benefits of a human factors investment extend beyond the manufacturer. Within hospitals, an easy-to-use medical device can reduce staff training time and boost worker productivity. It can also reduce a hospital's exposure to liability claims associated with device use errors, such as misprogramming an infusion pump. Potential benefits to patients include better medical care, partly due to reduced use errors, and less physical and emotional pain. These benefits can transfer to manufacturers in the form of increased customer goodwill and future sales, but the financial benefits are difficult to calculate.

Costs

Despite the notable benefits, manufacturers might find the up-front cost of a human factors program daunting. This is particularly true if they have not accounted for such costs in their budgets. Costs may run as high as $300,000 or more for a complex product, such as a dialysis machine or anesthesia workstation. However, they may be less for arguably simpler devices, such as infusion pumps or blood gas analyzers. That said, there are ways to limit human factors program costs without sacrificing quality.

Table II presents an example of a prospective budget for human factors work. It is based on a complex medical device—one that has several operational modes, requires intense user interactions, accommodates several user populations (e.g., doctors, nurses, therapists, and technicians), and serves a life-critical function.

Table II also presents a prospective budget for human factors work on a less-complex medical device. This device has a single operational mode, calls for limited user interaction, and accommodates a single user population (e.g., nurses). However, it might still serve a life-critical function.

Both budgets incorporate several activities and costs that manufacturers already incur in the course of a traditional development effort. After all, every product operated by people requires a user interface. However, these budgets subsume the activities into a comprehensive human factors program. Also, not all medical devices warrant a $100,000 or greater investment in human factors. Simple devices that do not serve life-critical functions may call for far less human factors effort and expense.

Return on Investment

With the estimated benefits and costs of human factors in hand, we can calculate an ROI over the estimated extended product life cycle of six years. Again, for simplicity's sake, it may be useful to disregard the time value of money. One-time benefits amounted to $275,000. Recurring, annual benefits over six years—the extended product life cycle—amount to $3,174,000. Thus the cumulative benefits would reach $3.5 million over the six-year product life cycle. Given a human factors investment in the range of $100,000 to $300,000, the ROI ratio ranges from about 12:1 to 35:1.

However, all of the costs are up front (prior to product launch), and many of the benefits accrue for years after product launch. Therefore, we can now escalate the costs to reflect the time value of money. Accordingly, a $300,000 investment at an interest rate of 5% over six years escalates to $402,000.4 Similarly, a $100,000 investment escalates to $134,000. With these adjustments, ROIs decrease to about 9:1 and 26:1, respectively. Recognizing the time value of certain financial benefits, one could reasonably round up to an ROI of at least 10:1.

Finance folks might consider these estimated ROI ratios flawed or lacking in face validity because they are very high compared with traditional ROI rates. Indeed, they sound quite high. It might be easier to campaign for human factors program funding by citing an ROI of 3:1, for example. However, this analysis of benefits and costs supports higher ratios. That said, it also depends on other aspects of the development, marketing, and sales effort to proceed smoothly.

One factor that could drive the ROI downward is competition from other products with excellent user interfaces that may limit the increase in sales. Similarly, technological advances may abbreviate a product's life cycle, even if users remain pleased with its user interface. Also, any design and manufacturing challenges unrelated to human factors that delay market introduction might cancel out the added profit and competitive advantages. However, a manufacturer would still realize some benefits, including simpler learning tools, reduced customer-support demands, and product liability protection. Using the escalated value of $402,000, those three benefits alone might exceed $2.7 million, yielding an ROI no less than 7:1.

Ultimately, the return on an investment in human factors depends on the nature of the development organization, product, market conditions, and user population. Furthermore, few companies have the interest or resources to collect the data necessary to make precise ROI estimates, leaving them uncertain about the financial benefits of human factors. However, human factors proponents would be well served to justify their proposals in financial terms, augmenting other justifications, such as the pursuit of design excellence and meeting regulatory requirements.

Accordingly, projecting a human factors ROI of 10:1 seems prudent. The estimated value of reduced product liability exposure alone yields an ROI of at least 5:1, even if you assume a large human factors investment at the outset.

Conclusion

Many product developers view human factors as an elective activity that is subject to reduction or elimination when budgets are tight. Paradoxically, reducing or eliminating a human factors program may negatively affect the company's future profitability.
Some companies have spent millions to litigate and settle claims arising from medical device user errors, while others suffered sagging sales against products offering greater usability. Given the chance to go back in time, it is almost certain they would invest heavily in human factors to identify and remedy usability problems before market introduction. Meanwhile, every manufacturer has the opportunity to invest today to prevent future usability problems, and almost certainly make a handsome return on its investment.

References

1. “Human Factors Implications of the New GMP Rule Overall Requirements of the New Quality System Regulation” [online] (Rockville, MD: FDA, CDRH, 1998 [cited 8 June 2005]); available from Internet: www.fda.gov/cdrh/humfac/hufacimp.html.
2. ANSI/AAMI HE74:2001, “Human Factors Design Process for Medical Devices” (Arlington, VA: AAMI, 2001).
3. “The Product Life Cycle” [online] (Chi-chester, UK: Marketing Teacher [cited 8 June 2005]); available from Internet: www.marketingteacher.com/lessons/lesson_plc.htm.
4. “Compound Interest Calculator” [online]; available from Internet: www.moneychimp.com/calculator/compound_interest_calculator.htm.

Copyright ©2005 Medical Device & Diagnostic Industry