FDA guidance on human factors evaluation of medical device designs contains a loophole that allows bad design to perpetuate as long as it remains unchanged.

William A. Hyman

FDA recently issued a pair of guidances related to human factors evaluation of medical device designs. One is a guidance document on recommendations for human factors testing to be included in 510(k) and PMA applications. Dated February 3, 2016, it replaces the four-and-a-half-year-old draft of June 21, 2011. The second is a draft guidance document that provides a list of 16 generic devices categories to which the guidance document will first be applied.

The guidance document tells us it is intended to “maximize the likelihood that new medical devices will be safe and effective for the intended users, uses, and use environments.” “Maximize the likelihood” is careful terminology in that it avoids the more common assertion that a standard will in fact improve safety.

Also note the wording “new medical devices.” Because the guidance document addresses premarket submissions, it only applies—even as recommendations—to devices being newly brought to market. Well, almost.

While use-related modifications would trigger application of the guidance document, leaving usability unchanged would not if a submission “does not involve any changes to users, user tasks, user interface, or use environments from those of the predicates.”  This means that if you have a bad design from the human factors perspective, you can continue to use that bad design on new devices as long as you don’t change it.

This seems to be a demotivation to change (and possibly improve) a design since changing it would trigger a “recommended” human factors study, while not changing it would not.

Even putting such change aside, the focus on new premarket submissions means existing designs, including those now defined by FDA to be error prone, will continue to be used unfettered. This is despite the fact that FDA has said the 16 device types facing initial premarket human factors scrutiny have demonstrated use error issues “based on knowledge gleaned through Medical Device Reporting (MDRs) and recall information.”

FDA believes it knows that there are devices with “clear potential for serious harm resulting from use error” that are currently in use, but at least through this mechanism, the agency has taken no steps to have these devices removed from use or have users otherwise alerted.

Over a sufficiently long time period, these legacy devices with bad designs will be replaced by newly cleared or approved devices with presumably better designs, but how long will this take and what happens in the interim? Notably there is no mention here of recalls, safety communications, alerts, or planned phase-outs, and no direct notification to users that the devices they have do not meet what will become the new de facto standard for human factors design and testing.

When there is a mix of legacy and new devices for sale and in use, there will be no direct way to know which is which except perhaps by the date of clearance or approval. Generally, manufacturers have not been able to promote a device that went through the 510(k) process as superior to its predicate(s), especially if the predicates were its own products. Maybe this needs to be relaxed for a device that is compliant with a guidance document that its predicates did not comply with. While food companies may not hesitate to tout new recipes, medical device manufacturers may not be excited about proclaiming, “Now with better usability!” In fact, the alternative, which is to never admit there was anything wrong with your old design, is more common.

On the other hand, there is no proof that new designs that meet the recommendations of the guidance document are actually any safer than old designs that did not formally meet it. Some older designs may be very good as a result of good human factors design being applied without FDA having forced it to be done. In fact, some older designs may be better than forthcoming new designs that will have been subjected to the increased FDA usability scrutiny.

Here it might be good to note that there are many reasons to do good engineering design other than FDA has or has not recommended, expected, or required something. In this regard, what FDA requires can generally be characterized as the minimum that a manufacturer must do. I was once challenged on this assertion, but my knowledge of math tells me that if you can’t do less and you can do more than a requirement, then the requirement is the minimum.

It is also worth remembering that guidance documents contain nonbinding recommendations and thus are, in theory, not requirements or regulations. This distinction, if there really is one, arises from the fact that regulations come about by a formal process that ends up with the regulation being in the Code of Federal Regulations. Therefore, FDA cannot create new requirements via a guidance document, no matter how much something in a guidance document may look both new and required. A manufacturer is in principle allowed to not follow the recommendations of a guidance document. Whether this is ever fruitful, I do not know, and transparency has not led to clarity on the degree to which manufacturers are successful in their choices to not follow a guidance document.

There is another interpretation of how a guidance document can have things that approach a requirement and appear to be new. This is that the guidance document is FDA’s acknowledgment that it wasn’t doing a very good job before, but it is going to do a better job now. Thus the requirement isn’t new; only the enforcement is new. Perhaps this is like the highway patrol doing away with the proverbial 10 mph over “rule” and more strictly enforcing the speed limit. It is not something new that is being enforced; they are just increasing enforcement of the old standard.

Thus the legacy loophole is twofold. One part is that devices already in the market can continue to be used and sold even though FDA thinks they are error prone and they will not necessarily have had an FDA-recommended human factors evaluation. The second part is that even new devices can escape this evaluation if the usability is not changed compared with a predicate. Over time, better designs may come to replace currently error prone designs if you believe the new FDA recommendations will actually make a difference.

Author’s note: Both of the cited documents use the term “use error.” This distinction from “user error” first was seen in the pages of MD+DI in a 1995 commentary I wrote, titled “The Issue is ‘Use’ Not ‘User’ Error.” This is meant to capture that “user error” already seems to assign blame while “use error” just describes what happened, leaving open blame determination. Since the human factors issue here addresses the role of design in error, this distinction is more than something only an academic would love.

William A. Hyman is a professor emeritus in the department of biomedical engineering at Texas A&M University and adjunct professor of biomedical engineering at the Cooper Union. Reach him at [email protected].

  [image courtesy of MASTER ISOLATED IMAGES/FREEDIGITALPHOTOS.NET]