Are You In Control of Your Software Analysis?

FROM THE EDITORS

After quietly implementing a new method for analyzing software, FDA's Software Forensic Laboratory was able to find some problems in a couple of high-profile devices that were recalled for adverse events. Called static code analysis, the technique is commonly used in the automotive and aeronautical industries and has been around for about 10 years. Simply put, it reads the lines of code without running the program. It is designed to check for errors that might keep the code from running correctly. It has been used extensively in aeronautics, and it is seen as essential in that safety-critical industry. Sounds ideal for the medical industry, right? Or, maybe not. It is costly and labor intensive—both of which can be impediments for the medical device industry.

FDA consulted with other federal agencies that deal with software integrity issues—including the Department of Defense, FBI, National Institute of Standards and Technology, and NASA—before it started using this technique, which was outlined in a single paragraph in the 2006 CDRH annual report. The agency determined that such analysis would be particularly valuable in understanding the root causes of adverse events due to software failures. Whether this a good investment of FDA's regulatory enforcement budget remains to be seen.

Often, the result of this analysis is an itemized list of coding errors that is sent back to the manufacturers. The list can contain any error, from minor misspellings or typos to major errors. But the list does not prioritize the code errors, and OEMs are expected to respond to all items.

FDA says it is currently reviewing only software that presents “an imminent public health threat.” According to MedSun, the FDA laboratory has used the new method with success to analyze software in several high-profile compliance cases. It is clear that the agency is expecting to find errors in the software that the manufacturers themselves have overlooked.

However, FDA has also said that it expects manufacturers to “increasingly” use such tools during their product development process. FDA generally avoids prescribing specific methods to industry. Does this expectation mean that FDA is advising manufacturers on how to design their software?

Some designers are already using this technique in the development of their software. Full Spectrum Software (FSS; Framingham, MA) uses both static and runtime analysis. “This is a part of our quality system, and the results are clearly and demonstrably beneficial,” says Andrew Dallas, president of FSS. “We provide a service called product hardening, which allows us to improve overall product stability and reliability.” The components include not only static and runtime analysis, but also manual code review.”

As a design tool, it is important to recognize that static analysis does not just find insignificant errors. These processes actually find coding errors created by the software engineers, explains Dallas. “The types of errors we can find are much more profound than spelling errors,” he says. “Consider a product that is designed to run for 12 hours and is acquiring patient information during that time. If a programmer has failed to initialize a variable, the value within that variable can be any value.”

He says that a major error can occur if the code uses that variable to make a decision: “If x is zero, shut down.” Static and runtime analyses can find such errors as well as others that are potentially even more serious. “Really the only downsides of the hardening process are cost and time,” he says. “These downsides, however, are quickly recouped in reduced maintenance costs.”

Static analysis can be a good tool when used at the design stage. And if you do, it may keep you from ending up in FDA's forensic laboratory after an adverse event. FDA expects manufacturers to adopt this method of analysis, but it has not specified a timeline.

Perhaps FDA's decision to do such analysis will also have some unintended consequences. It would be a tragedy if manufacturers cut back on validation activity in favor of waiting to simply respond to FDA's itemized list of coding errors.

Sherrie Conroy for the Editors