Your AI Initiative is Likely to Fail

Imagine regaining consciousness just as you’re being wheeled into the ER with excruciating pain radiating from your chest. The doctor tells you that, although your ailment isn’t life-threatening, she believes she knows what’s wrong and can address it with a very expensive procedure. Although you’re semi-lucid, you think to ask a few questions:

“Just how expensive are we talking?” 

The doctor states that the surgery will be extremely expensive, then whispers in your ear that more than half the time the total cost is over 189% of the original estimate. “How can you put a price on your health,” you ask yourself rhetorically. 

“How long will the procedure take?”  

The duration is much longer than you expected, and you’re told that 40-50% of the time it takes even longer. “Fine; better to be thorough,” you think.

Most importantly, “What’s your success rate?” 

After more than a moment of awkward silence, you hear…14%. 

Do you go forward with the procedure? As an executive tasked with green lighting any software development project, this is your dilemma as these numbers represent the success (or failure) rates for such undertakings.[1] If you add an inconclusive diagnosis to this scenario to make it even more complex, you’ve got an AI initiative. 

Developing any type of software is an extremely complicated endeavor, and AI in a regulated environment takes it to another level. If they’re being honest, many executives don’t even know what AI is or why they (or their customers) need it. Like digital transformation and big data, AI has become the buzzword du jour.  

With this in mind, let’s demystify some of the AI complexity by starting with a few high-level definitions. Then, we’ll highlight the benefits, and ultimately discuss three key factors that must be considered before embarking on the AI path.

Neural Network

In biology, human brain nerve cells (neurons) form a complex, interconnected network, sending and receiving electrical signals to process information.

Artificial intelligence (AI)

A branch of computer science in which systems are developed that can simulate intelligent human behaviors such as learning, reasoning, and problem solving. The core elements of an AI system are:

Machine learning (ML)

A subset of AI in which systems can learn and evolve over time, albeit with human guidance/retraining, by:

Deep learning (DL)

A subset of ML in which algorithms:

Clinical accuracy

A healthcare provider who can derive actionable information from massive data sets can make better-informed decisions during case planning and in real-time. For example, by leveraging an AI solution, a surgeon can better optimize the margins needed for tissue removal, effectively reducing or eliminating collateral damage to healthy anatomy.

Patient outcomes

In the case of diagnostic imaging, AI’s ability to analyze copious amounts of data with great speed and precision can identify subtle patterns that wouldn’t be evident otherwise. A 2020 study[1] illustrates striking differences between the accuracy rate of tumor classification by radiologists as compared with an AI-enabled system, depending on the type of tumor. 

Adding phenotype data to the equation — a manifestation of a patient’s genetics combined with environmental data — enables predictive analytics which can yield a better diagnosis of the severity of a health issue, at the time of evaluation and in the future. 

Economic value

With value-based care gaining momentum, compensation to healthcare organizations is starting to be determined by patient outcomes rather than the amount of care provided. As such, these organizations are scrutinizing every aspect of their operations to improve their financial health, while concurrently striving to provide the best possible patient care. Higher clinical accuracy and better patient outcomes can prevent unnecessary procedures and reduce readmission rates. The proper application of an AI solution can help streamline workflows, potentially decreasing the staff required to support certain procedures, while increasing the number of cases that can be performed in a day.   

Given these benefits, it’s no surprise that AI applications have gained the most traction in radiology, with cardiology trailing far behind and other areas of medicine “in the dust” (for now). 

Data Source: The Food and Drug Administration | Chart Created by MedAcuity 

Given all the latent potential of AI in healthcare, what’s stopping you from diving right in? In our experience, there are three primary considerations that can impact the success or failure of an AI initiative, each of which is interrelated.  

1. Solve a problem

A solution requires a problem to be solved, which remains a best practice for software development. A market-driven approach, therefore, is the right way to approach any software development project, especially a complex one involving AI. 

Define: Start by identifying an unmet need, which demonstrates an understanding of the market and provides a foundation on which a solution can be built. The benefits of some AI applications are evident, although not all use cases are good candidates for a variety of clinical, practical, or business reasons. Be sure that you can answer these questions:

The answers to these first two questions represent your initial set of requirements to describe what the market needs and serve as inputs to the architecture.

Postulate: Once you’re confident in your answers, generate a hypothesis. Then, create a system architecture and a design based on the requirements as well as a set of well-defined benchmarks and expected outcomes, noting the benefits for each use case. 

Iterate: Develop a proof-of-concept system and test your hypothesis extensively with an appropriate set of participants. Then, “rinse, repeat” making changes based on feedback, much like a formative study, to hone and validate the efficacy of your solution. The following excerpt from the 2020 study referenced earlier illustrates the complexity of this process for an AI initiative:   

When applied properly, a market-driven approach can prevent organizations from making massive investments in solutions before the right problems are identified and the viability of an AI solution is confirmed.

2. Build on a solid foundation

Looking back on the scenario that opened this article, I presume that most people would opt to receive modern anesthesia over biting on a stick or taking a shot of whiskey to control the pain during such an invasive procedure. Of course, modern pain management techniques and equipment have evolved exponentially in effectiveness and complexity, compared with those used in the 1800s. Similarly, one could argue that the evolution of AI technologies represents a quantum leap in the complexity of medical devices, as the amount of planning and the volume of choices is orders-of-magnitude greater than those required for traditional “unintelligent” systems. 

While the best practices surrounding the generation of solid requirements apply to all software development initiatives, the effort required to develop a robust architecture is exponentially greater in an AI system. 

Architecture

Because architecture describes what we are building, it is foundational and represents tenets such as reliability, scalability, security, etc. These tenets take on a whole new meaning in AI systems, along with some new ones that require thoughtful consideration:    

Data quality

The output of an AI model is only as good as the training datasets used as inputs. Thus, it is important to define the types of data that will be involved and to create benchmarks against which the accuracy, relevancy, and consistency of the data can be measured. 

Data bias

One of the greatest challenges in confirming the accuracy of data is to ensure that it is not biased due to subconscious preconditioning. If not addressed adequately up front, bias will be perpetuated by the model and expand as the model evolves. In healthcare, this can potentially lead to misdiagnosis, selection of the wrong treatment option, and ultimately, patient harm.   

Model selection

The number of AI model archetypes is vast, with three of the most prominent being supervised (trained by humans), unsupervised (trained by software), and semi-supervised learning (a combination of the two). As such, a decision regarding which (and how many) models to leverage in an AI solution should be based on many factors including the type, volume, and availability of the data, the nature of the algorithms (static vs dynamic), and the complexity of the problem being solved.   

Performance

Given their potential for rapid evolution, AI systems must be continuously monitored against performance benchmarks, focusing on the model itself and its impact on other integrated systems.  Without this commitment, a system can be susceptible to many conditions that can negatively impact the accuracy of the learning model, causing it to degrade over time. Two of the most prevalent conditions include:

Coupled with routine monitoring, periodic recalibration can be a useful tool in overcoming these conditions. 

Integration

This concept can be reviewed from multiple angles:

Safety

When a medical device is involved in causing patient harm, the FDA’s first reaction is to look at whether the device was “designed for usability,” rather than assuming user-error. As such, when designing an AI solution for healthcare, it’s important to consider how users will interact with it, and to incorporate guardrails to prevent misuse and ensure that the results produced by an evolving system fall within safe, acceptable ranges. Further, there should be a means of checks and balances to enable a human user to validate the information being presented. Finally, in the event of a problem, AI solutions must have a transparent mechanism for tracing the behaviors and results of the system to facilitate root cause analysis.      

3. Don’t skimp

An AI initiative is neither simple or easy. Shortcuts will result in sunk costs, therefore, it’s important to make the right investments to provide the greatest chance of achieving a successful outcome.     

People

AI initiatives require a different way of thinking and people who think differently. While engineers with deep AI experience are in short supply due to the nascent state of the technology, there is an increased focus on education in this space, and therefore much more knowledge is coming to the market.  While these roles are critical for building, monitoring, and maintaining the system, the contributions from a cross-functional team consisting of data scientists and domain experts are just as impactful in AI solution development, especially in the definition step of the problem-solving process. 

Time

As described in the Iterate step, there is a myriad of options and decisions to be considered, and they are all interrelated. The problem-solving process and development of the architecture cannot be rushed, as short-changing either of these activities up-front will significantly increase the risk of a failed initiative. 

Budget

Complexity adds cost. More than half of software development projects exceed their initial budget by 189% or more, and AI adds even more complexity and risk. Once your team provides a budget estimate, tack on an additional 50% (which still may be low) to give you a better projection of what the initiative will realistically cost. If that number is completely untenable, you may want to reconsider the ROI potential of your AI initiative.   

AI has the potential to transform many aspects of healthcare in a positive way, but without the proper forethought or guardrails in place, the results could be tragic. Most people wouldn’t gamble with their lives on a medical procedure with only a 14% success rate, however, taking a hasty approach with an AI initiative has the potential to yield a similar likelihood of failure. In contrast, making the right investments and following best practices related to problem-solving and architecture creation reduce the likelihood of your project flat-lining.    

Author:

Jim Roman, VP of business development at MedAcuity

For over a decade with MedAcuity, an engineering firm specializing in medtech software development, Roman has dedicated himself to working with clients to clear the most challenging hurdles and to accelerate the right products to market.

Additional Contributions from:

Simon Johnson, Sr. vice president of sales & marketing at MedAcuity

Johnson leads the go-to-market strategy, sales, and marketing for MedAcuity.

[1] GITNUX Market Data: “The Most Surprising Software Project Failure Statistics and Trends in 2023.”

[2] 3D deep learning-based classification of pulmonary ground glass opacity nodules with automatic segmentation, Wang et al., Journal of Computerized Medical Imaging and Graphics 2020