Design Verification vs. Design Validation: Right Questions or Right Answers?

When it comes to verification and validation, medical device companies need to ensure that what they're doing actually makes sense.

August 21, 2017

8 Min Read
Design Verification vs. Design Validation: Right Questions or Right Answers?

When it comes to verification and validation, medical device companies need to ensure that what they're doing actually makes sense. 

Jon Speer

How confident is your medical device company when you reach the design verification and design validation stages?

Known colloquially as "V&V," for many it feels like you're on the homeward stretch to market, yet there are often issues causing companies to get stuck.

For example, do you want to ensure that you have the "right" answers? Or are you more interested in asking the right questions? The distinction can be very important and is a big part of what should inform your V&V processes.

I had a chat with Mike Drues, president of Vascular Sciences and a long-time consultant for medical device companies, FDA, and Health Canada. We discussed our thoughts on V&V and what companies can do to use processes that make sense.

Right questions or right answers?

Design Verification and Design Validation

We both often get asked about V&V and the difference between verification and validation. These are very much part of design controls and are distinct from one another while being applicable across different scenarios. In fact, when it comes to preparing a 510(k), you'll quickly realize their importance.

Verification is proof that you designed the product correctly. Do your outputs meet your inputs? Have your inputs been written well and have you recorded sufficient outputs? The overall question is "did I design this device correctly?"

Validation is subtly different. The overriding question is "did I design the right device?" This is, of course, a case of asking the "right questions" rather than looking for the right answers. What are the user needs and intended uses? Is this the device to meet those needs? Validation happens fairly late in your product development process, yet it's a reflection of how well you did the early work--did you clearly define user needs?

The overall purpose of V&V is to demonstrate that the device total outputs (design, manufacturing, software, etc.) meet what you wanted it to do. Fundamentally, the definitions of verification and validation will remain the same in different contexts for your medical device.

A Common Question

Drues frequently receives questions regarding change management within companies. If they have a device that's already on the market and through 510(k), manufacturers are often confused by the requirements for making changes.

"If I want to make a change to my medical device, how do I know if I need to go to FDA with either a special 510(k) or PMA?"

This is usually versus a question of knowing when a change is not considered significant and when it is managed by filing a letter internally.

This generates a lot of problems for medical device companies and comes down to V&V. You must demonstrate that the change doesn't affect the safety and efficacy of the device. You need to test and research--do the work to prove it. This is the essence of V&V.

At the same time, I prefer that the regulatory requirements be factored in as part of the decision-making. That doesn't mean doing things just because the regulations say so, but doing things that make prudent, engineering sense, too. The end goal is to demonstrate that your device is safe and effective. Companies get caught up trying to please regulations, that "right answers" bit, then sometimes find themselves doing things that don't make a whole lot of sense just to please a checkbox on a form.

Drues advocates that you do what makes sense for your device, then bring in the regulations. He had an example where someone was designing some complex testing and wanted some help but couldn't answer as to why they were doing it. You have to know why you're doing the test to begin with! It's not a "going through the motions to meet V&V" thing.

Change Happens

A lot of companies get products through submission, launch into production, and assume they won't need to make a change. This is naive. It could be as simple as a logo change for your company or as complex as a material change in the construction of the device. Understand the impact of the change. Conduct those V&V activities and consider the regulatory impact without blindly doing things that don't make sense. It shouldn't simply revolve around "I need to do X, Y, and Z for verification and validation in order to put in my submission."

An ISO auditor that I've met has said, "We need to do a better job of root causes." He recommended the "Five Whys," which can also be useful for V&V. If your root cause is "to satisfy a regulatory requirement", you need to rethink your approach.

Drues finds that when he asks why companies are doing things a certain way, he often gets answers like "I don't know" or, worse, "because it's required." If the test provides no other value and is only happening because it's "required," there's no point. He suggests going to FDA separately and explaining why it's not practicable in the case of the company and what would be useful instead. FDA is open to reason where a testing requirement may not be suitable for you.

Always Seek Feedback

We both are huge advocates for companies getting feedback early in the process through the pre-submission program, before you put together that official submission.

Communicate frequently with FDA or relevant regulatory agencies. Drues says "tell, don't ask," and  "lead, don't follow," when it comes to dealing with FDA. It's a case of knowing the right questions to ask.

Don't go in and ask, "What do I do?" First, it's your job to know what to do. Second, you might be opening up Pandora's box with the regulatory authority. They can now instruct you to do whatever they ask. Tell them what makes sense and why. Reviewers will ask questions if they have them. Expect that if they have to provide you with "right answers" or guidance, they will be looking for those in a formal submission and they had better be impeccable.

One of the few requirements of the pre-submission request is that you must submit your questions to FDA in advance in order to get a meeting. To Drues, this violates his rules of not asking FDA questions first. In order to successfully get your meeting, you must ask those questions, though. His advice is that when you craft those questions, you take a "design" approach. Create leading questions so that the only answer they can come to is the one you want them to. Ask the question in such a way that gives you the answer that you want.

As Stephen Covey wrote in The Seven Habits of Highly Successful People, "Begin with the end in mind." Work out where you want to be and move backward from there.

Testing and Proof

Many companies tend to obsess over getting studies done. Don't get so hung up on clinical studies. You have to have good engineering in the first place. There's a place for benchtop, animal, and clinical testing, but no kind of test will be perfect. You'll have limitations every time. Human factors, usability, etc. will always come into it.

The most important thing, I think, is to realize that verification doesn't always specifically mean a test. Engineers always want to use testing methods to prove things, but verification can be simpler; for example, through analysis or inspection. Prove that outputs mean inputs; that is the overriding requirement. Use testing where it makes sense to do so.

One of the most common questions Drues hears is, "If we came to FDA with our new device, what will they want to see in terms of safety, efficacy, and performance?"

Sure, that's an important question and we understand why you're asking, but look from a different perspective. One day it could be your own family member or friend who needs to use this device. As their concerned loved-one, what do you expect to see in terms of safety and efficacy? What would put your own stamp of endorsement on it? When you can show this (and only then) is when you should go and have a discussion with the regulatory authority. "We designed the device perfectly, yet the patient died anyway" is a scenario you hope to avoid. The solution to mitigate this is to get people to think of the device being used on their family.

Right Answers or Right Questions?

As you can gather from our discussion, we lean the way of crafting and seeking the questions that are going to form a solid foundation for your verification and validation activities.

Sometimes, looking for "answers" will leave you constrained to being told what to do and then monitored to ensure you did it.

Keep what makes sense for your device at the forefront and, if you are in a pre-submission situation, formulate questions so that they point to the answers that you know will be the best scenario for you. This is with safety and efficacy at heart, of course. Are you happy for your device to be used by your own family?

Jon Speer is cofounder and vice president of QA/RA at greenlight.guru

[image courtesy of QIMONO/PIXABAY.COM]

Sign up for the QMED & MD+DI Daily newsletter.

You May Also Like