The Moral of This Story? Use Test Method Validation

An expert illustrates why test method validation is important in the real world.

José Ignacio Mora

Tom was ready to send the multi-cavity fixtures. I asked him to perform a Gage Repeatability and Reproducibility (R&R)—one more time—before they were to be sent to Europe where they would serve as online inspection fixtures for the new multi-cavity mold. It passed with flying colors!

A few weeks later, the bad news came from John in the United Kingdom. They repeated the same Gage R&R study, on the same fixtures, and they failed.

How could that be? Everything had been done to the letter. Design controls, studies, etc.

The easy answer was that “someone didn’t do the test right” or “the samples were not properly prepared.”

As the supplier, Tom agreed to honor the guarantee of his work, and accepted the gages back from the United Kingdom. When they arrived at his lab in the United States, he repeated the Gage R&R. Again, it passed with flying colors!

The more difficult answer was that everyone, in fact, did everything right. Yet the test failed in the United Kingdom—and passed again in the United States!

Needless to say, both parties began to distrust each other’s work. Solving a problem, especially when two different parties are involved, can be stressful.

So, what else could explain the discrepancy? As it turns out, there were three factors involved:

But it is even more difficult to step back and see the bigger picture.

What were we trying to do? What were these fixtures for?

They were intended to measure part-to-part shot-to-shot variation—for parts coming out of the same mold. In other words, the same parts!

What happened here is that we asked the gage to do what it wasn’t designed to do—to measure parts that were the output of a mold before and then after a tooling modification.

In other words, this was no longer simple part-to-part shot-to-shot variation. When the mold cavity itself was dimensionally modified, this resulted in a different part as far as the gage was concerned. The very nest of the part had been compromised.

John used parts coming from his modified mold. Tom used parts from a prior mold version.

In other words, same gages, same test method, but parts from different origins.

Test method validation isn’t just about performing Gage R&Rs. It is about knowing the requirements, applying design controls, verifying the requirements, and validating that the test method actually performs as intended.

And it isn’t just about following regulations. One must truly understand the process, the nature and origin of the materials, the environment, and also the design itself.

And that is why we validate.

This was a true story. In fact, it will be one of the case studies I plan to present during the MD&M East conference.

José Ignacio Mora is an instructor in Lean Quality System, Process Validation, and Lean Manufacturing at Atzari Enterprises, LLC.