One of the most popular activities at the HealthPack conference is the Nurses Panel, where real-life operating room (OR) nurses critique medical device packages and then answer general and specific questions from an audience of medical device manufacturers (MDMs).

This incredibly useful “Voice of the Customer” feedback tells MDMs and packaging suppliers which packaging designs work, and which don’t — and why. It’s insight like this that will help advance innovation in medical device packaging.

Usability testing is also now a requirement in the 2019 update of the ISO 11607 standard “Packaging for terminally sterilized medical devices.”

Over the years, I’ve heard many medical packaging professionals say they wish they could regularly get constructive criticism like this, but without taking the time, effort, and cost of organizing and managing a Nurses Panel themselves.

Well, now you can. A new service from Packaging Compliance Labs (PCL) lets you gain insight into the usability of your packaging designs. PCL has set up a mock operating theater in its 20,000-square-foot lab in Grand Rapids, MI. Within this theater, which is about the size of a large conference room, you can test sterile barrier systems (SBS) for medical devices and gain both quantitative and qualitative data about your packages. This feedback can help you create better packages now and in the future.

Many MDMs are realizing the value of, and the urgent need for, more innovation in medical device packaging. Let’s see if packaging department budgets will support this.

The benefits of real-life experience.

We initially heard from PCL that this service was in development at HealthPack 2020, right before the COVID-19 lockdowns. PCL has now formally opened its mock operating theater.

Ryan Erickson, PCL’s Co-Founder and Vice President, Package Engineering, says, “There’s a real appetite in the market for that direct link to the end user. By creating a program by which we can make those connections possible … that’s what we’re excited about being able to do and why we’ve invested in developing this mock operating theater space within our lab.”

The ISO 11607 (2019) standard has clear expectations that usability evaluations should be performed by real users. That’s one of two options the PCL mock operating room offers. Clients can have their packages tested (1) by actual OR nurses or (2) by trained PCL lab technicians. Pricing varies depending on the path chosen.

Erickson explains, “We recognize there are a lot of customers that already have products on the market and may not [need] the benefits of a full-blown clinical user evaluation when they’re getting that through their normal sales and marketing channels already.”

But using real nurses might be worth it, especially when you consider the unexpected benefit the depth of their experience brings beyond packaging usability.

Erickson explains that when PCL did the pilot study, they were surprised that, beyond the insight into the packaging, the end users had good feedback into product development, too. For example, one client learned that improving how its device coupled to a widely used and popular component might mean their device would be selected more often.

Additionally, the desire for making a connection with these healthcare professionals is not just one-sided. PCL has special relationships with several large hospitals in the Grand Rapids area. Through this project, they have found a pool of well-experienced hospital staff who appreciate the opportunity to give their wisdom and experience back to industry.

Theater represents actual environments.

In addition to working with trained lab personnel and healthcare professionals, PCL has set up the environment to reflect real-world conditions, whatever those may be.

For example, in one case, Erickson says, “To make it a more authentic experience, it was important that we turn the lighting levels down to be representative of the real-world experience.”

Low light could make it challenging to see or follow opening instructions on a package. But you might not realize that if you were testing the package in a well-lit space.

Sounds matter, too. “There might be some surgery or case work where you could have other machines and loud noises that [could cause] distraction,” Erickson says. “And that may be important to represent to get the appropriate and authentic feedback.”

PCL uses a clinical consultant to identify the appropriate users required for each project, depending on the product. You might need a perioperative nurse or a circulating nurse. The advisor can help PCL understand the qualifications and credentials needed before having the actual people come in to participate in the testing.

This set up and leg work on the front end to replicate the actual environment and “staff it” with the right healthcare workers is critical to the success of this service, according to Erickson.

The area is also furnished with the patient bed, back tables, cabinets, and overhead lights on wheels — normal things in an operating room that people need to move around during the performance of their jobs.

Product inside? Simulation to supplement actual data?

One question that comes up regularly for MDMs and medical packaging suppliers as it relates to validation tests, and now usability tests, is … Is it necessary to test packages with the product inside, or can you test without it? How does that impact the accuracy or results of the test?

Erickson answers, “Where possible, it’s always best to include actual or representative product when conducting any sort of packaging qualification work.”

But he does admit that some features of the qualification process may allow more flexibility by being able to use an empty package for some specific purposes. If including the product is an issue because of cost or time, “It’s better to do something than to do nothing,” he says.

Along the lines of doing something … with the newer computer technologies such as augmented/virtual/mixed reality, what is the value or differentiation in doing medical package usability testing “live” vs. doing it virtually? Are there cost benefits doing it virtually? Obviously, COVID-19 social distancing and safety at work might be an advantage. But if you really, really have to be sure of actual opening, virtual probably wouldn’t be the best option.

“It’s possible that there could be a way that that solves a certain challenge or meets a certain type of need,” Erickson says. “We’re open minded and would definitely be interested to see if things come up along the way that look like they could enhance the program or the customer experience. We would be definitely interested in taking a look.”

For now — until augmented/virtual/mixed reality is allowed in a future version of 11607 — MDMs and their packaging suppliers might use simulation as a design tool to narrow down which physical packages to test in person.

Formal, consistent evaluation is key.

To ensure they get accurate and consistent results during the testing, PCL developed an evaluation form that captures quantitative and qualitative feedback — and maintains a consistent process for the packaging review.

“Anytime you’re trying to capture end-user feedback, it’s important that you’re sensitive and careful about setting rules around how you, as a moderator, are going to engage with these users — to the sort of prompts and questions [you use], and how you solicit their feedback without asking leading questions or somehow creating bias in the way that you’re having that dialogue,” Erickson explains.

You’ve got to know what to ask, and how to ask it. And then do that the same way each time. PCL followed best practices in the research industry to create the form and process.

It’s most important to understand the specific questions you need feedback on and then score it in a way that quantifies it so customers can compare it, apples-to-apples, to other packages that also might be tested.

“We’ve been trying to research and build in best practices to our process and make sure all the questions and the way that we’re going to evaluate results for pass or fail is thought out beforehand,” Erickson says.

Already looking ahead.

PCL’s Grand Rapids location is central in the United States and fairly close to the Minneapolis medtech hub. The success of this venture will guide the company’s leadership team on expansion possibilities, both geographically (perhaps near other medtech hubs on the East and West coasts) or into other applications of medical packaging use (such as in ambulances or long-term care facilities).

Scale-up plans will be guided by PCL’s mission to deliver solutions that enable life-changing medical innovation and make sure the company can execute on that competently with respect to sterile packaging, according to Erickson.

“The next 12 to 18 months are going to be really important for us to capture feedback from our customers, from the market, as this program is getting rolled out and being scaled up,” Erickson says. “We’re making sure that any decisions we make here in the short term are going to keep us nimble so we can react to whatever the response is. We are prepared that one type of response we might get asked to meet is to grow the variety and the depth and the range of the whole concept of human factors — and be a provider that can support industry in a larger way than just sterile packaging, because those needs are definitely present and other aspects of the industry.”