User Research Must Be Minimally Invasive, Too

PRODUCT DEVELOPMENT INSIGHT

Healthcare professionals, patients, and other medical product consumers benefit from the maturing user-research practices in medical product development. Ethnographic research and human factors are forefront in the minds and the project plans of the medical product development community. Progress should continue with the forthcoming publication of AAMI's new standard, HE-75, Human Factors in Medical Device Design. HE-75 will be an updated set of guidelines and recommendations for designing ergonomic, usable medical devices; it is slated for release in late 2008.

But success, in this case, is leading to new challenges. User research has seen a transition from the question of whether to perform such research, to how to do it more effectively. For example, field observations typically result in copious notes, transcriptions, photos, and videos. As a result, data collection is a lesser challenge compared with organization, analysis, and communication of actionable findings. In other words, effectively gathering user data has created a secondary research problem—understanding the meaning and value of such information.

In more familiar terms, user research now runs the risk of becoming invasive to the medical product development process. Traditional resistance to user research included the argument that it added extra time and cost to already-tight development timelines and budgets. However, by most measures, its effect on timelines and budgets is relatively small in the context of engineering or software development. More often than not, user research provides valuable cost savings or revenue-producing insights. However, now that user research is an integral (rather than ad hoc) activity for many successful medical product teams, it has the potential to become unwieldy (e.g., by generating too much nonactionable data) if not managed appropriately.

At the same time, as field research becomes more common, product development teams spend more time observing and interviewing end-users. Consequently, frequently visited customers (e.g., lead users) may start to find user research invasive in the literal sense, as it adds interruptions and administrative overhead to their professional or personal lives.

In both senses, invasiveness is a potential side effect of successful user-research programs. As such, research teams must perform efficiently and unobtrusively. Fortunately, in research, as in medicine, thoughtfully chosen and applied technology can reduce invasiveness while improving the quality of outcomes for product developers and end-users alike. Although no technology is a substitute for skill and expertise, new tools can improve the speed and benefits of user research while reducing invasiveness to all involved.

Some technologies presented in this article represent potential solutions to a range of challenges in user-research data capture, analysis, and presentation. Although several products are discussed, the goal here is to increase awareness of the growing field of user-research technologies, rather than endorse particular products. In fact, as some of the examples suggest, researchers should be hesitant about adapting new technologies too quickly. Reliability and simplicity are the most important attributes for research tools to ensure that they don't interfere with or impede the observation process.

Audio Recording and Video Recording Devices

Anyone who has conducted user research, whether observing a medical procedure, interviewing a patient, or testing a prototype, knows that multitasking is essential for capturing data. The ability to see and hear what is occurring while simultaneously writing down key information and insights is a challenging skill to develop. Audio and video recording can back up the researcher, but handwritten notes are the primary means of documenting and reviewing data. At the same time, notes tend to be succinct and coarse, conveying the concept but not the verbatim of an event.

The Livescribe Pulse is a pen-based computer that offers the ability to narrow the gap between what is observed and what is written. The pen records ambient audio and synchronizes it with what the observer writes. Subsequently, a researcher can tap on anything written, and play back audio that corresponds to what occurred. For user interviews, the researcher can quickly reference notes directly to the audio of interviewee's words for clarity and idea expansion. For ethnographic observations, both conversations and environmental sounds can be unobtrusively recorded (in stereo) during note taking. Although the device does not provide the highest-resolution audio quality, it increases the working bandwidth and accuracy of the design researcher.

Recommendation: For the interviewee, the combination of a pen and audio recorder in a single, synchronized package reduces both the perceived and physical presence of potentially obtrusive technology in the research context. For the researcher, synchronization speeds up data retrieval and provides veridical evidence to support conclusions.

High-Definition Video

High-definition (HD) video recorders are rising in popularity and falling in price. Consequently, many design researchers are considering such products' applicability to field observation. Greater visual clarity can be useful for studying fine motor control tasks, small control and interface element usage, and visually rich environments. But for most situations, HD's higher resolution is only incrementally valuable—the added visual resolution is nice, but not necessary.

Although higher visual resolution may seem like the way to go, there are definite drawbacks, as follows:

Recommendation: With all of these caveats, high definition may not be a worthwhile short-term upgrade for most researchers (especially the additional file management time). Although the consumer electronics world is going high definition, there are pragmatic reasons to stay behind the curve. If making a new purchase, it makes sense to go with HD, but don't assume that everything needs to be recorded at the highest resolution.

High-Speed Photography and Video

The speed at which an activity occurs is often a challenge to understanding rapid, complex maneuvers, such as emergency surgical procedures. Improving the design of a surgical instrument often depends on understanding its use under extreme circumstances. But even with video recording, the manual dexterity of a skilled professional may be impossible to clearly observe. High-speed photography and video, which can capture dozens or even hundreds of images per second, help slow down time to view rapid and subtle movements that are not clearly visible or differentiated with standard video and photographic technologies. High-speed photography has been around for many years, and is now available in digital formats. Adoption, however, has been slow because of its prohibitive expense and the high degree of technical skill required for use.

To address those barriers, Casio has launched the Exilim Pro EX-F1, a consumer-targeted digital camera with high-speed capabilities. The camera can capture photographs at up to 60 shots per second and can take video at up to 1200 fields per second. It also incorporates a novel user-interaction function—a buffer to prerecord images when the shutter is only halfway depressed, allowing room for timing errors when capturing a quick event. Extending visual perception to micro­seconds can reveal informative sub­patterns of movement that are overlooked or invisible at a standard timescale. Moreover, it introduces a new perspective on observing physical behavior that expands the potential for identifying innovative product design opportunities.

Casio is the first to roll out this technology commercially, but expect other competitive brands to deliver similar features in the near term. Note that high-speed video also requires significantly large file storage and power needs, which will have to be accommodated in preparation and analysis.

Recommendation: High-speed photography and video open a new aspect of analysis in task flow and ergonomics. In the field, they are valuable for capturing more granular data without making additional demands on the research participant, but there is a back-end trade-off. A high-speed camera will capture significantly more data and require associated expertise to analyze and interpret.

Anthropometric Databases

The majority of reference anthropometric data used to guide design is based on the body dimensions of European and North American populations. As such, it provides limited applicability and fit to broader user populations. The availability of three-dimensional scanning technology, although time-consuming and expensive, is driving the inclusion of additional populations.

For example, Size China is a program to create the first-ever digital database of Chinese head and face shapes for items such as protective goggles and surgical masks, and potentially measurements for surgical procedures. It was inspired by the lack of a comprehensive anthropometric database of Asian head and facial features. Derived from digital scans of more than 2000 Chinese citizens, the project has the potential to go beyond what is available for Caucasian populations.

Recommendation: Global, up-to-date anthropometric resources provide an essential starting point for guiding form and size in medical product designs. Of course, they are not a replacement for subsequent fit testing with real participants. Access to such data sets will reduce the time and gap between initial product designs and acceptable user fit.

Data Visualization Tools

After research data are gathered, the team needs to make sense of them. A common impediment for medical design is organizing qualitative data into meaningful information. Identifying and prioritizing observations, user input, and other raw content depends on an effective process. The right tools can streamline these efforts.

Often the biggest challenge is knowing where to start. For example, consider trying to make sense of the transcriptions from several dozen interviews with consumers of a home-healthcare product. There is likely to be a mix of positive and negative feedback, anecdotes, opinions, and narratives.

You could manually sift through all of the conversations, highlighting meaningful terms or search for keywords. Or you could leverage software that provides structure to text-based data to help guide the analysis.

One such tool is IBM's Many Eyes, a Web-based application that provides a suite of data visualization tools. Although primarily geared toward quantitative data charting, it also provides text-based visualization tools including Word Tree. Word Tree lets a user choose a word or phrase and shows all the different contexts in which it appears. The contexts are arranged in a treelike branching structure to reveal recurrent themes and phrases.

When applied to qualitative data, Word Tree allows a researcher to quickly scan through text-based content by searching via keyword or phrase. For example, to see what the home-health consumers said about aspects of the product, the researcher can create a model around keywords and phrases of interest such as specific features (e.g., entering medications, dispensing) or user reactions (e.g., difficult to, problem with). Visualizing the content around critical terms provides a starting point for reviewing and understanding qualitative data in an efficient manner. It is not a substitute for thoughtful analysis.

Recommendation: Data visualization has been promoted as an effective means to present data, but its enormous value in analyzing data has been largely overlooked. Visualization can streamline the translation of user research to the product development teams. One significant caveat in the case of Many Eyes is that all data submitted are available to the public, so it's not necessarily suitable for proprietary data analysis.

Data Management Tools

On a broader scope, another challenge in design research is managing data across a project, or across multiple projects, and then sharing that information across the organization. Besides the short-term value of improving communication on a particular project, an organized user-research database can grow into a valuable corporate asset.

Collecting and organizing research across projects is rarely done efficiently, if at all. Consequently, there is a need for techniques and tools that support better research data management and communication. New specialized software tools are providing a platform for accomplishing this. For example, QSR International recently launched NVivo 8. It provides a framework for entering, tagging, and querying various forms of qualitative data (including audio and video) across multiple projects. HTML output can be produced to readily communicate and present research findings.

These types of tools enable more-effective collaboration among both localized and geographically distributed researchers and can provide a centralized repository for observational data.

Recommendation: The value of well-conducted research is extremely limited if it is not organized for effective communication. Developing a system for storing and sharing research data can have a greater organizational benefit than any other research-related technology.

Getting the Most from User-Research Tools

As user research takes on a greater role in device design and development, teams will experience an embarrassment of riches (i.e., more data than they know what to do with). This can lead to a negative effect, as the researchers struggle to manage data and to not inconvenience participants. Experienced research leadership and processes can guide teams from data to actionable results and, ultimately, innovative product ideas. Appropriately applied technology can play a key part in minimizing the invasiveness of user research by providing tools for collecting information quickly and accurately, as well as by providing a framework to analyze and share results.

Rob Tannen is director of research at Bresslergroup Inc. ( Philadelphia). He can be contacted at [email protected].

Copyright ©2008 Medical Device & Diagnostic Industry