The Future Arrives for Medical Displays

An MD&DI January 1997 Feature Article

Back when color television was still a relatively new technology,Canadian educator Marshall McLuhan said that the medium is the message. By this,he meant that the physical attributes of a display medium such as print ortelevision will affect the message it carries; for example, the same movieviewed on a "cool" TV in an empty living room will have a differentmeaning than when viewed on a "hot" big screen in a crowded theater.

Now, some 30 years later, the idea has taken on significance for the medicaldevice industry as display manufacturers find revolutionary new ways to presentcritical information to health-care providers. Flat-panel displays, holograms,and head-mounted displays are dramatically changing the way medical messages arecommunicated.

New display technologies are becoming part of all types of medical devices, fromsolid-state thermometers to bedside monitors, and endoscopes to magneticresonance (MR) scanners. They are being used to show patient and equipmentstatus, as well as patient anatomy in two and three dimensions.

At Tripler Army Medical Center (Honolulu), John Collins, MD, chief of theDepartment of Neurosurgery, studies holograms before and during brain surgery.These holograms are provided by a new display product called the Voxcam (Voxel,Laguna Hills, CA), a laser camera that stacks MR or computed tomography (CT)images on film. The 3-D holograms that form when light shines through the imagesappear suspended in space. "We are able to navigate point bypoint--cross-referencing between a tumor and the hologram--taking a measurementof our depth with the probe and putting the probe in the hologram to the samedepth to see exactly where we are," Collins says. "As more tumor isremoved, we can monitor our depth to see how close we are getting to the backwall."

Vista Medical Technologies, Inc. (Carlsbad, CA), has developed a head-mounteddisplay called CardioView, which provides data about patient vital signs as wellas real-time ultrasound images of the beating heart to cardiologists duringminimally invasive cardiac surgery. The data are presented on two flat-paneldisplays that are positioned one above each eye. "It provides an ergonomicand, more importantly, intuitive way for surgeons to see the digital informationcaptured from the anatomical site," says Nancy Briefs, vice president andgeneral manager of cardiothoracic surgery at Vista.

Miniature cameras sending images to displays and special 3-D glasses are nowenabling physicians to see deep within the patient as they investigate withendoscopes, laparoscopes, and sigmoidoscopes. One of the newest of theseendoscopic imaging systems, developed by International Telepresence (Vancouver),reportedly imitates the process of natural vision. "The vast preponderanceof endoscopy is done in 2-D--a flat image on a monitor," says NigelHorsley, vice president at Telepresence. "This is real 3-D."

Real-time navigation of imaging data on computer screens promises to allowclinicians to tour patient anatomy and make adjustments to provide the bestdiagnostic view. The first application has been to develop a kind of virtualcolonoscopy, which provides images similar to those obtained with anendoscope--but from CT data. At the forefront of that effort is Vital Images(St. Paul, MN), which is developing 3-D workstations in collaboration withradiology departments at Duke University (Chapel Hill, NC) and StanfordUniversity (Palo Alto, CA). "We are convinced that this work will bring 3-Dvisualization to a broad range of clinical applications," says G. AllanJohnson, professor of radiology at Duke.

FLAT-PANEL DISPLAYS

The fundamental building blocks of this revolution in display technology areflat-panel displays (FPDs). By expanding the readability of cathode-ray tubes(CRTs), these technologies promise to have an enormous impact on all types ofcommon health-care products: patient monitors; defibrillators; machines foranesthesia and dialysis; control panels for x-ray, MR, ultrasound, and CTscanners; blood analyzers; and any other medical device that requires anelectronic readout.

According to Albert Smith, vice president at Noritake Co. (Arlington Heights,IL), a developer of flat-panel technology, improved displays are needed inclinical settings today. "When I go into a hospital and see some of thepoor displays, I am shocked, because this is such a critical place," Smithsays. "With medical equipment, you should be able to read the data."

Four major types of FPDs are offering improved capabilities for medical devices.

Vacuum Fluorescent Displays (VFDs). Composed of highly efficientphosphors coated directly onto transparent anodes, VFDs are fashioned intomatrices of widely spaced phosphor dots. The displays offer the advantage ofdrawing very low voltages and can be seen easily in sunlight, but they aretypically limited to applications that require low informational content.

Noritake Co. is one maker of active-matrix VFDs, which are noted for highbrightness, high density, low power consumption, and simple interfaces.Noritake's new technology, which will be on the market this winter, is based ona 16 × 16-phosphor- dot array formed on silicon chips. The company hasstandardized its first line of products for customers needing high-brightnessgraphics or character displays with low voltage and power consumption. Smithsays that the brightness levels of these VFDs are twice those of other types ofdisplays. "These particular devices are small devices, but they can displayanything from graphics to alphanumeric characters," Smith says. "Sothey can be used on any device type, whether blood analyzer or x-rayreadout--wherever digital or small graphical information is needed."

Electroluminescent (EL) Displays. This type of display can be thought ofas a sandwich comprising a layer of light-emitting material nestled between twoinsulating layers, and then placed between two conducting electrodes. A lightemitter or phosphor is placed between the insulating layers. Thin and compact,EL displays provide very fast video rates and exhibit good readability andbrightness.

Planar America (Beaverton, OR) released two new flat-panel products based on ELtechnology in the summer of 1996--a 512 × 256-pixels-per-inch (ppi) display witha 6.3-in. diagonal screen, called the Q2, and a 480 × 240-ppi display with a6.1-in. screen, called the PR2. Both feature a 1-millisecond response time. TheQ2 is optimized for a high resolution of 96 dots per inch (dpi) to address thepatient monitor marketplace, particularly for the display of waveforms such ascardiac cycles. The PR2 is optimized for extended life. The lifespan of ELtechnology commonly ranges between 30,000 and 50,000 hours mean time betweenfailure (MTBF). Tests show that PR2 delivers an unprecedented 100,000-hour MTBF."It doesn't have a backlight to wear out," says Jerry Vieira, directorof strategic marketing at Planar.

Both displays exhibit the characteristic benefits of EL--high brightness,striking contrast, and wide viewing angle. Both are designed to be highlyresistant to shock, vibration, humidity, and temperature changes. "Thatcounts for a lot in ambulances when the temperature's ­40°F inMinnesota or 105°F in Arizona," Vieira says. They also includelow-power modes to address the needs of battery-powered portable applications,making them ideal for ambulatory portable instruments.

Field-Emission Displays (FEDs). Like EL displays, FEDs use rowelectrodes, column electrodes, and phosphors. However, unlike EL displays, FEDsalso have a light emitter made of tungsten, diamond, or silicon and connected tothe row electrode. Field emission occurs when an electrical field is applied tothe tip of the emitter in a vacuum, and electrons tunnel from the tip into thevacuum.

PixTech (Santa Clara, CA) has developed a graphical flat panel based on FEDtechnology, which is sometimes called cold-cathode technology. Two majoradvantages of the product are its 160° viewing angle, both horizontally andvertically, and its range of operating temperatures: ­40° to 85°C.The company is in limited production on the FED product and expects to ramp upto full capacity in 1997. "Our strategic focus is to concentrate on marketssuch as medical devices, particularly patient monitors, that need our uniquefeatures and benefits that can't be provided by other display technologies,"says Greg King, a sales manager at PixTech. "Viewing angle is a criticalissue in a lot of medical applications, because space is generally at a premiumand you don't necessarily have the luxury of being right in front of thedisplay."

Liquid Crystal Displays (LCDs). These types of displays, based on liquidchemicals, share much of a liquid's dependence on temperature, stress,electrical field, and chemical environment. They have the advantage of usingvery little power and can be fabricated into very thin housings. Most LCDs thatare currently in use suffer from slow response time and readability limitations,because of both lighting and angle. But as this technology continues to developrapidly, these problems may soon be solved.

The Palo Alto, CA, firm dpiX is developing active-matrix LCDs based on thin-filmtransistors (TFTs) capable of being fabricated into page-size flat panels withup to 7 million pixels. These TFTs act as switching elements that controlcritical voltage levels affecting the liquid crystal. The switches control veryspecific amounts of light passing through to each pixel. Tight control allowsthe display to be highly responsive to signals while allowing maximum lightoutput. "Each pixel is 90-µm in size and can produce 200 fL--much morebrightness than on a CRT," says Einar Anderson, marketing manager forsensor products at dpiX.

Aydin Displays (Fort Washington, PA), another developer of active-matrix LCDs,already has 16- and 13-in. high-resolution units in production. Aydin's vicepresident of sales David Smith says that early applications of this technologywill probably soon appear on devices that are designed for the operatingenvironment, such as anesthesia machines and patient monitors. Eventually theproducts might be used to display diagnostic images from an MR or CT scanner. "Ourplan in going to medical equipment vendors is to be neutral in terms of specificuses," Smith says. "But where we see the greatest demand is in devicesthat require a lot of portability."

HEAD-MOUNTED DISPLAYS

Because they are lightweight and compact, and have low power requirements, flatpanels are well suited for use in head-mounted displays. The CardioView deviceis a good example of how head-mounted displays can be used in medicalapplications. If a surgeon is doing a coronary artery bypass graft, he or shecan see real-time images of wall motion with CardioView. Or, in the case ofheart-valve surgery, the flat panels may provide images and data aboutregurgitation or about the valve itself. The device resembles a bicycle helmetwith two small LCD screens mounted in front. "It is less intrusive than apair of sunglasses," says Vista Medical's Briefs. "The beauty of theheadset is that it allows for a lateral look down so the surgeon can see hishands and instruments, but it still puts the information about the anatomicalsite right in front of his eyeballs."

Planar America is collaborating with the Defense Department to develop ahigh-resolution LCD with pixels that are just 12-µm wide--more than seventimes smaller than is possible with the best radiographic film printers, thestandard against which medi-cal imaging technologies are compared. Vieirapredicts that in the future surgeons will be able to wear high-resolutionhead-mounted displays to view MR scans that show the results of their efforts asthey work. "You would be able to see, as you were doing surgery, whetheryou had completely removed all the pathology," says Vieira. "Noguesswork."

Such futuristic surgery is still a few years off. "While there is noquestion that it can be accomplished--all the technological pieces arethere--it's unlikely that anyone but the most advanced research hospitals willtry anything like that soon," Vieira says.

In the interim, head-mounted displays may solve an especially vexing problem forradiologists as they prepare for the expected wave of new diagnostic x-raysystems, such as digital full-breast mammography systems, which are beingdeveloped by several companies. The x-ray machines will produce high-quality,high-resolution images of the breast of 4000 × 5000 ppi--far beyond thecapability of the most sophisticated 2000 × 2000-ppi monitors nowcommercially available.

At National Information Display Laboratories (Princeton, NJ), engineers aredeveloping a head-mounted display to be used initially by analysts at the CIAand Pentagon to study surveillance images taken by satellite. These displaysmight be adapted to view digital mammograms at full resolution, says DanielKopans, MD, director of breast imaging at Massachusetts General Hospital(Boston), who visited the laboratory and tried out the display. "When youmove your head closer to the wall, the image gets bigger; when you move furtheraway, it gets smaller." According to Kopans, if the technology is adaptedto viewing digital mammograms, "it would be almost like looking at a seriesof viewboxes."

The aerospace industry has been the catalyst for developing head-mounteddisplays, not only in military settings, but also in space. A head-mounteddisplay developed by NASA to study the effects of microgravity encountered inearth orbit is being adapted for use by surgeons at Stanford University MedicalCenter. The system, known as a full-immersion head-mounted display, creates avirtual environment in which surgeons can plan and even visualize the results ofcomplex craniofacial reconstructive surgery. Software integrates laser imageswith CT head scans, creating precise 3-D images of the patient's face and skull.

Scientists at the NASA Ames Research Center (Mountain View, CA) and theDepartment of Plastic and Reconstructive Surgery at Stanford University MedicalCenter are developing this virtual workbench. "Our goal is to allowsurgeons to 'see' the patient's face and the skull behind it and enable them touse the same tools in a virtual environment that they would use in the actualsurgery," says Muriel Ross, PhD, director of Ames's Biocomputation Center,where the work is being done. "Surgeons will be able to removecomputer-simulated pieces of bone, cut them into appropriate shapes, and placethe pieces as desired. Then they can simulate the replacement of soft tissuesand observe the new features before they operate."

To build the 3-D image, plastic surgery resident Michael Stephanides makespreoperative laser scans of the patient's face. Ames computer specialists matchfacial features to the skull features in CT scans by extracting the bonestructure from the series of scans, contouring it, and then using customsoftware to reconstruct the skull. Because the reconstructed face istransparent, the bone structure is visible underneath.

A primary focus of the collaborative NASA/Stanford team is to work with childrenwho need reconstructive surgery to correct deformities of the head and face.Another focus is working with mastectomy patients who need breastreconstruction. The system is flexible enough that it could be applied in othersurgical situations as well.

Ross says the technology, which could be ready for clinical testing within ayear, will have several benefits. Patients with disfigurements could be bettersatisfied with their appearance after surgery, because surgeons will "seewhat the result will be before the surgery is even started," he says. Thisvirtual reality system also may increase efficiency by improving the outcome ofthe surgery and by reducing the time the patient actually spends in surgery,thus also reducing the cost of the procedure.

Finally, the system promises to allow the training of craniofacial surgeons,even those in remote areas, without their having to do actual surgery. Surgeonswould be able to pick up the same surgical instruments and make cuts and movebone as they would in an actual surgical procedure--but in a virtualenvironment. Eventually, the training program will include feedback such assound and tactile sensation. "There are real financial stresses on medicinetoday, so training becomes a critical issue, as does time in actual surgery,"says Stephen Schendel, MD, chair of the Department of Plastic and ReconstructiveSurgery at Stanford.

HOLOGRAMS

Lightweight flat panels fitted inside helmets heighten the 3-D effect of thedisplay medium. Holograms, images projected into space, do not need even thisartifice to make the images appear real to the viewer; they do not require a 2-Ddisplay surface at all. Since the late 1980s, computer algorithms have been usedto reformat data obtained with CT and MR into 3-D images. With its Voxcamholography product, Voxel has made these types of images appear even morelifelike.

Neurosurgeon Collins at Tripler Army Medical Center uses these life-sizeholograms to assess his progress as he removes brain tumors, comparing hisongoing surgical efforts with the holographic representation. He does this byinserting a cotton ball into the tumor cavity created by the surgery so that thecotton molds to the shape and volume of the cavity. The cotton is then removedwith a pair of forceps and held inside the floating hologram, revealing wherethe cotton ball and the hologram differ. "It is surprising to find thatwhen, after you have been working for hours and think you have probably removed99% of the tumor, you can take that volume measurement and hold the cotton ballin the hologram and see that there is a whole lobule that you have missed, andfind you have only got 50% of the tumor," Collins explains.

Voxgrams provide the added benefit of simulating changing perspectives of up to180°. Just by moving his head, Collins can get a different viewing angleon the hologram, or he can flip the film that is being projected to look at theanatomy from the opposite side. The Voxgram is also transparent, which allowsCollins to see the tumor in relation to normal brain structures, which can helphim avoid damaging healthy tissue.

FLY-THROUGHS

Static 3-D hologram displays are not useful, however, in evaluating anatomicalcavities such as the colon. Vital Images has solved that problem byreconstructing CT images to simulate movement through the colon. Suchreconstructions, sometimes called fly-throughs, hold the promise ofallowing CT to at least partly supplant endoscopy, a more than mildlyuncomfortable procedure, and in the process expand applications for the morethan 10,000 CT scanners installed around the world.

The concept is not unique to Vital Images; other companies have generated suchvirtual internal voyages. But these reconstructions have typically beenprocessing-time intensive, slowing down the clinical procedures. Vital Imageshas developed a volume-rendering technology that produces these reconstructionsin real time on a standard Silicon Graphics (Mountain View, CA) computerworkstation.

"Previously, to generate a fly-through, one had to establish key frames,then let the computer interpolate between these key frames. This process cantake up to an hour. Now, fly-throughs can be done interactively," saysVincent Argiro, Vital Images' founder and chief technology officer.

The company has spent eight years pioneering volume rendering on standardcomputer workstations. This most recent version of the technology is expected toserve as the backbone for a family of new products being developed by VitalImages engineers. The cornerstone of that family is the Vitrea workstation,which is now in FDA review. This workstation will enable the user to navigateinteractively through 3-D images representing any cavity in the body--not onlythe colon, but also the spine, lungs, and even arteries. "Real-timenavigation will be the primary enabling technology for the proliferation of 3-Dinto diagnostic radiology, surgical planning, and surgical navigation,"Argiro says.

MINOR PROBLEMS, BRIGHT FUTURE

As with any new technology, there are, of course, some limitations. For example,although practitioners working in virtual environments don't have to deal withthe gravity forces of an aerial dogfight or the effects of space sickness, theydo sometimes develop nausea after viewing 3-D images for prolonged periods.

According to Horsley at International Telepresence, nausea commonly occurs whenlooking at 3-D displays obtained from endoscopes and other such minimallyinvasive devices. The 3-D effect is typically produced by two cameras set atslightly different angles. "Twin camera 3-D depends on having average eyespace and a lot of people are not average," Horsley says. "If you arenot average, your brain has to work a lot harder to process that image."Over long periods, such as an hour or more, this processing can cause eyestrain, headache, and nausea.

Telepresence professes to have solved the problem with proprietary optics thatreplace the two cameras with one. "We have found no limit to the amount oftime you can use our system. Single-camera 3-D seems to be totally compatiblewith the brain," Horsley says. "Otherwise, with twin cameras, peoplewill pop out after just 30 minutes or so. That's why a lot of otherwise goodsystems that came on the market and ignored this basic fact have run into a lotof resistance."

Cost is also a potentially limiting factor in the adoption of these new displaytechnologies. But here again, companies are working to overcome this.

The goal of Vital Images, says Andrew Weiss, "is to provide a workstationat a cost that will encourage hospitals to purchase in quantity." Thetarget price is $50,000.

The CardioView product, which the company expects to launch this spring afterFDA clearance is obtained, will be priced between $70,000 and $100,000. Briefssays she doubts that the price will become a barrier to the adoption of thisimportant new technology. "Cardiac surgeons need visualization technology,"she says.

The flat panels, such as those integrated into CardioView, are themselves about50% more expensive than standard LCDs found in many electronic medical devices.Industry experts predict that because of the potential for volume sales of thetechnologies, costs will come down.

Like many who are developing these new technologies, Briefs is comfortable aboutthe future. Cardiac surgery, she says, is only the first application for hercompany's CardioView. "The whole world is going digital, and we have thefirst digital platform that provides surgeons access to this information,"Briefs says. "Within two years, they will be using it for things we haven'teven imagined today."

Photos courtesy of NASA Ames Research Center and Vista Medical Technologies.

Copyright © 1997 Medical Device & Diagnostic Industry