An MD&DI January 1997 Feature Article
Back when color television was still a relatively new technology, Canadian educator Marshall McLuhan said that the medium is the message. By this, he meant that the physical attributes of a display medium such as print or television will affect the message it carries; for example, the same movie viewed on a "cool" TV in an empty living room will have a different meaning than when viewed on a "hot" big screen in a crowded theater.
Now, some 30 years later, the idea has taken on significance for the medical device industry as display manufacturers find revolutionary new ways to present critical information to health-care providers. Flat-panel displays, holograms, and head-mounted displays are dramatically changing the way medical messages are communicated.
New display technologies are becoming part of all types of medical devices, from solid-state thermometers to bedside monitors, and endoscopes to magnetic resonance (MR) scanners. They are being used to show patient and equipment status, as well as patient anatomy in two and three dimensions.
At Tripler Army Medical Center (Honolulu), John Collins, MD, chief of the Department of Neurosurgery, studies holograms before and during brain surgery. These holograms are provided by a new display product called the Voxcam (Voxel, Laguna Hills, CA), a laser camera that stacks MR or computed tomography (CT) images on film. The 3-D holograms that form when light shines through the images appear suspended in space. "We are able to navigate point by point--cross-referencing between a tumor and the hologram--taking a measurement of our depth with the probe and putting the probe in the hologram to the same depth to see exactly where we are," Collins says. "As more tumor is removed, we can monitor our depth to see how close we are getting to the back wall."
Vista Medical Technologies, Inc. (Carlsbad, CA), has developed a head-mounted display called CardioView, which provides data about patient vital signs as well as real-time ultrasound images of the beating heart to cardiologists during minimally invasive cardiac surgery. The data are presented on two flat-panel displays that are positioned one above each eye. "It provides an ergonomic and, more importantly, intuitive way for surgeons to see the digital information captured from the anatomical site," says Nancy Briefs, vice president and general manager of cardiothoracic surgery at Vista.
Miniature cameras sending images to displays and special 3-D glasses are now enabling physicians to see deep within the patient as they investigate with endoscopes, laparoscopes, and sigmoidoscopes. One of the newest of these endoscopic imaging systems, developed by International Telepresence (Vancouver), reportedly imitates the process of natural vision. "The vast preponderance of endoscopy is done in 2-D--a flat image on a monitor," says Nigel Horsley, vice president at Telepresence. "This is real 3-D."
Real-time navigation of imaging data on computer screens promises to allow clinicians to tour patient anatomy and make adjustments to provide the best diagnostic view. The first application has been to develop a kind of virtual colonoscopy, which provides images similar to those obtained with an endoscope--but from CT data. At the forefront of that effort is Vital Images (St. Paul, MN), which is developing 3-D workstations in collaboration with radiology departments at Duke University (Chapel Hill, NC) and Stanford University (Palo Alto, CA). "We are convinced that this work will bring 3-D visualization to a broad range of clinical applications," says G. Allan Johnson, professor of radiology at Duke.
The fundamental building blocks of this revolution in display technology are flat-panel displays (FPDs). By expanding the readability of cathode-ray tubes (CRTs), these technologies promise to have an enormous impact on all types of common health-care products: patient monitors; defibrillators; machines for anesthesia and dialysis; control panels for x-ray, MR, ultrasound, and CT scanners; blood analyzers; and any other medical device that requires an electronic readout.
According to Albert Smith, vice president at Noritake Co. (Arlington Heights, IL), a developer of flat-panel technology, improved displays are needed in clinical settings today. "When I go into a hospital and see some of the poor displays, I am shocked, because this is such a critical place," Smith says. "With medical equipment, you should be able to read the data."
Four major types of FPDs are offering improved capabilities for medical devices.
Vacuum Fluorescent Displays (VFDs). Composed of highly efficient phosphors coated directly onto transparent anodes, VFDs are fashioned into matrices of widely spaced phosphor dots. The displays offer the advantage of drawing very low voltages and can be seen easily in sunlight, but they are typically limited to applications that require low informational content.
Noritake Co. is one maker of active-matrix VFDs, which are noted for high brightness, high density, low power consumption, and simple interfaces. Noritake's new technology, which will be on the market this winter, is based on a 16 × 16-phosphor- dot array formed on silicon chips. The company has standardized its first line of products for customers needing high-brightness graphics or character displays with low voltage and power consumption. Smith says that the brightness levels of these VFDs are twice those of other types of displays. "These particular devices are small devices, but they can display anything from graphics to alphanumeric characters," Smith says. "So they can be used on any device type, whether blood analyzer or x-ray readout--wherever digital or small graphical information is needed."
Electroluminescent (EL) Displays. This type of display can be thought of as a sandwich comprising a layer of light-emitting material nestled between two insulating layers, and then placed between two conducting electrodes. A light emitter or phosphor is placed between the insulating layers. Thin and compact, EL displays provide very fast video rates and exhibit good readability and brightness.
Planar America (Beaverton, OR) released two new flat-panel products based on EL technology in the summer of 1996--a 512 × 256-pixels-per-inch (ppi) display with a 6.3-in. diagonal screen, called the Q2, and a 480 × 240-ppi display with a 6.1-in. screen, called the PR2. Both feature a 1-millisecond response time. The Q2 is optimized for a high resolution of 96 dots per inch (dpi) to address the patient monitor marketplace, particularly for the display of waveforms such as cardiac cycles. The PR2 is optimized for extended life. The lifespan of EL technology commonly ranges between 30,000 and 50,000 hours mean time between failure (MTBF). Tests show that PR2 delivers an unprecedented 100,000-hour MTBF. "It doesn't have a backlight to wear out," says Jerry Vieira, director of strategic marketing at Planar.
Both displays exhibit the characteristic benefits of EL--high brightness, striking contrast, and wide viewing angle. Both are designed to be highly resistant to shock, vibration, humidity, and temperature changes. "That counts for a lot in ambulances when the temperature's 40°F in Minnesota or 105°F in Arizona," Vieira says. They also include low-power modes to address the needs of battery-powered portable applications, making them ideal for ambulatory portable instruments.
Field-Emission Displays (FEDs). Like EL displays, FEDs use row electrodes, column electrodes, and phosphors. However, unlike EL displays, FEDs also have a light emitter made of tungsten, diamond, or silicon and connected to the row electrode. Field emission occurs when an electrical field is applied to the tip of the emitter in a vacuum, and electrons tunnel from the tip into the vacuum.
PixTech (Santa Clara, CA) has developed a graphical flat panel based on FED technology, which is sometimes called cold-cathode technology. Two major advantages of the product are its 160° viewing angle, both horizontally and vertically, and its range of operating temperatures: 40° to 85°C. The company is in limited production on the FED product and expects to ramp up to full capacity in 1997. "Our strategic focus is to concentrate on markets such as medical devices, particularly patient monitors, that need our unique features and benefits that can't be provided by other display technologies," says Greg King, a sales manager at PixTech. "Viewing angle is a critical issue in a lot of medical applications, because space is generally at a premium and you don't necessarily have the luxury of being right in front of the display."
Liquid Crystal Displays (LCDs). These types of displays, based on liquid chemicals, share much of a liquid's dependence on temperature, stress, electrical field, and chemical environment. They have the advantage of using very little power and can be fabricated into very thin housings. Most LCDs that are currently in use suffer from slow response time and readability limitations, because of both lighting and angle. But as this technology continues to develop rapidly, these problems may soon be solved.
The Palo Alto, CA, firm dpiX is developing active-matrix LCDs based on thin-film transistors (TFTs) capable of being fabricated into page-size flat panels with up to 7 million pixels. These TFTs act as switching elements that control critical voltage levels affecting the liquid crystal. The switches control very specific amounts of light passing through to each pixel. Tight control allows the display to be highly responsive to signals while allowing maximum light output. "Each pixel is 90-µm in size and can produce 200 fL--much more brightness than on a CRT," says Einar Anderson, marketing manager for sensor products at dpiX.
Aydin Displays (Fort Washington, PA), another developer of active-matrix LCDs, already has 16- and 13-in. high-resolution units in production. Aydin's vice president of sales David Smith says that early applications of this technology will probably soon appear on devices that are designed for the operating environment, such as anesthesia machines and patient monitors. Eventually the products might be used to display diagnostic images from an MR or CT scanner. "Our plan in going to medical equipment vendors is to be neutral in terms of specific uses," Smith says. "But where we see the greatest demand is in devices that require a lot of portability."
Because they are lightweight and compact, and have low power requirements, flat panels are well suited for use in head-mounted displays. The CardioView device is a good example of how head-mounted displays can be used in medical applications. If a surgeon is doing a coronary artery bypass graft, he or she can see real-time images of wall motion with CardioView. Or, in the case of heart-valve surgery, the flat panels may provide images and data about regurgitation or about the valve itself. The device resembles a bicycle helmet with two small LCD screens mounted in front. "It is less intrusive than a pair of sunglasses," says Vista Medical's Briefs. "The beauty of the headset is that it allows for a lateral look down so the surgeon can see his hands and instruments, but it still puts the information about the anatomical site right in front of his eyeballs."
Planar America is collaborating with the Defense Department to develop a high-resolution LCD with pixels that are just 12-µm wide--more than seven times smaller than is possible with the best radiographic film printers, the standard against which medi-cal imaging technologies are compared. Vieira predicts that in the future surgeons will be able to wear high-resolution head-mounted displays to view MR scans that show the results of their efforts as they work. "You would be able to see, as you were doing surgery, whether you had completely removed all the pathology," says Vieira. "No guesswork."
Such futuristic surgery is still a few years off. "While there is no question that it can be accomplished--all the technological pieces are there--it's unlikely that anyone but the most advanced research hospitals will try anything like that soon," Vieira says.
In the interim, head-mounted displays may solve an especially vexing problem for radiologists as they prepare for the expected wave of new diagnostic x-ray systems, such as digital full-breast mammography systems, which are being developed by several companies. The x-ray machines will produce high-quality, high-resolution images of the breast of 4000 × 5000 ppi--far beyond the capability of the most sophisticated 2000 × 2000-ppi monitors now commercially available.
At National Information Display Laboratories (Princeton, NJ), engineers are developing a head-mounted display to be used initially by analysts at the CIA and Pentagon to study surveillance images taken by satellite. These displays might be adapted to view digital mammograms at full resolution, says Daniel Kopans, MD, director of breast imaging at Massachusetts General Hospital (Boston), who visited the laboratory and tried out the display. "When you move your head closer to the wall, the image gets bigger; when you move further away, it gets smaller." According to Kopans, if the technology is adapted to viewing digital mammograms, "it would be almost like looking at a series of viewboxes."
The aerospace industry has been the catalyst for developing head-mounted displays, not only in military settings, but also in space. A head-mounted display developed by NASA to study the effects of microgravity encountered in earth orbit is being adapted for use by surgeons at Stanford University Medical Center. The system, known as a full-immersion head-mounted display, creates a virtual environment in which surgeons can plan and even visualize the results of complex craniofacial reconstructive surgery. Software integrates laser images with CT head scans, creating precise 3-D images of the patient's face and skull.
Scientists at the NASA Ames Research Center (Mountain View, CA) and the Department of Plastic and Reconstructive Surgery at Stanford University Medical Center are developing this virtual workbench. "Our goal is to allow surgeons to 'see' the patient's face and the skull behind it and enable them to use the same tools in a virtual environment that they would use in the actual surgery," says Muriel Ross, PhD, director of Ames's Biocomputation Center, where the work is being done. "Surgeons will be able to remove computer-simulated pieces of bone, cut them into appropriate shapes, and place the pieces as desired. Then they can simulate the replacement of soft tissues and observe the new features before they operate."
To build the 3-D image, plastic surgery resident Michael Stephanides makes preoperative laser scans of the patient's face. Ames computer specialists match facial features to the skull features in CT scans by extracting the bone structure from the series of scans, contouring it, and then using custom software to reconstruct the skull. Because the reconstructed face is transparent, the bone structure is visible underneath.
A primary focus of the collaborative NASA/Stanford team is to work with children who need reconstructive surgery to correct deformities of the head and face. Another focus is working with mastectomy patients who need breast reconstruction. The system is flexible enough that it could be applied in other surgical situations as well.
Ross says the technology, which could be ready for clinical testing within a year, will have several benefits. Patients with disfigurements could be better satisfied with their appearance after surgery, because surgeons will "see what the result will be before the surgery is even started," he says. This virtual reality system also may increase efficiency by improving the outcome of the surgery and by reducing the time the patient actually spends in surgery, thus also reducing the cost of the procedure.
Finally, the system promises to allow the training of craniofacial surgeons, even those in remote areas, without their having to do actual surgery. Surgeons would be able to pick up the same surgical instruments and make cuts and move bone as they would in an actual surgical procedure--but in a virtual environment. Eventually, the training program will include feedback such as sound and tactile sensation. "There are real financial stresses on medicine today, so training becomes a critical issue, as does time in actual surgery," says Stephen Schendel, MD, chair of the Department of Plastic and Reconstructive Surgery at Stanford.
Lightweight flat panels fitted inside helmets heighten the 3-D effect of the display medium. Holograms, images projected into space, do not need even this artifice to make the images appear real to the viewer; they do not require a 2-D display surface at all. Since the late 1980s, computer algorithms have been used to reformat data obtained with CT and MR into 3-D images. With its Voxcam holography product, Voxel has made these types of images appear even more lifelike.
Neurosurgeon Collins at Tripler Army Medical Center uses these life-size holograms to assess his progress as he removes brain tumors, comparing his ongoing surgical efforts with the holographic representation. He does this by inserting a cotton ball into the tumor cavity created by the surgery so that the cotton molds to the shape and volume of the cavity. The cotton is then removed with a pair of forceps and held inside the floating hologram, revealing where the cotton ball and the hologram differ. "It is surprising to find that when, after you have been working for hours and think you have probably removed 99% of the tumor, you can take that volume measurement and hold the cotton ball in the hologram and see that there is a whole lobule that you have missed, and find you have only got 50% of the tumor," Collins explains.
Voxgrams provide the added benefit of simulating changing perspectives of up to 180°. Just by moving his head, Collins can get a different viewing angle on the hologram, or he can flip the film that is being projected to look at the anatomy from the opposite side. The Voxgram is also transparent, which allows Collins to see the tumor in relation to normal brain structures, which can help him avoid damaging healthy tissue.
Static 3-D hologram displays are not useful, however, in evaluating anatomical cavities such as the colon. Vital Images has solved that problem by reconstructing CT images to simulate movement through the colon. Such reconstructions, sometimes called fly-throughs, hold the promise of allowing CT to at least partly supplant endoscopy, a more than mildly uncomfortable procedure, and in the process expand applications for the more than 10,000 CT scanners installed around the world.
The concept is not unique to Vital Images; other companies have generated such virtual internal voyages. But these reconstructions have typically been processing-time intensive, slowing down the clinical procedures. Vital Images has developed a volume-rendering technology that produces these reconstructions in real time on a standard Silicon Graphics (Mountain View, CA) computer workstation.
"Previously, to generate a fly-through, one had to establish key frames, then let the computer interpolate between these key frames. This process can take up to an hour. Now, fly-throughs can be done interactively," says Vincent Argiro, Vital Images' founder and chief technology officer.
The company has spent eight years pioneering volume rendering on standard computer workstations. This most recent version of the technology is expected to serve as the backbone for a family of new products being developed by Vital Images engineers. The cornerstone of that family is the Vitrea workstation, which is now in FDA review. This workstation will enable the user to navigate interactively through 3-D images representing any cavity in the body--not only the colon, but also the spine, lungs, and even arteries. "Real-time navigation will be the primary enabling technology for the proliferation of 3-D into diagnostic radiology, surgical planning, and surgical navigation," Argiro says.
MINOR PROBLEMS, BRIGHT FUTURE
As with any new technology, there are, of course, some limitations. For example, although practitioners working in virtual environments don't have to deal with the gravity forces of an aerial dogfight or the effects of space sickness, they do sometimes develop nausea after viewing 3-D images for prolonged periods.
According to Horsley at International Telepresence, nausea commonly occurs when looking at 3-D displays obtained from endoscopes and other such minimally invasive devices. The 3-D effect is typically produced by two cameras set at slightly different angles. "Twin camera 3-D depends on having average eye space and a lot of people are not average," Horsley says. "If you are not average, your brain has to work a lot harder to process that image." Over long periods, such as an hour or more, this processing can cause eye strain, headache, and nausea.
Telepresence professes to have solved the problem with proprietary optics that replace the two cameras with one. "We have found no limit to the amount of time you can use our system. Single-camera 3-D seems to be totally compatible with the brain," Horsley says. "Otherwise, with twin cameras, people will pop out after just 30 minutes or so. That's why a lot of otherwise good systems that came on the market and ignored this basic fact have run into a lot of resistance."
Cost is also a potentially limiting factor in the adoption of these new display technologies. But here again, companies are working to overcome this.
The goal of Vital Images, says Andrew Weiss, "is to provide a workstation at a cost that will encourage hospitals to purchase in quantity." The target price is $50,000.
The CardioView product, which the company expects to launch this spring after FDA clearance is obtained, will be priced between $70,000 and $100,000. Briefs says she doubts that the price will become a barrier to the adoption of this important new technology. "Cardiac surgeons need visualization technology," she says.
The flat panels, such as those integrated into CardioView, are themselves about 50% more expensive than standard LCDs found in many electronic medical devices. Industry experts predict that because of the potential for volume sales of the technologies, costs will come down.
Like many who are developing these new technologies, Briefs is comfortable about the future. Cardiac surgery, she says, is only the first application for her company's CardioView. "The whole world is going digital, and we have the first digital platform that provides surgeons access to this information," Briefs says. "Within two years, they will be using it for things we haven't even imagined today."
Photos courtesy of NASA Ames Research Center and Vista Medical Technologies.