Implementing Computerized Data Management in Clinical Research

Medical Device & Diagnostic Industry Magazine
MDDI Article Index

An MD&DI March 1999 Column

CLINICAL TRIALS

Technologies relatively new to medical device clinical trials promise to change the face of more traditional studies.

Automated tools being introduced into medical device clinical trials can increase the overall efficiency of conventionally managed paper-based clinical studies. These information-management techniques, when chosen well, enable research sponsors to increase the accuracy and security of their research data and to save time and money in the process.

When selecting one of the less traditional techniques discussed in this article, medical device companies should take into account that each clinical trial is different. For example, a stent study of 1200 patients followed at 20 sites for three years may benefit significantly from such automation methods as remote data entry. However, a cosmetic laser study of 60 patients that is carried out at two centers probably would cost less and finish more quickly if it utilized a paper-based system. And while it is generally the case that more-complex studies with longer follow-up times benefit most from automation, medical device companies should consider that automating certain tasks, such as measuring structures on medical images, can work well for a study of any size, eliminating both bias and variability. Research sponsors should also evaluate the regulatory implications of their choices, especially regarding software validation.

The technologies discussed here can handle data generated by the observations of experienced personnel or from a wide range of sources, including computer-controlled therapeutic and diagnostic devices and manually operated devices like thermometers and stethoscopes.

DATABASES, DATA MANAGEMENT, AND STATISTICAL ANALYSIS

Data analysis is probably the first task that clinical research professionals think of when they discuss computerized systems. No sponsor of clinical research would consider conducting a clinical trial, no matter how small, without entering the data into a computerized database and using a software package to perform the statistical analysis. Data from simple studies can be entered into a spreadsheet and analyzed using the features available in that program, while larger, more-complex studies require relational databases and sophisticated statistical analysis. Research sponsors must consider carefully their choice of data-management and statistical analysis software. Above all, they must develop a plan both to process the data reliably and to analyze those data for the development of accurate conclusions.

A requirement of both the most basic spreadsheet and the most complex image-processing system is a procedure capable of ensuring data integrity. In its simplest form, maintaining data integrity may involve saving a spreadsheet file to a disk and locking the disk away whenever it is not in use, as well as keeping a paper log of all changes made to the file. More-complex software provides an electronic means for capturing the same information. Next, some formal method must be adopted to ensure that the correct data has been entered into the study's database. This step can be accomplished by duplicating data entry, by manually verifying previously entered data, or by combining manual and automated systems.

Validation is another requirement of automated systems. Like software that controls manufacturing processes, software that processes data must be validated to demonstrate that it functions in a consistent, acceptable manner. In most cases, validating the code for the application is not necessary, but any macros or programs prepared by the research sponsor must be tested and well documented.

OPTICAL SCANNING TECHNOLOGY

Optical scanning technology replaces the first manual data-entry step in a conventional system. Traditional paper case-report forms (CRFs) completed by clinical researchers at the investigational site are returned to the data-management site, where they are scanned to create a machine-readable document. Alternatively, researchers might scan the CRFs at the investigational site and fax the scanned images directly to a computer at the data-management site, thus eliminating the transfer of paper. A software program then reads the text in the fields completed by the investigator and enters this text into a database. Because the bulk of the data-entry process is automated, the speed at which data are entered increases significantly. Some system users have reported tenfold reductions in the cost of data entry.

If the optical scanner encounters questionable characters, they are identified and set aside for manual review. This review may include the display of unknown data, either in isolation or in its original textual context. Various convenience features are built into the software to structure the manual review of questionable characters for maximum efficiency.

This technology fits into the workflow of companies that already have data-entry departments, and no additional equipment or training is required at the clinical investigator site. For a data-entry department that uses PC-based workstations, the only major hardware expense would be one or more scanners, depending on the expected volume of CRFs. Products with many of these capabilities are available from such firms as Clinical DataFax Systems Inc. (Hamilton, ON, Canada), Universal Systems Inc. (Chantilly, VA), and Parexel International Corp. (Waltham, MA).

Manufacturers installing optical scanning technology must work closely with their vendors to ensure that their systems are capable of performing the expected tasks. Once it is certain that a system has been installed properly, all functions should be tested to verify that they perform as the developer intended. These steps often are conducted in collaboration with the software developer. The system should also be tested with actual CRFs. All of these efforts must be documented carefully. Once these tasks have been successfully completed and data-entry personnel have been trained, the system is ready for routine use.

REMOTE DATA ENTRY

Remote data entry (RDE) can generate significant savings of both time and money, especially for larger, more-complex device studies.

RDE can be as simple as a spreadsheet designed by the research sponsor. The clinical investigator uses the spreadsheet for data-entry purposes. During each monitoring visit, the clinical research associate (CRA) retrieves the disk and forwards it to data management, allowing available data to be incorporated quickly into a central database. This type of RDE system is relatively inexpensive to implement, since spreadsheets can be designed internally, and because nearly all sites have the hardware and software necessary to run them.

While spreadsheets may be suitable for tracking such information as device accountability or patient-enrollment statistics, they present several challenges when used for actual study data. The digital equivalent of the single-line cross-out, dating, and initialing system of making changes is inconvenient, and there is no mechanism consistent with FDA's electronic signature regulation to ensure that only authorized personnel are able to access the data. In addition, the macros or cell formulas used to generate the spreadsheet must be validated, and this effort must be documented. Using this type of system for critical applications requires a significant investment of time and yields few performance capabilities.

However, more-sophisticated commercial RDE systems—available from vendors like Technilogix (Ridgefield, NJ), IBM (Boston), MiniDoc Inc. (Cary, NC), and Parexel International Corp.—overcome these limitations in many respects. Research sponsors who utilize such systems will save the most time and money with larger, more-complex studies. This is especially true for those studies in which a relatively small quantity of data is collected for a large patient population, as systems can be programmed either to accept only data of a certain type (e.g., numerical) or to accept numbers within a specified range. This capability eliminates common key-entry errors at their source.

RDE systems usually operate on PCs, but the software is proprietary. Because site personnel type data directly into the system, traditional data entry is eliminated. In fact, many of these systems are capable of transmitting the data to the research sponsor via either a modem or the Internet, thus eliminating manual data-entry costs and the errors associated with the manual data-entry process. Although each system must be customized to a specific study in a process that typically takes four to eight weeks, it is possible to recover this time at the end of the trial since fewer corrections to the data will be required. Other system capabilities include generating progress reports that enable study managers to track site performance parameters and creating reports for CRAs that increase the efficiency of on-site monitoring visits, or, in some cases, that replace some of these visits with electronic monitoring. Sponsor personnel can use a variety of methods, including e-mail, to communicate with the investigational site. Several systems permit authorized users to append notes to data fields, which is especially useful when queries are processed.

From a quality assurance (QA) perspective, sophisticated RDE systems offer several advantages over paper systems. In-depth monitoring of a study's progress is possible, and missing data fields and CRFs become obvious immediately. Most importantly, audit trails are created automatically whenever data are modified. RDE systems can facilitate the generation of specialized QA reports, including such items as the number of adverse device effects per site, the number of corrections made to each CRF per site, and the number of patients terminating early from the study per site. These data can be useful when selecting sites to audit.

When selecting RDE software, device manufacturers must obtain a clear understanding of the validation status of the product. The validation process—from planning through execution and maintenance—must be thoroughly documented. Any modifications made to customize the program must also be validated. Often, developers will transfer source code and validation documentation to a third party, where it can be made available if regulatory authorities have questions. Another regulatory issue is the implementation of electronic signatures. The RDE system must provide a way to ensure that only authorized users are able to access the system. Research sponsors should refer to the electronic signature regulation for more information.

The training of site personnel must also be considered whenever new procedures are introduced at the investigational site. A system, no matter how well designed, will provide only limited benefits if site personnel are not trained to use it properly. Some organizations are offering training via interactive Internet connections. This permits CRAs to concentrate on study-related activities, while experienced specialists conduct RDE training.

INTERACTIVE VOICE-RESPONSE SYSTEMS

Especially for larger multicenter studies, an interactive voice-response system (IVRS) can provide significant cost savings, improved audit trails, and faster transfer of key study data. Credit-card companies, airline reservation centers, and financial service providers, for example, use these complex systems, which require customers to input identification numbers via telephone in order to query databases.

An IVRS can help to randomize and enroll patients quickly. Unlike in more-conventional studies that require at least one individual not otherwise connected with the study to maintain the randomization list and assign patients to treatment groups, the study's investigators can contribute patient data and receive an assignment via phone.

One large multicenter investigation that effectively used an IVRS involved a device designed for the critical-care environment. Patients were to be randomly assigned either to the investigational device or to the comparison product within a relatively short period of time. Each site was given an identification code and a toll-free telephone number to call. When an authorized investigator dialed into the IVRS and entered his or her password, a brief menu was presented. The caller could press a specific number to randomize a patient, enter a small amount of patient demographic data, and receive a treatment assignment. These transactions could be completed in two minutes or less. The IVRS then logged each transaction into a database that was available at all times to the study sponsor. Additional capabilities of the IVRS included automated reordering of CRFs and consumable supplies.

Using an IVRS offers numerous advantages. Project managers can have real-time patient-enrollment data, CRAs can better determine timing for the next monitoring visit, and auditors can easily identify high-enrolling sites. And, from a regulatory standpoint, each transaction is date- and time-stamped and saved in a database for documentation purposes. The sponsor can be certain that, once appropriate validation is complete, all transactions will be processed in a consistent manner according to study protocol.

However, an IVRS is not foolproof. Like any other computerized system, it must be validated. Furthermore, the system relies on telephones and telephone lines whose quality is beyond the control of the study sponsor. When an IVRS is employed in a multinational study, for example, static on the telephone line can be misinterpreted as input. Thus, a system must be designed to recognize noise on the telephone line and to reject it as a data-entry error.

IMAGE PROCESSING

Digitized radiography for viewing internal body structures and digitized photography for capturing images of wounds have led to the development of analytical algorithms and the automation of image analysis. Consequently, image data can be analyzed in the same manner throughout the course of a study and across all investigational sites, eliminating potential sources of bias from the data-analysis process. Certain parameters can be quantitated with minimal bias—for example, the lumen diameter measured during an interventional cardiology study or the tumor size measured in a radiation-therapy device trial. These types of systems can also be used to analyze data that, at first glance, may not seem to be images, including EEGs, ECGs, and EMGs. Digitized data can be masked and sent to reviewers quickly, reliably, and inexpensively.

Products for analyzing, manipulating, and storing medical imaging data, such as 3 D Doctor (Able Software Co.; Lexington, MA) or Cheshire (Parexel International Corp.; Waltham, MA), have been available in the clinical environment for more than 10 years. Systems designed for clinical research build on these capabilities and add others to carry out the tasks specified in the study protocol. In a typical application, images obtained at investigational sites are transmitted to a core lab for evaluation. The core lab can remove patient-identification data and any diagnostic notations to ensure that independent evaluators receive well-masked data. Once the evaluation of an image begins, measurement tools can be used to generate quantitative data. These tools are often more accurate and consistent than manual methods, which is particularly important when a study's protocol requires several readers to evaluate an image.

Device manufacturers must validate any system that manipulates, analyzes, transmits, or compresses data. They should make sure that the validation includes all the features of their systems. In addition, they must specify the computer system and the peripherals that will be controlled by the software, as various types of hardware may react differently to the same programming code. Generally, systems cleared by FDA for clinical use have been validated; however, sponsors of clinical research must evaluate carefully any changes or enhancements that might be made to these systems.

Image analysis is not a fully automated process. In most cases, either a technologist or a radiologist must interpret the image, and steps must be taken to ensure that variability—between readers and with the same reader over several sessions—is minimized. Two solutions are effective training and monitoring. Protocols can also be designed to ensure that more than one individual reads each image or that the same individual reads a selected subset of images more than once.

CONCLUSION

By automating certain tasks of paper-based systems, sponsors of medical device clinical trials can increase the accuracy of their research data and save time and money. While it may not be feasible to switch to an entirely electronic clinical trial data system—total conversion can be expensive and time-consuming—medical device firms that are comfortable with paper-based systems may still find it beneficial to incorporate new technologies. In fact, some organizations have found that these various automated systems, when implemented together, can be used to create a new clinical research process capable of generating accurate data in a rapid and cost-effective manner. Implementation of such systems will be a challenge for medical device companies well into the next decade.

ACKNOWLEDGMENTS

The author would like to express his gratitude to Denis Bilodeau and Gregg Cohen of the Advanced Technology Group at Parexel International Corp. for their technical guidance during the preparation of this article.

BIBLIOGRAPHY

Code of Federal Regulations, 21 CFR 11, "Electronic Records, Electronic Signatures, Final Rule." Federal Register, 62 FR: 13429, 1997.

Guidance for Off-the-Shelf Software Use in Medical Devices, Draft Document. Rockville, MD: FDA; Office of Device Evaluation, 1997.

Guidance for Industry: Computerized Systems Used in Clinical Trials, Draft Guidance. Rockville, MD: FDA, 1997.

Barry Sall is a senior regulatory consultant with Parexel International Corp. (Waltham, MA).

Copyright ©1999 Medical Device & Diagnostic Industry