Reducing the Duration of Clinical Investigations

Medical Device & Diagnostic Industry Magazine
MDDI Article Index

An MD&DI May 1998 Column

CLINICAL TRIALS

One company's team shows how to undertake a large-scale effort to reduce clinical investigation time and save money.

Clinical investigations demonstrate a device's performance and are a prerequisite of product launch. Because clinical investigations are included as part of product development time, any reduction in their duration creates corresponding cuts in both the human and financial resources required to develop products and, ultimately, time to market. A decision to evaluate and reduce the durations of clinical investigations while still conducting them in such a way as to meet customer demands and to comply with regulatory and internal requirements prompted the clinical services department of the patient monitoring business unit of Johnson & Johnson Medical, Inc. (Tampa, FL), in 1995 to assess the way it was conducting its clinical investigations.

The clinical services team understood that evaluating and refining its clinical investigation processes was a large undertaking. The goal had to be accomplished using existing resources already committed to other projects. The team agreed that it would take small steps over a long time, making the goal part of a continuous improvement process.

The clinical services department conducts clinical investigations and acts as a clinical resource to the business unit, which designs, develops, and markets multiparameter physiological monitoring equipment. Clinical services team members develop hazard analyses for new products, complete risk assessments for marketed products, and communicate directly to customers about product performance. The department's clinical testing covers the safety and efficacy of physiological multiparameter monitoring equipment and encompasses testing software, hardware, and system components. Product approval is achieved primarily through the 510(k) process with a focus on parameter accuracy, equivalency testing, and overall system performance testing. Investigations are completed with the subjects and in the clinical units for which the device is ultimately intended. The clinical services department is also represented on the product optimization review committee that evaluates the performance trends of marketed products and assesses any need for corrective action.

Although the clinical service team's main focus is carrying out clinical investigations during feasibility and product development stages, its responsibilities extend in both directions from those early development phases. The clinical services team also outlines, coordinates, and evaluates outside bench testing during the preclinical testing period. Following development-focused investigations and prior to the market launch of the final product, the team manages the marketing acceptance tests and focuses on understanding and resolving any remaining customer issues with the new product.

IDENTIFYING AREAS FOR CHANGE

Rather than make changes to the clinical investigation process based on intuition or perception, the department established a mechanism to quantitatively evaluate how, and in which parts of the clinical investigation process, the most effort was being expended. Working together, team members made a flowchart of each step in the clinical investigation process as it was implemented. Standard operating procedures (SOPs) derived from the flowcharts provided the team with a common foundation from which to identify and evaluate changes.

The major tasks and subtasks were extracted from the flowcharts, and for the next six months, team members recorded the amount of time they spent on each subtask, recording information in 15-minute units on a spreadsheet on their personal computer. Approximately 10 minutes a day were required to enter data. Since the goal was to modify the tasks with the longest duration, those with shorter durations were not recorded in the main spreadsheet. However, a separate list of minor tasks was maintained and used later to build composite durations for clinical investigations. The data from the common spreadsheet were analyzed to identify tasks with consistently longer durations.

The team identified which tasks and subtasks would be the focus of process refinement. Ideal durations were identified for each subtask, based in part on a review of available competitor information. Ideal times were compared to actual times, and the tasks to address were chosen based on the following criteria: activities that had a large mismatch between ideal and actual durations, activities with longer durations than those of competitors, activities for which completion was primarily under the control of the team, and activities the team felt needed modification. Activities with a large scope or fundamental to other tasks were given high priority.

The team gained several benefits from actively thinking about, measuring, and categorizing tasks and processes. The flowcharts and SOPs developed during this first step became part of a training manual for new employees and were used as checklists to ensure that all tasks were completed during an investigation. A portion of the weekly team meetings was used to discuss the work-improvement processes, talk about potential solutions to problems, and complete specific action items. Part of the performance appraisal and goal-setting process for each team member included continuous improvement activities focused on these processes. Recording the time spent on activities also provided an overview of what work outside the typical clinical investigation was being requested of and completed by the team.

Over the next two years, the team worked through a prioritized list of action items that were identified as having a large potential impact on clinical investigation durations. Processes were to be streamlined for ease of use, kept consistent with current regulatory and internal quality standards, and revised for any anticipated regulatory changes.

ANALYZING THE PROCESS

The scope of clinical investigations for any project is determined by the clinical services team during the concept phase of the business unit's product development process. The scope of the investigation is based on the preclinical data available for the product, federal regulatory requirements, intended claims, national testing standards, clinical investigational guidelines, and internal standards of performance quality. The clinical investigation process (Figure 1) guides how investigations are conducted, regardless of their purpose. Using the same process reduces training time for new associates and eliminates errors associated with selecting an incorrect process.

Figure 1. The clinical investigation process.

The first step is to meet with representatives from other functional units, such as product management, sales, regulatory affairs, and product development and determine a mutual set of objectives for the investigation. The team has derived a standard form, called the protocol development questionnaire (PDQ), that guides the meeting agenda to ensure that all relevant issues are discussed. The PDQ focuses on the device and accessories to be tested, the proposed labeling claims, marketing performance goals, test methodology, functional product specifications, equipment required for testing, and patient populations and clinical units to be targeted. It limits the scope of the investigation and reminds individuals about their respective action items. The completed PDQ is distributed to attendees. Multidisciplinary input at this first stage encourages commitment to the investigational plan.

After consensus is reached about the scope of the investigational plan, one clinician is designated as coordinator and continues throughout the investigation. A decision is made about whether the device poses significant or nonsignificant risk, as defined by FDA, in order to determine whether an investigational device exemption (IDE) must be obtained. The investigational protocol is drafted from the agreed-upon objectives, specifying methodology so that data collection procedures that affect investigator selection and the budget proposal can be outlined. The draft protocol, budget, and data-collection timeline are reviewed by potential investigators, and sites are qualified through presite monitoring visits.

Next, the investigational plan, including any revisions by the investigator or sponsor, is sent to the institutional review board (IRB). Before beginning a clinical investigation, the patient monitoring business unit requires sign-off of the design history file by all applicable departments, indicating that all internal requirements for quality have been met and the equipment is suitable for clinical testing. This file includes the results of bench testing, software release notes, and documentation of the hardware configuration being tested.

Subject enrollment and data collection begins after IRB approval. Data collection system validations are completed and documented by the clinical engineer before data are collected. Data are downloaded directly from the test and reference devices. Any necessary changes to the data collection methodology follow the protocol amendment procedure required by the IRB at the clinical site. The data are analyzed, and a final report is generated and disseminated throughout the business unit.

After this, a debriefing meeting is held at the clinical site and with internal functional unit representatives to review the investigation's conclusions and seek input for continuous improvement activities. A standard format, the clinical investigation evaluation form, has been developed to help capture any recommended changes in the site, personnel, or methodology. Site personnel often are asked to provide their input about product performance directly to business unit functional representatives. The site is then closed out, paid, and the final monitoring procedures completed.

STREAMLINING THE PROCESS

The team decided to streamline several of these steps in its typical clinical investigation. Specifically, identifying level of risk, clinical sites, and investigators was done earlier in the clinical process. Work instructions and SOPs were developed to simplify the process of submitting investigation protocols to the IRB, and databases were developed to more quickly estimate project budgets. Steps were taken to more quickly choose clinical sites and find consultants to work on projects. The protocol amendment procedure was incorporated into the new database to permit automation and easy access. Finally, SOPs and other documents were created on computer, linked together, and controlled as part of the business unit's quality systems management.

Determining Risk. Once the investigation's objectives have been identified, the accompanying methodology is outlined. At that point the level of risk to subjects can be determined. The sponsor's rationale for choosing the risk status is provided in the investigational plan for the investigator and IRB. Since the IRB's decision about the level of risk affects whether or not the IDE process is initiated, which in turn affects product development time, agreement about the risk designation is sought from the investigator up front. The investigator can then, if necessary, provide the IRB with additional rationales to assist the decisionmaking process. The determination of risk is made at the same time that clinical sites are identified.

Identifying Clinical Sites and Investigators. A research site/investigator selection form is used to screen potential investigators according to the Code of Federal Regulations and good clinical practices guidelines. The protocol, informed consent forms, and budget are sent to the investigators for review after clinical services receives a signed secrecy agreement. Promising clinical sites are visited and assessed by clinical services early in the selection process because the investigations are equipment-intensive and require networking, power, and phone capabilities. Often, software and hardware changes in the same device result in challenging the parameters iteratively in the same clinical arena. Sites and investigators that work well with the team by expeditiously enrolling patients and meticulously adhering to the protocol are solicited repeatedly. Since the parameters that are tested and the standards to which they are compared do not change frequently, over time investigators become increasingly skilled with the team's computerized data-collection systems and methodologies, thereby reducing the length of their training.

Submitting Protocols to the IRB. Work instructions delineate the minimum requirements for investigational protocol design to promote consistency and ensure the elements that make up good science are incorporated into all investigational plans. An SOP details the components required for informed consents to ensure consistency and compliance with regulations.

After the IRB approves the clinical investigational plan, information regarding approval dates, the principal clinical investigator and coinvestigator or coinvestigators, and IRB addresses is entered into the clinical data management system (described in more detail later). Subsequent amendment/addenda and annual IRB review cycles are also added to provide a calendar system that tracks each investigation throughout the year. This prevents missing a renewal date and provides at a glance all historical IRB activity to date. The database also provides automatic outputs of current subject enrollment, allowing the study monitor to decide which records require source document review.

Budgeting. The costs for all investigations are continuously tracked and maintained in a database on the department server. Business unit and departmental budgets are forecast using specific categories that are the same as those used in the investigational budget tracking system. The cost database also contains a description of the investigation that can be used to compare the one being budgeted with similar investigations previously completed. Using information about historically similar investigations and the standard charges associated with particular sites that are already recorded in the database, accurate budget proposals can be rapidly constructed the first time.

Although site costs do not change markedly from year to year, the team has noted an increase of approximately 30% for specific investigations using the same sites during the past three years. Increases in the technical complexity of the data-collection procedure are most directly linked to increases in cost.

The proposed budget is sent to the investigator following a general discussion of reimbursement. The investigator reviews the budget and makes modifications. If the changes are acceptable, the budget is finalized and included in the investigator agreement. Budgetary differences are usually resolved. When they are not, an alternative site for the investigation is sought.

Coordinating Clinical Data Collection. The clinical engineer in the clinical services department uses the information provided from the PDQ to further define equipment operation and interface issues with product development engineers. These discussions begin as soon as potential sites are identified because interface limitations could disqualify a site from the investigation. Sites are visited early to evaluate their ability to meet equipment interface requirements. The equipment is then tested in the clinical services laboratory in as close to actual use conditions as possible, using simulators and volunteers to provide data input. Performance problems are noted and referred back to product development for corrective action. When all internal documentation requirements are fulfilled, the equipment is inventoried, packed, and delivered to the site for biomedical testing and approval before data collection is initiated.

Outside support is sometimes used to augment internal data-collection resources. Consultants are extensively trained in the devices to be tested and in the data-collection system. A database of clinicians listed by clinical specialty, hospital or academic affiliation, and professional association is continually updated by team members and serves as a repository of local and out-of-state contacts. These contacts sometimes act as site coordinators, investigators, or clinical experts. Successful long-term relationships with these clinicians provide immediate clinical feed-back about product development ideas or feature cost/benefit tradeoffs.

Amending Protocols. Protocols are usually amended to accommodate methodological changes, extend sample sizes, or add different clinical units if software and hardware modifications are made to the device being tested. Subject enrollment is stopped if a methodological change alters the level of risk to subjects or invalidates the data already collected. If amendments increase sample sizes or clinical units, subject enrollment continues within the previous approval limits and is extended after IRB approval. The IRB at the site determines whether expedited or full review of an amended protocol is required, basing its decision on the extent of any perceived change in risk to subjects.

All of this information is collected and automated using the relational database of the clinical data management system (described below).

Developing SOPs and Other Documents. Clinical services currently has 17 SOPs that outline the basic process of conducting clinical research and related administrative activities. Linked to those SOPs are 15 work instructions that describe these processes in greater detail and also double as training and orientation materials for new employees, outside consultants, and temporary personnel. Seventeen form templates are also linked and controlled in this way, as part of the business unit's quality systems.

All of the documents created by clinical services, such as clinical investigation protocols, informed consent forms, medical screening forms, research advertisements, follow-up-care instructions, and case report forms reside on the department server, which can be accessed by all clinical services associates. All of these documents can be stored indefinitely as templates.

Creating and modifying these documents is discussed at team meetings and considered part of the continuous improvement goal of the clinical research process. These documents are reviewed at least annually for accuracy, compliance, and opportunities for streamlining.

CLINICAL DATA MANAGEMENT SYSTEM

In addition to the changes listed above, changes to the clinical data management system were made to gather data into one database and streamline data collection and archiving procedures. A number of computer software and hardware changes were incorporated into the clinical data management system to allow the team to view up-to-the-minute investigational progress.

A commercially available relational database application was chosen to enhance clinical investigation management, including case report form design, data entry, editing, and export of data for statistical analysis. The database replaced a drawing application for case report form design and a spreadsheet application for data and subject enrollment tracking.

The database's backbone is the set of master control checklist logs that functionally support the electronic case report form data-entry screens, department work responsibilities list, permanent archiving system tracking, study monitoring, clinical investigation amendment and renewal tracking, and modification or correction audit trails for each clinical investigation. Automated programming scripts allow data to be consolidated for export to the statistical application. The master control checklist logs are password-protected and continuously updated as subjects are enrolled, with data entry and verification performed throughout the investigation cycles. As a result, 95% of data entry, verification, cleaning, and processing (e.g., record scanning, storage, and archiving) has been completed by the time the last subject is enrolled. This software substantially reduces the time that is needed to statistically analyze the data, complete final administrative work, lock the database, and write the final investigational report.

Case Report Forms (CRF). The database allows the team to create data-entry screens as well as provide the hard-copy CRF to the clinical site for limited items such as demographic data. The database also incorporates any required graphics, such as anatomical pictures, needed to assist the site in proper device and accessory placement. Certain fields execute programs that automatically calculate age and convert units of measurement, thereby reducing error and effort during data entry. Specific descriptive statistics required for export into the statistical analysis program can be automatically formatted. Engineers in product development have access to certain subject data files as soon as those files are brought into the clinical services department. This allows the engineers to quickly perform up-front troubleshooting without needing to copy files, saving time and money and reducing the potential for error. The system handles diverse data formats, including analog outputs such as waveform, serial data as text, and digital data.

Paperless CRF. In-house investigations are occasionally conducted to evaluate discrete developmental improvements. Employees act as subjects, and data are collected in the department laboratory under the same clinical investigation processes previously outlined. Independent IRBs are used instead of site-specific ones. These investigations have provided the team with valuable information about processing data and documents using only electronic CRFs on a laptop computer. The CRFs were designed on spreadsheets within the same software application used for statistical analysis, saving time and resources by eliminating additional data entry and importation to a statistical package. However, the availability of the laptop computers, individuals' software application skills, and the underlying risk of losing the data during input because there was no hard-copy backup were all issues that came to the team's attention. In addition, concerns about correcting CRF errors and data verification prompted the team to create new audit trail processes using CRF hard copies, which defeated the purpose of a paperless system. Although the team has not given up on the project, it now better understands the issues that need to be addressed before a paperless system can be implemented.

Modem. Clinical data were previously collected on floppy disks or hard drives and carried to the company. Data sets from individual subjects that were too large to be accommodated on one floppy disk were split, a potential source of data corruption. When data for a patient were collected over several hours, the hard drive had to be transported, risking both the hardware and the data and incurring extra cost.

When the streamlining project started, the cost of processing clinical data was estimated at about $12 per subject. Transmitting data from sites by modem instead of carrying it by hand saved an estimated $46,000 over the year, if used for all investigations. Table I compares the advantages and disadvantages of using modems instead of traditional delivery systems. The process of transferring data by modem must be validated and documented for potential data corruption and distortion.

Table I. Advantages and disadvantages of sending data by modem.

Optical Scanning. The corporate record retention policy requires that all clinical investigation documents be kept permanently. Spreadsheet software is used to track these paper files. Accumulating paper records has presented problems such as fading thermal inks, inaccessibility caused by off-site storage, and the difficulty of tracking huge volumes of records. Before the streamlining project began, all paper CRFs, waveform strips, and monitor printouts were photocopied for permanent storage. Microfilming was abandoned early on because of equipment access problems, vendor expense, and the risk involved in transporting original documents to be photographed. Instead, the team began optically scanning its records. Table II lists the advantages of doing this.

Table II. Advantages of optical scanning.

Archiving. All electronically created clinical investigation documents are archived on optical-magnetic cartridges for permanent storage. These cartridges are then duplicated for storage off-site as part of the company's disaster recovery program. A spreadsheet database is used to log the contents and location of these backup cartridges for retrieval and is being reformatted to link to the relational database. Archiving is performed continuously as subjects are enrolled in the investigation and tracked by the master control checklist log databases. When an investigation is finally closed, the files are locked on the server, allowing read-only access.

Paper Records. Paper administrative records, such as outside correspondence and legal documents, are maintained in a central clinical investigation file within the department. Subject records, CRFs, original waveform strips, copies of informed consent forms, and medical screening forms are placed in record retention boxes, tracked on a spreadsheet database, and permanently placed off-site. Everything enclosed in a subject's file is indexed and available electronically on the server for immediate access. This spreadsheet application is also being converted to the relational database structure.

Bar Coding. According to federal regulations, investigational device accountability must be tracked between the sponsor and the clinical sites. The bar code—tracking database was initially created to fulfill that function and eliminate several other documentation processes required by the business unit, such as leasing equipment and supplies, and tracking equipment repairs and calibration cycles. The database was later expanded to include other functions, including reconciliation of equipment and supplies sent to the clinical sites; inventory checking for supply purchases and restocking; eventual disposition of equipment and supplies, including destruction of disposables; transfer of equipment between internal departments; and companywide fixed-asset inventories. Since the same equipment had entries under several of these categories, consolidating the records in one database reduced redundancy.

To date, the team has more than 2000 records in the bar code database, providing a running history of activity on each piece of equipment since it was acquired. Equipment checkout and check-in time has been reduced by more than 50%. Customized bar code labels allow the team to scan new supplies straight into the database as they are received. New features such as calibration due dates, documentation for cleaning equipment when brought back from the sites according to OSHA requirements, and continual updating of a linked automated glossary file of the business unit's actual product codes are also part of the system, as is the ability to run reports for each clinical investigation, whether closed or active.

Nonperformance Log. In order to collect data rapidly and only once, the team attempts to eliminate any factor that keeps data from being used in an analysis. Nonperformance data are categorized into patient/site personnel issues, actual test product nonperformance, and site equipment interface issues. The latter two categories, the team believes, are under its control and can be eliminated. The data are captured, categorized, and distributed to product development and marketing personnel to spot and resolve problems immediately. Specifically tracking the circumstances of each nonperformance problem has resulted in identifying the 50% of problems that can be resolved internally. This represents significant potential savings in effort, product development time, and cost.

REDUCTIONS IN CLINICAL INVESTIGATION DURATIONS AND COSTS

The clinical services team's primary goal was reducing the duration of clinical investigations. Typically, the team runs 40 such investigations at any given time—some are being set up, some are in active data collection, and some are being closed out. During 1995, the team set up and closed out eight investigations using the new and improved processes described above. In addition, one of the team's goals, independent of any time reductions due to the changed processes, was a 10% reduction in duration for all investigations based merely on having a stable workforce and, concomitantly, more experienced team members. Of the eight investigations that started and finished in 1995 using only the new processes, seven of those had shorter-than-expected durations. The rate of reduction in investigation durations averaged 41% overall. This is a slight underestimation because the 10% planned reduction was already contained in the original estimate.

The savings caused by these improved processes, including one study that ran longer than expected, was 30% in project costs alone. Again, this percentage is an underestimate because the effects of the 10% planned reduction were not counted. Costs associated with human effort represent incremental savings and, assuming team members were applied across all projects, would also equal 30%.

The cost savings were either applied to new investigations that were undertaken because funds became available or returned to the business unit. The savings in human resources were considered to be counted toward effort applied to other projects.

CONCLUSION

The clinical services team demonstrated that taking incremental steps to meet improvement goals can result in significant savings of money and effort. These savings, passed along to the business unit, were a 30% reduction in costs and a 41% reduction in clinical investigation duration. The team now generates more improvement ideas than can be implemented, and its new task has become that of setting clear criteria for determining the value of undertaking each suggested change.

Deborah A. Gorny, RN, PhD, is former director of, and Susanne C. Panzera is senior clinical information specialist for, Clinical Services Critikon, a division of Ethicon (Tampa, FL).

Illustration by Warren Gebert

Copyright ©1998 Medical Device & Diagnostic Industry