Formal literature reviews play a significant role in device development. This is especially true when they support quality and regulatory compliance for these medi- cal devices. But some people may react to being assigned with conducting a formal literature review with apprehension.

It's understandable that a quality, regulatory, clinical, or R&D professional would be intimidated when asked to conduct and report on a literature review. Many of these project recipients do not regularly perform formal research on scholarly articles. A literature review is a large task that could easily require 60–80 hours of focused effort.

But literature reviews are a wise investment. They are often used to support design of clinical investigations, and they may even be used to support a literature route to conformity for CE mark approval. Several guidance documents are available that can help to ensure compliance with applicable regulations. Two regulatory guidance documents that focus on conducting literature reviews are MEDDEV 2.7.1, “Evaluation of Clinical Data: A Guide for Manufacturers and Notified Bodies,” and Annex A of ISO 14155-1, “Clinical Investigation of Medical Devices for Human Subjects: General Requirements.” Following these documents can help a reviewer organize the literature and ensure that the information gathered is of high quality.

As with most complex and challenging projects, literature reviews are less overwhelming if they are approached in a systematic fashion, one small piece at a time. To make the project a success, here are the steps to follow:

It should be noted that a formal literature review is not the result of a Google-type search. It requires considerable planning, methodical searches, and attention to detail. The information discovered in the process provides valuable input to the development of safe and effective medical devices.

Start with a Plan

Creating a plan is the first step in the literature review process. It is always tempting to skip the planning stage with any project. Override this temptation and follow the MEDDEV suggestion to develop and approve a written plan. A detailed written plan ensures a systematic review of the literature. At a minimum, the plan should define the literature review purpose, objectives, and methods.

Purpose. Declare the purpose of the literature review and provide background information as necessary (e.g., The purpose of this literature review is to evaluate the risks associated with coronary implant devices to support development of the XYZ Company's Alpha device).

Objectives. Declare the objectives of the literature review in measurable terms. In many cases, it is helpful to identify questions that the literature review is intended to answer (e.g., This literature review will attempt to answer the following questions: What is the rate of death associated with coronary implant devices? What other serious adverse events are reportedly associated with coronary implant devices? What is the reported incidence of adverse events?).

Methods. Define the methodology for searching and selecting the literature, including:

While preparing the literature review plan, it is important to perform preliminary searches to experiment with various inclusion and exclusion search terms. Although these searches are not considered formal searches, they do help ensure that the plan is designed to find relevant scientific articles that align with the objective.

For more-complex literature reviews, it may be necessary to develop methods for weighting the different articles and studies. It may also be useful to define the statistical methods of analysis to be employed. Statistical analysis of data from several studies is known as a meta-analysis. The University of Maryland Department of Measurement, Statistics, and Evaluation provides helpful meta-analysis resources on its Web site: http://edres.org/meta/.

Perform the Search

Perform the literature search in a reputable search engine following the methodology defined in the plan. Use the correct format for Boolean operators (e.g., “and,” “or,” etc.) as required by the search engine, and note that the use of quotation marks around search terms may significantly affect search results.

Review the list of search results. Because research is serendipitous, it may be necessary to add or remove inclusion and exclusion criteria as necessary. This helps narrow the search results to a reasonable quantity and to eliminate search results that do not align with the literature review objective. For example, if only 10 search results appear, it may be useful to add inclusion criteria or remove exclusion criteria. If about a thousand search results appear, remove inclusion criteria and add exclusion criteria. In most cases, an initial search results list of 20–50 articles is sufficient.

Citation indices provide an alternative way of finding documents. Two such indices include Science Citation Index and Social Sciences Citation Index. Note that additional articles may be discovered when reviewing bibliographies of selected documents.

It is prudent to perform additional cross-check searches to ensure that all relevant major studies have been captured in the literature review. For example, if the original search did not include the trade names of similar products, perform a search using these names with the same search limits defined in the plan. Scan the search results. If relevant and important articles appear on the list, it may be necessary to refine and save the search.

When the search process is complete, save and print the search results lists for inclusion as a literature review report attachment, providing traceability on the extent of searches performed and the sources of the data.

Select and Obtain Articles

Determine whether all of the articles on the search results lists are relevant to the literature review objectives. Irrelevant articles can be screened out initially by reviewing the titles and, if it is unclear from the title, reviewing article abstracts. Graphs and charts may also be developed during this step to complement data tables. When using PubMed as the search tool, abstracts are easily available for review with the click of a mouse button. Develop a coding system as necessary to document rationale as to why the screened articles do not meet the selection criteria.

When reviewing articles, pay attention to the authors and investigational centers. If the same group of authors published several articles on the same study, consider which articles are most relevant to the literature review, and exclude any articles that feature duplicate information. Be sure to provide exclusion rationale in the literature review report.

Obtain copies of all selected articles. Note that copyright charges add up quickly, so be prepared for a large expense. Many companies have an internal reference library and subscribe to scientific journals, so use whatever literature resources are already available. Articles may be purchased from various sources including the publisher, PubMed, and biomedical libraries. Delivery options range from online downloads of PDF files to mail delivery of photocopies.

After receiving the articles, make a temporary working copy for each article. Add highlights and notes to the working copies and place the originals in the final report or the company reference library.

Review and Summarize Selected Articles

Peruse each of the selected articles and underline or highlight information that relates to the literature review objectives. This step of the process helps literature reviewers become acclimated to the topic, and it provides an initial survey on the level of reporting consistency in the selected articles. Some may find it easier to take separate notes, rather than highlighting, to help with retention of the material.

It is helpful to develop a literature article summary table to include relevant and key information from each of the selected articles. The table provides a handy resource in the appendix of the literature review report. In addition, the process of creating the table further helps to distill and broadly categorize the large volume of information in the selected articles.

Consider the following columns for the literature article summary table (or customize as appropriate): author and article reference, study design and devices used, follow-up, and miscellaneous comments and findings. A final column should be specific to the literature review objectives (e.g., reported adverse events). See Table I for an example of a literature article summary table.

Prepare a Data Analysis Spreadsheet

The next step is to develop a data analysis spreadsheet (see Table II). The spreadsheet is critical to ensure that the data are consistently categorized to allow for an apples-to-apples comparison. Such a comparison can be quite challenging considering the inconsistent reporting methods that are used by various authors. When necessary, use footnotes to explain unusual or unique circumstances and any special data-handling methods used. For example, if an article reports the percentage of events and all other articles report a quantity, it may be necessary to calculate the quantity for comparison purposes. A footnote can clearly document that the quantity was calculated and to show exactly how the values were derived.

If one article does not address a category that is described in other articles, enter NR (not reported) in the data field rather than entering a zero. It is not reasonable to assume that zero events occurred just because the event was not mentioned in the article; the protocol may not have required observation and recording for that particular event.

It is possible that articles will need to be excluded during the data analysis process. If so, explain the rationale for exclusion in the literature review report. It is likely that the summary table created in the previous step may require changes and updates based on information learned during this step.

Create Report Tables

Analyze the objectives defined in the literature review plan and determine what data should rise to the level of report tables in the literature review report. Give special consideration to the categorization of data that would be most beneficial for the particular product or subject of the literature review. For example, with stent products it may be beneficial to categorize the data by drug-eluting stents and bare-metal stents. Several tables may be required, depending on the complexity of the literature review. See Table III for an example report table.

Write the Report

If a systematic approach has been used, writing the report is relatively easy. There are some format and content guidelines for the literature review report including purpose, objectives, review method, search results, device description, analysis, and conclusion.

Purpose. Rewrite the purpose statement from the literature review plan using past tense (e.g., The purpose of this literature review was to evaluate the risks associated with coronary implant devices to support development of the XYZ Company's Alpha device).

Objectives. Rewrite the objectives from the literature review plan using past tense (e.g., This literature review attempted to answer the following questions: What is the rate of death associated with coronary implant devices? What other serious adverse events are reportedly associated with coronary implant devices? What is the reported incidence of adverse events?).

Literature Review Method. Define the methods used for obtaining the literature that was reviewed. The information in this section should be similar to what was defined in the literature review plan. Describe the actual methods used and provide rationale for any deviations from the plan. Include information such as the data sources, the search terms included, and the search terms excluded.

Search Results and Article Exclusions. Provide a summary of the searches that were performed, the number of results, and the number of articles excluded. Also, provide a reference to the appendix where the search results documentation will be located, along with the rationale for article exclusions. As previously mentioned, it is helpful to develop a simple coding system that can be used to document article exclusion rationale directly on the search results list(s).

Device and Product Description. Provide a brief description of the device or product referred to in the purpose section. Include intended use and principles of operation.

Analysis of Reviewed Literature. This is the heart of the literature review report. This section should provide a comprehensive summary and analysis of the information discovered in the literature review process, both favorable and unfavorable, and it should address each of the literature review objectives. The report tables that were created in the previous step of this procedure provide the outline for writing this section. Write a brief introductory paragraph for each table. Below the table, discuss the most notable points from the table. For example, describe the overall rate and range of events, and then discuss the articles that reported the greatest number of events. If relevant, mention articles that reported the least number of events. Provide author quotes and references, where appropriate. Rather than just reproducing the literature, write this section with a natural flow that tells the story of an unbiased and critical analysis of the literature.

Conclusion. Provide a brief summary to indicate how the objectives of the literature review have been met. If appropriate, include a top-level data summary table.

Appendices. Suggested appendices for the literature review report include

Report Approval. The literature review report should be signed by the author, or if it was ghost-written, by the report owner. In either case, it is important that the report approver is adequately qualified and has demonstrated objectivity. (Refer to MEDDEV 2.7.1 for guidance.)

Trust the Process

Literature reviews are complex projects that require a lot of time and effort. But these projects can also be enjoyable because they involve research, discovery, learning, and focus. In addition, literature reviews result in a valuable treasure trove of information for both medical device developers and manufacturers.

When the next literature review project shows up on the product development project plan or the department action list, follow the steps of this systematic approach. The project may seem overwhelming at first, but relax and trust the process. Take one small step at a time to break the challenge into manageable tasks, and plan to cross the finish line of this marathon project with the ultimate deliverable: a comprehensive literature review report.

Barb Danson is founder of QWS Consulting LLC (Maple Grove, MN). She can be reached via e-mail at [email protected].

Copyright ©2007 Medical Device & Diagnostic Industry