Deciphering the system of a systematic review.

All clinical decisions and research essentially begins with the review of pre-existing literature. It helps to make our clinical decisions based on an overview of all the literature concerning the particular clinical problem. This requires a lot of time and effort, which is becoming scarce in this day and age. Literature reviews shorten the time by offering relevant evidence in a concise form. However, narrative reviews might not be objective and unbiased. This drawback is overcome by systematic reviews. There is still some apprehension regarding undertaking systematic reviews due to lack of information. This article is an attempt to explain the concept of systematic reviews and methods used for conducting the same, which will be helpful in day-to- day clinical practice as well as research.

INTRODUCTION

As dental practitioners we often face clinical decisions regarding a patient's oral health. It is important that these decisions incorporate the best available scientific evidence in order to maximize the potential for successful patient care outcomes. This has led to the concept of “evidence based dentistry (EBD),” which is defined by the American Dental Association as follows:

“EBD is an approach to oral health-care that requires the judicious integration of systematic assessments of clinically relevant scientific evidence, relating to the patient's oral and medical condition and history, with the dentist's clinical expertise and the patient's treatment needs and preferences.”[1]

An essential step in this approach is to isolate quickly and efficiently the best current literature that purports to answer our clinical problems. However, there is a plethora of scientific literature available and too little time to process all that information making it difficult to keep abreast of all the scientific literature. This requires that various primary research studies should be summarized in the form of reviews so as to provide clear and concise information, which will help in decision making.

Traditionally, reviews have been narrative by nature or commentaries within the literature. These have been defined as attempts to synthesize the results and conclusions of two or more publications on a given topic.[2] These reviews tend to be descriptive in nature and may show several biases such as selection bias,[3] language bias,[4] publication bias[4] etc. It is also seen that different reviewers reach different conclusions from the same evidence due to lack of rigor in the selection of studies. The drawbacks of these reviews led to the concept of systematic reviews. According to Sackett et al.[2] a review may be termed as systematic review when it strives to comprehensively identify and track down all the literature on a given topic. Systematic review is defined by Center for Evidence Based Medicine glossary of terms as the application of strategies that limit bias in the assembly, critical appraisal and synthesis of all relevant studies on a specific topic.[5]

KEY CHARACTERISTICS OF A SYSTEMATIC REVIEW

A clearly stated set of objectives with pre-defined eligibility criteria for studies;

An explicit, reproducible methodology;

A systematic search that attempts to identify all studies that would meet the eligibility criteria;

An assessment of the validity of the findings of the included studies; and

A systematic presentation and synthesis of the characteristics and findings of the included studies.[6]

A SYSTEMATIC REVIEW MAY BE DIFFERENTIATED FROM A NARRATIVE REVIEW IN HAVING THE FOLLOWING CRITERIA

The research question is clearly formulated and focused.

Comprehensive and explicit search is carried out as per pre-defined inclusion and exclusion criteria.

The studies included are critically appraised using defined guidelines such as Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA).

Quantitative data synthesis.

The reasoning is based on scientific evidence.[7]

ADVANTAGES OF A SYSTEMATIC REVIEW

The tremendous amount of scientific evidence is summarized thus reducing the time required to reach a clinical decision.

End confusion by synthesizing the results of all the studies.

Yield new insights and highlight small, but important effects of a treatment or intervention.

The chances of duplicating research are less if the previous studies have already been analyzed to reach a meaningful conclusion. This helps to save the resources as well as time of researchers.

STEPS IN A SYSTEMATIC REVIEW

The steps in conducting a systematic review are summarized in Figure 1 and described below.

Figure 1

Steps in conducting a systematic review

Formulation of the review question

The first stage consists of formulation of the review question and a review title. The review title should be short yet descriptive enough to reflect research question. It should be possible to clearly define the research question as well as the title in terms of PICO[8] principle, i.e., there should be a clearly defined population (P) upon whom the intervention (I) is to be tested along with comparable intervention (C) with a measurable outcome (O).

The research question should be relevant to the researcher as well as the patient either directly or indirectly. E.g., if a parent wants to know whether toothpaste X is better than toothpaste Y for his child's teeth, the research question becomes “is toothpaste X (I) more effective when compared with toothpaste Y (C) in prevention of dental caries (O) in children (P)?”

The PICO approach holds true for all the questions that are quantitative in nature. In case of qualitative research, as there is no intervention or comparison, our question will consist of only the “P” and “O.”

Development of the search protocol

It is essential that the search protocol should be finalized at the very beginning of undertaking a systematic review. The review team should be constituted at this stage comprising of a minimum of two researchers. This enables one to minimize bias and error at all stages. Also, help should be enlisted from other health-care professionals, researchers, reference librarians, clinical and methodological experts etc.[9] The search strategy, inclusion and exclusion criteria, data extraction, quality assessment of selected studies, data analysis and plans for dissemination of the review should be addressed while developing the protocol.

Literature search

A systematic search for research is one of the major differences between a traditional literature review and a systematic review. For a systematic review both the published and unpublished literature is carefully searched for the required studies. The aim is to identify as many studies on the topic of interest as is reasonably possible. To aid in this, a comprehensive search strategy is developed and documented in the review protocol prior to commencement.[10] This should include the number of researchers who will screen titles and abstracts and then full papers and method for resolving disagreements about study's eligibility. Studies are obtained by using one or more of the following methods.

Searching electronic data bases

Initially a designated number of databases such as MEDLINE, EMBASE and COCHRANE are searched using standardized or customized search filters.[11]

Scanning reference lists of relevant studies

The reference list of all retrieved papers is searched to identify any additional studies of interest missed during database searches.[11]

Hand searching key journals

This involves scanning the content of journals, conference proceedings and abstracts, page by page. It is an important way of identifying very recent publications that have not yet been included and indexed by electronic databases and also of including articles from journals that are not indexed by electronic databases.[12]

Citation searching

This involves selecting a number of key papers already identified for inclusion in the review and then searching for articles that have cited these papers. This approach helps to identify a cluster of related and therefore highly relevant, papers.[11]

Searching for unpublished literature

This helps minimize publication bias, which is often seen to result from the tendency that research showing a positive outcome is more likely to be accepted and published in journals than research that fails to demonstrate any benefit. Using only published studies may therefore overestimate the effect of the intervention.[4] It is difficult to get access to this kind of literature because of lack of public record of their existence.[10]

Searches of specialized search engines, data bases or websites such as those listing conference proceedings, institutional or technical reports and higher degree dissertations or other documents not normally subject to editorial control or peer review may help uncover some of this literature.

Contacting experts and manufacturers

Research groups and other experts as well as manufacturers may be useful sources of research not identified by the electronic searches and may also be able to supply information about unpublished or ongoing research.[11]

Searching non-English language literature

Limiting searches to English language papers can introduce language bias in the systematic review.[13141516] Databases, such as MEDLINE, EMBASE, do include a small number of non-English language journals that can be assessed.[17] Furthermore, using additional databases such as Latin American and Caribbean Health Sciences Literature that contain collections of non-English language research can minimize potential language bias.

Whenever feasible, all relevant studies should be included regardless of language.[10] However, realistically this is not always possible due to a lack of time, resources and facilities for translation. It is advisable therefore, to identify all non-English language papers, document their existence, but record “language” as the reason for exclusion in cases where they cannot be dealt with.[11]

Assessment of studies

Literature searching results in a large number of potentially eligible studies that need to be assessed for inclusion against predetermined criteria. The aim of selection is to ensure that all relevant studies are included in the review. The process for selecting studies should be explicit and conducted in such a way as to minimize the risk of errors and bias.[111819202122] It should be documented clearly to ensure reproducibility.

Process for study selection

The selection of studies from electronic databases is usually conducted in two stages:

An initial screening of titles and abstracts against the inclusion criteria to identify potentially relevant papers.

Full text papers are obtained for those that meet the inclusion criteria.

Some searching methods provide access to full papers directly, e.g., hand searching journals and contact with research groups; in such cases, assessment for inclusion is one stage process.

Study quality assessment

After a full text selection stage, the studies are assessed for methodological rigor. Whatever the study design(s) included, it should not be assumed that all studies of the same basic design are equally well-conducted. It is essential to evaluate the methodological quality of a potentially eligible study before it is included as including studies of poor methodology might distort its conclusion.[10]

Parallel independent assessments should be conducted to minimize the risk of errors. If disagreements occur between assessors, they should be resolved according to a predefined strategy using consensus and arbitration as appropriate.[23] The study selection process should be documented, detailing reasons for exclusion of studies that are “near-misses.”

Reporting study selection

Showing the number of studies/papers remaining at each stage by a flow chart is a simple and useful way of documenting the study selection process e.g., PRISMA flow chart[24] [Figure 2]. It is essentially an update of QUORUM or quality of reporting of meta-analyses (1966). The PRISMA statement consists of a four phase flow diagram [Figure 2] and a 27 item checklist [Figure 3]. It helps authors to improve the reporting of systematic reviews and meta-analyses, minimizes the chances of repetition of reviews addressing the same question and ensures greater transparency in updating a review. It is also useful for critical appraisal of a systematic review which has already been published.[24] The PRISMA statement, checklist and flowchart as well as more information on the same are available freely from its website www.prisma-statement.org.

Figure 2

Preferred Reporting Items for Systematic Reviews and Meta-analyses flowchart

Figure 3

Preferred Reporting Items for Systematic Reviews and Meta-analyses checklist

Data extraction

Data extraction is the process of obtaining the necessary information about study characteristics and findings from the included studies. Data extraction forms should be designed and developed with both the review question and subsequent analysis in mind.[11]

Standardized data extraction forms provide consistency in a systematic review, thereby potentially reducing bias, improving validity and reliability.[25] Use of an electronic form has the added advantage of combining data extraction and entry into one step and facilitates data analysis and the production of results tables for the final report.

Ideally two researchers should independently perform the data extraction. As a minimum, one researcher should extract the data with a second researcher independently checking the data extraction forms for accuracy and detail. If disagreements occur between assessors, they should be resolved according to a predefined strategy.[11]

Data synthesis

The aim of this phase of the review is to synthesize the findings from individual studies in order to provide an overall estimate of the effectiveness, feasibility, appropriateness and meaningfulness of the intervention or activity. Synthesis should also explore whether observed intervention effects are consistent across studies and investigate possible reasons for any inconsistencies.

Depending upon the type of data within the review, the type of evidence synthesis is chosen e.g., synthesis can be carried out quantitatively using the formal statistical techniques such as meta-analysis, or if formal pooling of results is inappropriate, through a narrative approach.[11]

Dissemination of information

COCHRANE database publishes systematic reviews quite frequently. However, these are often quite detailed. A summary of the findings of the review should also be published in various relevant journals as well as simplified versions should be available for patients.

CONCLUSION

With increasing focus on generating guidance and recommendations for practice through systematic reviews, health-care professionals need to understand the principles of preparing such reviews. Here, we have attempted a brief step-by-step explanation of the principles.