Evidence based medicine in pediatric practice: brief review.

Practicing medicine according to the best evidence is gaining popularity in the medical societies. Although this concept, which is usually called Evidence Based Medicine (EBM) has been explained in many resources, it has not been addressed enough in pediatrics. In this review, we briefly explained Evidence Based Medicine approach and its applications in pediatrics in order to help the pediatricians to efficiently integrate EBM into their daily practice.

Introduction

Evidence Based Medicine (EBM) is integrating the best available evidence in the medical literature into the daily medical practice[1, 2]. This term was first introduced by Guyatt in 1991 and consists of systematic search of the literature for the best available evidence, critical appraisal of the retrieved literature, applying the results in the medical practice and finally evaluating the efficacy of the approach[3].

Due to a large volume of published medical literature, the task of finding the best evidence is very time consuming and is not considered cost effective by many. It is reported that inadequate time is the main obstacle for incorporating the best evidence into the daily medical practice[4, 5].

In Evidence Based Medicine (EBM) practice, a standard method is used for searching the literature and appraising the found evidence. This approach can significantly decrease the time needed for incorporating the evidence into the daily practice.

EBM has four steps which are shown in Table 1. These steps can make the final results more reproducible compared to the traditional practice in medicine and actually this is the main difference between these two approaches.

Table 1

Steps of Evidence Based Medicine practice

Step I: Asking an answerable question

Step II: Searching for the best evidence

Step III: Critical appraisal of the found literature

Step IV: Applying evidence in the daily practice

Applying EBM in pediatrics is growing and the pediatricians are getting more involved with this concept, however this figure is far from perfect[6, 7]. There are several resources (including web-based medical literature) for EBM practice in pediatrics which we explain in brief in this review. We also briefly reviewed the steps of EBM. The main focus of our review would be clinical trials.

1) Asking an answerable question (PICO approach)

The cornerstone of EBM is framing an answerable question. If this question is not structured well, the other steps will not be successful. Having this fact in mind, especial attention should be placed in this step[8, 9].

Asking an answerable question is putting our clinical question in a frame which can be used in the search engines in a simple manner.

Classically the clinical question is divided into four parts: 1- Patients or Population which is referred to the specific group the patient or population belong to. 2- Intervention which is the kind of treatment or procedure we want to know its efficacy. 3- Comparison which is the standard treatment for comparing the effect of Intervention. 4- Outcome which is the intended outcome we expected to have by our intervention. “PICO” is the acronym for this 4 part question which consists of the first letters of Patients, Intervention, Comparison and Outcome.

Example

Assume that you are working in a general hospital as a pediatrician. A 2.5 year old boy presented with acute non-bloody diarrhea. You want to know if probiotics such as Lactobacilius species are effective in decreasing the duration of diarrhea compared to oral re-hydration treatments (ORT).

The PICO format question of this clinical scenario is depicted in Table 2. It is recommended to structure the final question in a single sentence.

Table 2

PICO question of the patient

Patient or Population	A 2.5 year old boy with acute non-bloody diarrhea
Intervention	Probiotic therapy with Lactobacillus species
Comparison	Oral Re-hydration Therapy (ORT)
Outcome	Decreased duration of diarrhea
Question	In a 2.5 year old boy with non-bloody diarrhea, how effective is probiotic therapy with Lactobacillus species compared to Oral Re-hydration Therapy (ORT) for decreasing the duration of diarrhea?

2) Searching for the best evidence

There are several resources (mostly web-based) which can be very effective in searching for the best available evidence. An efficient search strategy is indispensable for pediatricians to save time and energy. It is recommended by Haynes to categorize the medical resources according to a hierarchy[10, 11].

This hierarchy from bottom to top consists of “Studies”, “Syntheses”, “Synopses”, “Summaries”, and “Systems” (Fig. 1). This is usually called “5S” hierarchy. When one moves from “studies” towards “Systems”, the quantity of the evidence would decrease and the quality would increase.

Fig. 1

The hierarchy of evidence according to Haynes RB. (Reproduced with permission from the publisher)[10]

Individual “Studies” are at the bottom of the hierarchy. Searching in this field is time-consuming and needs critical appraisal of retrieved articles. Several databases regularly index the individual studies. The most famous databases are Medline's pubmed (www.ncbi.nlm.nih.gov/pubmed), SCOPUS (www.scopus.com), and ISI web of knowledge (www.isiknowledge.com). Only Pubmed is freely available online. For academic pediatricians, the other two databases are also available through Athens system.

For searching these databases, the PICO question should be structured with Boolean operators. For example the PICO-based search of the example mentioned above would be something as follows: (Child OR Toddler OR Pediatric) AND (Diarrhea) AND (Probiotic OR Lactobacillus) AND (Duration OR frequency).

This search yielded 102 articles in Pubmed on 20 Sep 2009, which shows that many studies are available in this regard. However reading this large amount of articles is very time-consuming and the quality if each article is not apparent either.

An easy way to decrease the amount of literature is using the limit section of Pubmed. For example limiting the above-mentioned search to “clinical trials” would yield only 59 studies. Another useful tip in searching is “Clinical Queries” section of Pubmed services. Using this service for our search yielded 57 studies. As is obvious, the number of studies is still large.

Complete explanation of searching in the medical databases is beyond the scope of this review. Readers can consult several articles on this topic[12-15].

“Syntheses” are in the next level of evidence hierarchy. This level contains systematic reviews of special medical topics. Systematic review is a kind of review which systematically searches for all primary studies considering a specific clinical scenario, appraises, and summarizes them. Meta-analysis is a kind of systematic review which quantitatively summarizes and analyses all relevant studies[8]. Although databases such as Medline and SCOPUS index systematic reviews, the most efficient way to find these articles is databases dedicated to systematic reviews. The Cochrane library (www.cochrane.org) is the most famous in this regard.

The Cochrane Library has three sections. The first section is Cochrane Database of Systematic Reviews (CDSR). This part indexes the systematic reviews and meta-analyses which are prepared by Cochrane groups themselves. CDSR is the gold standard for systematic reviews. The second section is Database of Abstracts of Reviews of Effects (DARE). This is the database for the other systematic reviews and meta-analyses published in other journals. Systematic reviews in DARE need critical appraisal since the quality of them is not assured completely by Cochrane library. The third is Cochrane Central Register of Controlled Trials (CENTRAL). This is an international collection of Randomized Controlled Trials (RCT). The Cochrane library is most efficient for treatment or intervention of clinical questions.

“Synopses” are the next level of evidence hierarchy. These are synopses of original studies and systematic reviews which usually contain a commentary by an expert on the study results. Examples of this level of resources are DARE section of Cochrane library, ACP Journal Club (www.acpjc.org), and bandolier (www.medicine.ox.ac.uk/bandolier/index.html).

In contrast to the previous medical resources, “Summaries” gather information regarding several aspects of a clinical question and summarize “synopses”, “syntheses”, and “studies” on a health condition. “Summaries” usually consider all management options which can be found in the literature. The best examples of “summaries” are clinical guidelines.

www.guidelines.gov is a freely available website which provides numerous clinical guidelines. NHS clinical knowledge summaries (www.cks.nhs.uk/home) and EBM textbooks [such as uptodate (www.uptodate.com)] are other examples in this regard.

“Systems” are on the top of the evidence hierarchy. A'System“ would integrate and summarize all evidence about a clinical scenario and automatically link a specific patient's condition to the relevant information. For more details, the reader can consult Haynes article on this topic[10].

Where to find the best evidence

When searching for the best available evidence, one should search the top of the hierarchy first and in case of unsuccessful search, should move toward the bottom. This approach is very time and cost saving. If you are not familiar enough with resources mentioned above, several evidence based ”federated search engines" are available online[10]. Two examples of these search engines are TRIP database (www.tripdatabase.com) and SUMsearch (http://sumsearch.uthscsa.edu). Table 3 shows some useful websites which provide evidence based resources.

Table 3

Some free useful online resources for evidence based medicine

Name of the site	Internet address	Type of resource
Bandolier	www.medicine.ox.ac.uk/bandolier/index.html	Synopsis
National guideline clearinghouse	www.guidelines.gov	Summary
Tripdatabase	www.tripdatabase.com	Search engine
Sumsearch	http://sumsearch.uthscsa.edu	Search engine
Cincinnati Children's Hospital Medical Center	http://www.cincinnatichildrens.org/svc/alpha/h/health-policy/ev-based/default.htm	Summary
University of Michigan
Department of Pediatrics	www.med.umich.edu/pediatrics/ebm/	Summary
Evidence-Based Pediatrics Web Site
PedsCCM: The Pediatric Critical Care Website	http://pedsccm.org/EBJournal_Club_intro.php	Synopsis
Resources for Practicing Evidence-Based Medicine	www.pedsccm.org/EBJournal_Club_intro.php	Summary
National library guideline	www.library.nhs.uk/GUIDELINESFINDER/	Summary
Pubmed clinical queries	www.ncbi.nlm.nih.gov/corehtml/query/static/clinical.shtml	Search engine
The NHS Clinical Knowledge Summaries	www.cks.nhs.uk/home	Summary

If you were forced to search among individual studies (due to lack of resources on the top of the hierarchy), you will need to know how to appraise the individual studies and systematic reviews. The next part of this review briefly explains this issue.

3) Critical appraisal of the literature

Published studies are not always of high quality. It is recommended to read the full text of the found articles since the abstracts can be misleading and sometimes do not contain necessary data[16, 17]. For each article a level of evidence can be assigned and only studies with highest level of evidence should be considered for application in our clinical situation[12, 13, 18]. Oxford center for evidence based medicine has issued tables for levels of evidence of various study designs[19]. The levels of evidence for studies of therapeutic design are available in Table 4.

Table 4

Oxford levels of evidence for therapeutic study designs#

Level	Therapy/Prevention, Etiology/Harm
1a	Systematic review (SR) (with homogeneity*) of Randomized Controlled Trials (RCTs)
1b	Individual RCT (with narrow Confidence Interval)
1c	All or none†
2a	SR (with homogeneity*) of cohort studies
2b	Individual cohort study (including low quality RCT; e.g., <80% follow-up)
2c	“Outcomes” Research; Ecological studies
3a	SR (with homogeneity*) of case-control studies
3b	Individual Case-Control Study
4	Case-series (and poor quality cohort and case-control studies‡)
5	Expert opinion without explicit critical appraisal, or based on physiology, bench research or “first principles”

Reproduced with permission from Oxford levels of evidence for therapeutic study designs[19]

Homogeneity of a systematic review means that it is free of worrisome variations (heterogeneity) in the directions and degrees of results between individual studies. Not all systematic reviews with statistically significant heterogeneity need be worrisome, and not all worrisome heterogeneity need be statistically significant. Studies displaying worrisome heterogeneity should be tagged with a “-” at the end of their designated level.

Met when all patients died before the Rx became available, but some now survive on it; or when some patients died before the Rx became available, but none now die on it.

Poor quality cohort study means one that failed to clearly define comparison groups and/or failed to measure exposures and outcomes in the same (preferably blinded), objective way in both exposed and non-exposed individuals and/or failed to identify or appropriately control known confounders and/or failed to carry out a sufficiently long and complete follow-up of patients. By poor quality case-control study we mean one that failed to clearly define comparison groups and/or failed to measure exposures and outcomes in the same (preferably blinded), objective way in both cases and controls and/or failed to identify or appropriately control known confounders.

While critically appraising an individual study, two issues are to be addressed: 1- Are the results valid? 2- What are the results and are the findings important?

By answering these two fundamental questions, each study can be assigned to a specific level of evidence. Several checklists are available online for this purpose[20, 21].

Are the results valid?

The design of each study should be carefully scrutinized and if met the standards can be included in our every day practice. Several questions should be answered in this regard which can be found in Table 5. When the study design is proved to be of high quality, the next step would be to find out what actually the results are.

Table 5

Questions to be answered for assuring validity of the results of a RCT*

Was the assignment of patients to treatments randomised?
What is best?	Where do I find the information?
Centralised computer randomisation is ideal and often used in multi-centred trials. Smaller trials may use an independent person (e.g, the hospital pharmacy) to “police” the randomization.	The Methods should tell you how patients were allocated to groups and whether or not randomisation was concealed.
Were the groups similar at the start of the trial?
What is best?	Where do I find the information?
If the randomisation process worked (that is, achieved comparable groups) the groups should be similar. The more similar the groups the better it is.There should be some indication of whether differences between groups are statistically significant (ie P values).	The Results should have a table of “Baseline Characteristics” comparing the randomized groups on a number of variables that could affect the outcome (ie age, risk factors, etc). If not, there may be a description of group similarity in the first paragraphs of the Results section.
Aside from the allocated treatment, were groups treated equally?
What is best?	Where do I find the information?
Apart from the intervention the patients in the different groups should be treated the same, eg additional treatments or tests.	Look in the Methods section for the follow-up schedule, and permitted additional treatments, etc and in Results for actual use.
Were all patients who entered the trial accounted for? – and were they analyzed in the groups to which they were randomized?
What is best?	Where do I find the information?
Losses to follow-up should be minimal – preferably less than 20%. However, if few patients have the outcome of interest, then even small losses to follow-up can bias the results. Patients should also be analyzed in the groups to which they were randomized – ‘intention-to-treat analysis’.	The Results section should say how many patients were randomized (eg Baseline Characteristics table) and how many patients were actually included in the analysis. You will need to read the results section to clarify the number and reason for losses to follow-up.
Were measures objective or were the patients and clinicians kept “blind” to which treatment was being received?
What is best?	Where do I find the information?
It is ideal if the study is ‘double-blinded’ – that is, both patients and investigators are unaware of treatment allocation. If the outcome is objective (eg death) then blinding is less critical. If the outcome is subjective (eg symptoms or function) then blinding of the outcome assessor is critical.	First, look in the Methods section to see if there is some mention of masking of treatments, eg placebos with the same appearance or sham therapy. Second, the Methods section should describe how the outcome was assessed and whether the assessor/s were aware of the patients' treatment.

Reproduced with permission from Critical appraisal for therapy articles [20]

What are the results and are the findings important?

For each therapeutic study, several measures of effect size are available which are mentioned in the results section of the studies or can be calculated easily (A free software is available for this task[22]). These include: P-values, Relative Risk (RR), Relative Risk Reduction (RRR), Absolute Risk Reduction (ARR), Number Needed to Treat (NNT) and their Confidence Interval (CI). Table 6 shows the results of a putative study in a 2×2 format which we use for expressing the formulas.

Table 6

Results of a putative study in a 2×2 format

	Number of events (cured)	Number of patients without event (not cured)
Treatment Group	a	b
Control Group	c	d

Relative Risk (RR): The relative risk means how likely it is that an event will be seen in the treatment group compared to the control group (RR=b/(a+b)/d/(c+d)). An RR of 1 means no therapeutic effect. When the treatment is effective RR would be <1.

Absolute Risk Reduction (ARR): It is the absolute difference in the rates of events between the treatment and control groups (ARR=d/(c+d)-b/(a+b)). An ARR of 0 means that the treatment has no effect.

Relative Risk Reduction (RRR): This is the most commonly reported measure of treatment effects (RRR=1-RR). It is the measure of reduction in the rate of the outcome in the treatment group compared to the control group.

Number Needed to Treat (NNT): This is the number of patients one needs to treat to prevent one bad outcome or cause one additional good outcome (NNT=1/ARR). The NNT of 1 means that the treatment is effective in all patients. Treatment is less effective for NNT>1.

Confidence Interval: Providing the P-values, and the above-mentioned indices is not sufficient enough for interpretation. P-value is only a point probability that an outcome has occurred by chance and it does not provide any clue to the magnitude of effect. One solution to this problem is confidence intervals (CI) for each effect size index. When the confidence interval range is wide, usually the sample size is small and vice versa[23]. CI can be calculated by a CAT maker software which is freely available online[22]. Full explanation of this issue is beyond the scope of this article. The readers can refer to Cohen J articles for consultation[24, 25].

4) How to apply the best found evidence to our patients

The final question in evidence based medicine would be “Are the results helpful for my particular patients?” This is a very complicated matter which is intertwined with socio-economical, religious, ethical, and many other issues[26]. For example, using probiotics for treatment of diarrhea cannot be easily performed in Iran due to high cost[27]. Pain is a very important factor to be considered when treating pediatric patients. Usually less painful procedures (such as oral treatments) are preferred for children [28]. Full explanation of this issue is available elsewhere in the literature[29].