Towards evidence-based diagnosis in developing countries: the use of likelihood ratios for robust quick diagnosis.

Evidence-based medicine (EBM), a relatively new paradigm for clinical practice, stresses the use of research evidence in diagnostic evaluations and therapeutic interventions. Financial and instrumental scarcities in developing countries require cliniciansto visit patients under time constraints, especially in outpatient clinical settings. In this situation, clinicians need diagnostic approaches that reduce both diagnostic time and errors. This article discusses what EBM can do to help physicians in this regard. For quick history taking and physical examination, all physicians utilize certain "key pointers" (signs or symptoms or paraclinical tests that influence the pretest estimation of the disease prevalence). EBM emphasizes that these key pointers are nothing but signs or symptoms with significant likelihood ratios. Likelihood ratios are a practical means of interpreting clinical tests; physicians can derive likelihood ratios from critically appraised studies. The use of clinical tests with sizeable likelihood ratios and with likelihood ratios for key pointers from independent body systems may significantly decrease both diagnostic time and errors. EBM could be a significant aid to physicians in the developing world.

Evidence-based medicine (EBM) is the process of systematically finding, appraising, and using contemporaneous research findings as the basis for clinical decisions.1,2 Evidence-based physicians ask questions, find and appraise the relevant data, and exploit that information for everyday clinical practice.2 Although there is an increasing awareness of and tendency towards use of evidence-based practice among Middle Eastern physicians,3,4 research consistently has shown that clinical decisions are rarely based on the best available evidence.5,6

Time is a problem for any veteran clinician,7 and one of the main reasons for the underuse of EBM in the Middle East.8 In this review, we explain how the use of a particular parameter can help the physician to limit the amount of time spent in history taking and physical examinations, and reduce errors.

Outpatient settings in developing countries

International statistics reveal significant differences between industrialized nations and underdeveloped countries regarding health care resources.9,10 In the Middle East, for instance, total health expenditure per capita in the year 2001 was US$591 and US$422 for Saudi Arabia and Iran, respectively, compared with US$4887 and US$2567 for the United States and France, respectively.11 The populations per physician were 715, 953, and 182 in Saudi Arabia, Iran, and the United States, respectively.11 Given these and other scarcities (such as physician migration),12 even the least optimal visit lengths are not attainable in the outpatient settings of developing countries.13

Several studies from industrialized countries have analyzed the impact of visit time on different elements of patient care quality (such as patient satisfaction,14,15 physician satisfaction,16 prescribing practices,17 and risk of malpractice claims18). These problems are more evident in developing countries because of the faulty infrastructure of the health care systems.19 One of the most troubled of clinical skills in developing countries is the diagnostic approach.

Evidence-based decision making

From the decision-making standpoint, clinical diagnosis is opinion revision with the imperfect information derived from clinical investigations. The standard rule for this task is the Bayes’ theorem, the core foundation of evidence-based decision-making.20 Named after Thomas Bayes, the eighteenth century originator of ideas about conditional probabilities, Bayesian methods are now increasingly important in medical diagnoses.21,22 Conditional probability applies when the probability of an event is dependent on the occurrence of another related event. For instance, the probability of infection in a patient is dependent on the presence or absence of fever. Bayes’ theorem tells us how to update our prior estimation of the presence of something (e.g. probability of a disease) when new data becomes available (e.g., a clinical or paraclinical test). From this aspect, two parameters play the central role in decision-making about the diagnosis of a disease in a patient: pretest probability and the accuracy of clinical tests.1

Pretest probability

Without any additional information, pretest probability equals the prevalence of a disorder. If more information becomes available, then the probability of having or not having a disease (post-test probability) may change. Test information is the information derived from clinical investigations (such as history questions, physical examinations, and paraclinical tests). The point is that for estimation of the pretest probability, the clinical circumstances should be considered. Suppose that you want to estimate the pretest probability of pulmonary emboli for a patient with shortness of breath and nonspecific chest pain. Is this probability equal for a 78-year-old woman 10 days after a surgery and a 28-year-old man who is experiencing a high level of anxiety? Certainly not. Our clinical estimates about the probability of pulmonary embolism as the explanation for these two patients’ complaints are very different. In the older woman, the probability is high; in the young man, it is low. Consequently, even if both patients have equal results in subsequent tests, the post-test probability would still be different and management is likely to differ.23

However, estimation of pretest probabilities in clinical settings is not that straightforward. Research has shown that clinician estimates of probability vary widely and are often inaccurate.24–27 Experts remind us of common errors in estimation of pretest probability and the need for utilization of research evidence to reach unbiased estimates.28 For instance, physicians tend to overestimate probability in recently seen cases or in diseases with novel clinical features. Likewise, a physician who has once missed a rare disease will consider it in the upper levels of his differential diagnosis for similarly presented patients, at least for a time (being burned by a missing case). The good news is that clinicians’ pretest probabilities can become evidence based. In an inpatient medical service of a university-affiliated hospital almost all of the patients admitted for diagnostic evaluation during a period of 3 months had clinical problems for which evidence is available to guide our estimates of pretest probability.29

Likelihood ratio

Many of the questions that novices use in their clinical practice are of little help in changing the pretest probability of disease. Experts, especially in situations under time constraint, use “key pointers” to reach to a diagnosis quickly.30 Typically, these pointers (which could be signs, symptoms, or even paraclinical tests) have good predictive power, thus changing the pretest probability of diseases. EBM emphasizes that key pointers for decision-making in clinics are tests with significant likelihood ratios.1,23

Sensitivity (the proportion of patients with a positive test result) and specificity (the proportion of healthy subjects with a negative test result) are familiar to most physicians.31 However, application of these parameters is somewhat problematic in clinical situations as they run counter to our usual diagnostic approach (we want to know if someone with positive test result has the disease, not if someone with the disease has a positive test). This problem could be resolved by use of positive or negative predictive values, as these indicate the proportion of test-takers with positive or negative results that have or do not have the disease. These are the most practical test characteristics for use in clinical practice, but their use is limited due to their dependence on the prevalence of the disease (values derived from a study in one clinical setting cannot be generalized to other settings as the prevalence is usually different). In this context, likelihood ratios are preferred as measures of the accuracy of tests.31

Likelihood ratios indicate how many times more likely a test result is in a patient with the disease compared with a person free of the disease.32 Likelihood ratios can be easily obtained from diagnostic studies (the likelihood ratio for a positive test [LR+] is sensitivity/1-specificity and the likelihood ratio for a negative test [LR−] is 1-sensitivity/specificity). Likelihood ratios are preferred to the traditional parameters in that they can be applied straightforwardly for calculation of post-test probabilities in a series of tests.33 Using simple formulas or a nomogram, one can convert the estimated probability of the suspected diagnosis before the test result is known (pretest probability) into a post-test probability, which takes the result into account.1 For instance, the LR+ for the presence of third heart sound for diagnosis of myocardial infarction (MI) is shown in Table 1 to be 3.2.34 This means that patients with MI are 3.2 times more likely to have a third heart sound than suspected patients without MI. To reach the posttest estimate of MI in this instance, one can simply multiply the likelihood ratio by the pretest estimate of the disease, which increases it 3.2 times. Similarly, in a patient with pleuritic chest pain multiplication of the likelihood ratio by the pretest probability will decrease the probability of the presence of MI to 0.2 of pretest probability. In a patient with both a third heart sound and ST segment elevation, one can multiply the pretest estimate of MI by 35.2 (that is 3.2 × 11) to reach the posttest estimate. Application of such tools in clinical practice would significantly aid physicians by reducing the time to diagnosis and by more accurately estimating the presence of disease.

Table 1

Positive likelihood ratios of different clinical and electrocardiological signs for the presence of myocardial infarction in suspected patients (adapted from reference 34).

Test	Likelihood ratio (95% CI)
Radiation of pain to left and right arm	7.1 (3.6–14.2)
Third heart sound	3.2 (1.6–6.5)
Pulmonary crackles	2.1 (1.4–3.1)
Pleuritic chest pain	0.2 (0.2–0.3)
Sharp or stabbing chest pain	0.3 (0.2–0.5)
Positional chest Pain	0.3 (0.2–0.4)
Any ST segment elevation	11.0 (7.1–18)
Any ST segment depression	3.2 (2.5–4.1)

Two important points are first, that physicians can employ a nomogram in place of lengthy calculations for the estimation of posttest probabilities.23 Using a nomogram, one can easily estimate posttest probability in the least possible time. The nomogram shown in Figure 1 is easy to carry and have available at the point of care. It is composed of columns for pretest and posttest probabilities on the sides with a column for likelihood ratios in the middle. One can simply connect the estimate of pretest probability to the likelihood ratio of the test by a ruler and read the posttest probability in the third column. In the figure, a test with a likelihood ratio of 20 will increase a pretest probability of 10% to a posttest probability of 70%.32

Figure 1

A nomogram for calculation of posttest probability (adapted from reference 39).

Second, physicians can use the likelihood ratio as a basis for decision-making about using the test results. Generally, tests with likelihood ratios higher than 10 or lower than 0.1 can increase or decrease pretest probabilities significantly and tests with likelihood ratios between 2 and 0.5 have no important impact on pretest probabilities. Likelihood ratios of 5–10 (or 0.2–0.1) generate moderate shifts in pre- to posttest probabilities and likelihood ratios of 2–5 (or 0.5–0.2) generate small (but sometimes important) changes in probabilities.23 In the above example, a patient with both a third heart sound and ST segment elevation (likelihood ratio=35.2) should be considered an MI patient with any given pretest probability. However, decision-making about a patient with a third heart sound alone depends on pretest probability, as pretest estimates of about 1% and 10% will be increased to about 3% and 30%, respectively, which warrant different approaches.34

Another imperative point, highlighted by EBM, is the concept of independence of the tests when searching for different signs and symptoms in a multi-organ disease.1,23,28 Two tests (or signs or symptoms) are considered independent if the likelihood ratio for all combinations of results is the product of the likelihood ratio for the result on the first test multiplied by the likelihood ratio of the result on the second test. Generally, clinical tests related to a single body system are considered highly dependent.1 Suppose that, considering hypothyroidism as an example, likelihood ratios for change in the speed of thinking, memory access, and difficulty of mathematics during the previous year are known to be 2.5, 2.6, and 5.4, respectively.35 Could we multiply our initial estimation of odds of hypothyroidism by 35.1 in our patient with all symptoms? Of course not. These neurological symptoms are highly related and the magnitude of the presence of all may differ very little from the presence of each alone. In contrast, the addition of another symptom from an unrelated system (like dry skin, with a likelihood ratio of 2.0)35 to any of the aforementioned symptoms could double our estimate of hypothyroidism.

Hence, omission of unnecessary ‘dependant’ questions could help physicians to significantly reduce the time and error of history taking. In other words, in approaching any multi-system disease, it is better to use a variety of questions with significant likelihood ratios from ‘independent’ systems rather than many questions from one or two systems. As Elstein and Schwarz have argued, it is possible “for a clinician to be too economical in collecting data and yet to interpret accurately what is available.”20

Applications

The time taken in history taking in outpatient clinics of developing countries can be shortened with efficient use of signs and symptoms with significant likelihood ratios and with likelihood ratios for indicators from independent body systems. Errors can be reduced by derivation of likelihood ratios from valid research. Students need to become more familiar with concepts of pretest probability and likelihood ratios and should be reminded to search and memorize clinical signs with significant and independent likelihood ratios. Medical curricula need to change so that inpatient complete history taking at the beginning is gradually substituted with instruction on a briefer outpatient approach in the late internship.36 Although initially it may seem time-consuming, eventually the use of this approach will significantly decrease the time of history taking.

Veteran clinicians classically reach to their key pointers via a process of trial and error over the course of their long practice.30 However, two major concerns in this regard are first, there is no guarantee that all physicians will get to the stage of using these pointers, and second, experience and heuristics usually involve diagnostic and cognitive biases.37 Many of the key pointers used by experts may be far less useful considering their positive or negative likelihood ratio.1 For instance, a positive rebound sign for diagnosis of acute appendicitis, which many clinicians use as the hallmark of the disease, is shown to have a positive likelihood ratio of 1.9.38 As mentioned before, tests with a positive likelihood ratio higher than 10 or with a negative likelihood ratio lower than 0.1 are powerful key pointers, but as the likelihood ratio of a test (either positive or negative) trends toward 1, the test loses the power to change pretest probability, and hence, becomes useless.1

Summary

Evidence-based medicine can be taught to and practiced by clinicians at all levels of seniority and can be used to close the gap between good clinical research and clinical practice. To decrease diagnostic time and errors during history taking and physical examination, physicians need to use evidence-based pretest probabilities and tests with significant and independent likelihood ratios. Although initially, EBM seems to take more time, in the long run it can actually help in the developing world physicians struggle against time.