Evaluation of the performance of clinical predictors in estimating the probability of pulmonary tuberculosis among smear-negative cases in Northern Ethiopia: a cross-sectional study.

OBJECTIVES: To evaluate the performance of the predictors in estimating the probability of pulmonary tuberculosis (PTB) when all versus only significant variables are combined into a decision model (1) among all clinical suspects and (2) among smear-negative cases based on the results of culture tests.
DESIGN: A cross-sectional study.
SETTING: Two public referral hospitals in Tigray, Ethiopia. PARTICIPANTS: A total of 426 consecutive adult patients admitted to the hospitals with clinical suspicion of PTB were screened by sputum smear microscopy and chest radiograph (chest X-ray (CXR)) in accordance with the Ethiopian guidelines of the National Tuberculosis and Leprosy Program. Discontinuation of antituberculosis therapy in the past 3 months, unproductive cough, HIV positivity and unwillingness to give written informed consent were the basis of exclusion from the study. PRIMARY AND SECONDARY OUTCOME MEASURES: A total of 354 patients were included in the final analysis, while 72 patients were excluded because culture tests were not done.
RESULTS: The strongest predictive variables of culture-positive PTB among patients with clinical suspicion were a positive smear test (OR 172; 95% CI 23.23 to 1273.54) and having CXR lesions compatible with PTB (OR 10.401; 95% CI 5.862 to 18.454). The regression model had a good predictive performance for identifying culture-positive PTB among patients with clinical suspicion (area under the curve (AUC) 0.84), but it was rather poor in patients with a negative smear result (AUC 0.64). Combining all the predictors in the model compared with only the independent significant variables did not really improve its performance to identify culture-positive (AUC 0.84-0.87) and culture-negative (AUC 0.64-0.69) PTB.
CONCLUSIONS: Our finding suggests that predictive models based on clinical variables will not be useful to discriminate patients with culture-negative PTB from patients with culture-positive PTB among patients with smear-negative cases. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Methodological approach is probably limited to settings with poor performance of sputum smear microscopy and clinically suspected patients relying on chest radiography.

We evaluated the performance predictors for estimating the probability of tuberculosis when all versus only significant variables are combined into a decision model.

We did not follow-up predictive variables (or clusters of variables) of culture-negative pulmonary tuberculosis (PTB), progressing over time that may be useful in association between the clinical information and the eventual diagnosis.

Without reliable parameters for the natural course of culture-negative PTB, predictive models should be interpreted with caution.

We did not evaluate how often and to what extent uncertain diagnostic outcome is shared with patients during consultations so that they are empowered to reconsult if necessary.

Introduction

Pulmonary tuberculosis (PTB) with culture-negative, but positive clinical and chest radiographic findings is often encountered in routine practice.1 2 These cases are mostly also missed by sputum smear microscopy tests and their diagnoses have been problematic.3 4 The clinical presentation is insidious and early diagnosis is often difficult if tuberculosis (TB) is not advanced.5 6 Without a standardised clinical work-up, the rate of diagnostic error has been estimated as high as 35%–52%.7–9 However, in settings with adequate resources, allowing for comprehensive diagnostic work-up and the exclusion of other respiratory diseases, a substantial proportion of the cases are diagnosed as culture-negative PTB.2 Despite the considerable incidence of culture-negative PTB, little is known about its clinical manifestations compared with culture-positive PTB among smear-negative cases.

Recognising and diagnosing culture-negative PTB is clinically challenging. The Nguyen et al’s3 study and a Hong Kong-based study in the early 1980s4 10 found a lower frequency of cavitation in culture-negative PTB compared with culture-positive subjects. Thus, culture-negative PTB is an early disease state with low mycobacterial burden. It is proposed that the disease lies between incipient and active culture-positive PTB,11 12 a notion supported by increasing cough associated with the transition of culture-negative to culture-positive PTB.4 Although Nguyen et al3 and others4 10 observed a reduced frequency of clinical symptoms and radiographic lesions in cases with culture-negative PTB, their comparison groups (culture positive) had both smear-negative and smear-positive cases. Inclusion of smear-positive cases may potentially lead to different findings compared with a situation where the analysis is restricted only to smear-negative TB. Therefore, we hypothesise that culture-negative PTB cases may present with no significant differences in clinical and radiographic abnormalities when compared with those with culture-positive PTB among smear-negative cases.

In addition, while reviewing deaths due to respiratory illness, several authors13–15 found that missed TB diagnoses were likely due to less symptomatic states of TB premortem. Therefore, if bacteriological confirmation is needed before initiating anti-TB treatment as per the Ethiopian TB guideline,16 it will result in far too many culture-negative TB cases left untreated, as there is no reference standard test for the diagnosis.2 Moreover, the clinical definitions in existing guidelines are rather vague17 18 and do not allow to classify patients according to their probability of TB.

While application of predictive modelling in patients with clinical suspicion of PTB has been described before, these studies usually combined only independent significant predictive variables into the decision models.19 20 In addition, information from a single predictor is often insufficient to provide reliable estimates of diagnostic probabilities or risks.21 22 In particular for pragmatic reasons, clinicians in Ethiopia use all available information for each patient to determine whether TB is indeed present or absent. Therefore, in this study, we evaluated the performance of predictive models based on clinical variables for estimating the probability of PTB when all versus only significant variables are considered (1) among all clinical suspects and (2) among smear-negative cases.

Methods

Study setting, population and data collection

This study was conducted at the Ayder and Mekelle Hospitals, 500 and 350 inpatient bed public sector referral hospitals, respectively, that serve approximately 18 million people in Northern Ethiopia. From September to November 2018, a total of 426 consecutive patients admitted to these hospitals with suspicion of TB were evaluated in accordance with the Ethiopian guidelines of the National Tuberculosis Program.16 The sample size determination was made using G-Power V.3.1 software based on a priori sample size analysis for Fisher’s exact test.23 We employed a power of 83%, α error probability=0.15, 95% CI, 5% margin of error, 0.5 effect size.

The study included all patients aged ≥18 years, with a history of cough >3 weeks, night sweats, fever for 1 month, weight loss and/or loss of appetite. All suspects were asked to produce two sputum samples (spot and early morning sample), used for culture testing. Patients unable to produce sputum and unwilling to give written informed consent were excluded from the study. Patients were also excluded from the study if they had discontinued anti-TB therapy in the past 3 months and had known HIV infection. Furthermore, we collected data in four main domains: (1) patient history, (2) physical examination, (3) chest radiograph, and (4) sputum smear results. Personal identifiers were not collected and data were analysed anonymously. Authorisations to conduct this study at the hospitals were obtained from the hospital ethical board.

Diagnosis of culture-positive TB was based on Mycobacterium tuberculosis growth in at least one of the first three sputum cultures. Smear-negative TB was defined as no acid-fast bacillus (AFB) identified in the initial three sputum smears, while smear-positive TB was defined as at least one positive AFB smear. Consistent with Ethiopian TB guideline,16 culture-negative PTB was defined as clinical and/or radiographic presentation consistent with TB, three initial mycobacterial sputum cultures negative and no evidence of other respiratory disease.

Only early morning sputum specimens submitted to the hospital TB laboratory as part of routine sputum smear microscopy test were used for culture examinations. Sputum specimens were decontaminated on arrival at the reference laboratory using 2% sodium hydroxide and 0.5% N-acetylcysteine for 25 min, then neutralised to pH 7, concentrated by centrifugation (3000×g for 15 min) and inoculated into a single Mycobacterium Growth Indicator Tube (MGIT) 960 (MGIT, Becton Dickinson Microbiology Systems, Sparks, Maryland, USA).24 Smears were made from isolates obtained from the MGIT tubes, stained by the Ziehl-Neelsen staining method and examined under 100× magnifications using a light microscope for the presence of AFB. The growth on AFB-positive MGIT tubes was further inoculated into two Lowenstein-Jensen slants, one containing sodium pyruvate. The cultures were examined twice a week and their rate of growth and colonial morphologies recorded.

The chest radiography lesions were categorised in terms of the involved lung field. The involved field was categorised as upper and lower lung fields; left and right sides of lung affected with lesions. The chest radiography lesions were categorised as abnormal with lesions consistent with PTB.17 18 ‘Normal’ lung field was defined as the absence of any abnormal lesion on chest radiography. A targeted physical examination was performed for height, weight, axillary or oral temperature, pulse rate, respiration rate and blood pressure. Chest radiography was read by local site investigators.

Statistical analysis

Univariate comparisons were performed using the χ2 test and Fisher’s exact test for categorical variables and Student’s t-test for continuous variables where appropriate. In this study, active TB was determined by a culture-positive result. Receiver operating characteristic (ROC) analysis was used to compare the accuracy of each predictive variable compared with culture results. The area under the ROC curve (AUC) was used as a measure of diagnostic accuracy. Under common practice, AUCs of 0.60–0.69, 0.70–0.79, 0.80–0.89 and 0.90–1.0 were considered to correspond respectively to ‘poor’, ‘fair’, ‘good’ and ‘excellent’ diagnostic accuracy.

Unconditional logistic regression models were used to discriminate the outcome of culture results in clinical suspects and among smear-negative cases and generate ORs and 95% CIs as estimates of effect size. The independent variables were used in logistic regression models in two ways: continuous and binary. Binary values were determined by median values. Best cut-off values were chosen for those continuous variables with values that discriminate culture-negative from culture-positive TB among smear-negative TB cases using ROC. For both continuous and binary independent variables, following initial models that included all variables, a second model that excluded non-statistically significant variables (p>0.05) was run. The Hosmer-Lemeshow test was used to assess the fit of the logistic regression model. Estimates of sensitivity, specificity and AUC were determined by the final model fit. SEs for the AUC were calculated using the Mann-Whitney method. All statistical analyses were performed using SAS V.9.2 (SAS Institute). Statistical significance was at two-tailed p<0.05.

Patient and public involvement

Patients or the public were not involved in the design, or conduct, or reporting, or dissemination of our research.

Results

Among the 426 enrolled cases, 255 (60%) had an abnormal chest radiograph, of which 192 (75%) had lesions compatible with TB (figure 1). Forty-six (24%) patients with chest radiography compatible with TB had positive results of sputum smear microscopy test and were therefore diagnosed as smear-positive TB. Of these, 43 (98%) were confirmed culture-positive cases. The remaining 146 (76%) patients had negative sputum smears, while having positive chest radiography and were therefore considered to have smear-negative TB. However, 78 (64%) of these patients were culture-negative TB cases. None of the patients with a normal chest X-ray (CXR) had a positive smear test. A total of 72 patients without culture results were excluded from the final analysis (two with positive smears and 33 with negative smears among patients with abnormal CXR and 37 with negative smears among patients with normal CXR).

Figure 1

Participant flow diagram. *Total culture-positive cases (n=105). +Total culture-negative cases (n=249). Total excluded cases (because culture test was not done) (n=72).

Predictive variables of culture-positive TB cases among patients with clinical suspicion are given in table 1. The results of the univariate analysis revealed that patients previously treated with anti-TB drugs (OR 2.72; 95% CI 1.57 to 4.69) were more likely to have culture-confirmed TB than never treated patients. In addition, upper lung fields (OR 1.970; 95% CI 1.05 to 3.68) and having bilateral lesions (OR 5.58; 95% CI 2.85 to 10.91) were related to culture-positive TB. However, lesions present in the lower lung fields and on either side of the lungs were less significantly related. The strongest predictive variables both in univariate and multivariate analyses of culture-positive TB were a positive smear test (OR 172; 95% CI 23.23 to 1273.54) and having chest radiography lesions compatible with TB (OR 10.40; 95% CI 5.86 to 18.45). A prediction model based on only the independent significant predictors (sputum smear microscopy and chest radiography) had a good performance by ROC analysis (AUC O.84) for diagnosing culture-positive TB (figure 2). Combining all predictors in the model compared with only the independent significant variables did not really improve its performance to identify culture-positive TB (AUC 0.84–0.87). The discriminating ability of the model neither showed much differently to rightly classify culture-positive TB (AUC rose from 82 to 85).

Table 1

Predictive risk factors of culture-positive TB compared with culture-negative TB among all clinical suspects with culture results (n=354)

				Model 1
Predictive variables	Culture positive for TB (n=105)	Culture negative for TB (n=249)	Univariate OR (95% CI)	β (SE)	Multivariate OR (95% CI)	P value
Sex
Female	40 (25%)	119 (75%)	0.671 (0.42 to 1.07)	−0.489 (0.35)	0.613 (0.31 to 1.23)	0.167
Male	65 (33%)	130 (67%)	1.0		1.0
Age						0.423
45+	38 (27%)	104 (73%)	0.890 (0.47 to 1.67)	−0.007 (0.58)	0.993 (0.33 to 2.96)	0.990
25–44	46 (33%)	94 (67%)	1.192 (0.64 to 2.21)	0.449 (0.55)	1.567 (0.54 to 4.59)	0.413
0–24	21 (29%)	51 (71%)	1.0		1.0
BMI						0.605
BMI <16	15 (26%)	42 (74%)	0.981 (0.50 to 1.93)	−0.451 (0.47)	0.637 (0.26 to 1.59)	0.335
BMI 16>18.50	43 (35%)	78 (65%)	1.510 (0.92 to 2.50)	−0.004 (0.39)	0.996 (0.46 to 2.10)	0.992
BMI 18.5+	47 (27%)	129 (73%)	1.0		1.0
History of anti-TB
Previously treated	33 (47%)	37 (53%)	2.721 (1.57 to 4.69)	0.495 (0.39)	1.641 (0.77 to 3.52)	0.203
Never treated	65 (25%)	198 (75%)	1.0		1.0
Duration of cough						0.998
13 weeks and above	25 (36%)	44 (64%)	1.501 (0.83 to 2.69)	0.025 (0.44)	1.030 (0.43 to 2.43)	0.955
7–12 weeks	27 (29%)	65 (71%)	1.091 (0.63 to 1.90)	0.022 (0.40)	1.020 (0.47 to 2.24)	0.957
1–6 weeks	52 (28%)	137 (72%)	1.0		1.0
Smear results
Positive smears	43 (98%)	1 (2%)	172.001 (23.23 to 1273.54)	4.352 (1.05)	77.670 (9.89 to 609.68)	0.001
Negative smears	62 (20%)	248 (80%)	1.0		1.0
CXR findings
Compatible with TB	87 (52%)	79 (48%)	10.401 (5.86 to 18.45)	1.660 (0.43)	5.260 (2.280 to 12.14)	0.001
Not compatible with TB	18 (10%)	170 (90%)	1.0		1.0
Lung fields	(n=49)	(n=56)
Lower fields	20 (41%)	35 (63%)	1.970 (1.05 to 3.68)	0.411 (0.47)	1.509 (0.60 to 3.80)	0.495
Upper fields	29 (59%)	21 (37%)	1.0		1.0
Lung sides	(n=70)	(n=90)				0.161
Left	20 (28%)	32 (36%)	2.841 (1.46 to 5.54)	0.168 (0.54)	1.182 (0.41 to 3.41)	0.069
Bilateral	27 (39%)	22 (24%)	5.582 (2.85 to 10.91)	0.577 (0.54)	1.780 (0.62 to 5.11)	0.065
Right	23 (33%)	36 (40%)	1.0		1.0

BMI, body mass index; CXR, chest X-ray; TB, tuberculosis.

Figure 2

Receiver operating characteristic curve for the prediction of culture-positive tuberculosis (TB) and culture-negative TB among clinical suspects (n=354).

Almost all variables showed no significant difference between culture-positive and culture-negative TB among smear-negative cases (table 2). Female sex (OR 0.33; 95% CI 0.15 to 0.71) was the only independent negative predictor of culture-positive TB in both analyses, and could be due to less severe lung lesions compared with men. The ROC curve for female sex as the only independent predictor compared with combinations of all predictors (figure 3) both reported poor clinical performance of the model (AUC from 0.64 to 0.69).

Table 2

Predictive risk factors of culture negatives compared with culture positives among smear-negative TB cases (n=146)

				Model 1
Predictive variables	Culture-negative TB (n=78)	Culture-positive TB (n=44)	Univariate OR (95% CI)	β (SE)	Multivariate OR (95% CI)	P value
Sex
Female	46 (77%)	14 (23%)	0.330 (0.15 to 0.71)	−1.13 (0.40)	0.325 (0.15 to 0.71)	0.005
Male	32 (52%)	30 (38%)	1.0
Age (group)						2.67
45+	43 (67%)	21 (33%0	0.780 (0.23 to 2.68)	0.74 (0.91)	2.101 (0.36 to 12.43)	0.414
25–44	27 (60%)	18 (40%)	1.070 (0.30 to 3.79)	1.30 (0.93)	3.670 (0.59 to 22.78)	0.163
0–24	8 (62%)	5 (38%)	1.0
Body mass index (BMI)						0.721
BMI <16	23 (74%)	8 (26%)	0.551 (0.21 to 1.45)	−0.43 (0.540)	0.650 (0.22 to 1.88)	0.462
BMI 16>18.50	22 (60%)	15 (40%)	1.071 (0.46 to 2.52)	−0.05 (0.52)	0.952 (0.35 to 2.61)	0.924
BMI 18.5+	33 (61%)	21 (39%)	1.0
History of anti-TB
Previously treated	21 (57%)	16 (43%)	1.581 (0.71 to 3.53)	0.26 (0.48)	1.294 (0.51 to 3.29)	0.588
Never treated	54 (68%)	26 (32%)	1.0
Duration of cough						0.909
13 weeks and above	22 (65%)	12 (35%)	0.920 (0.37 to 2.27)	0.22 (0.53)	1.249 (0.44 to 3.53)	0.674
7–12 weeks	23 (66%)	12 (34%)	0.808 (0.36 to 2.16)	0.15 (0.52)	1.162 (0.42 to 3.25)	0.774
1–6 weeks	32 (63%)	19 (37%)	1.0
Lung fields with lesions	(n=37)	(n=22)				0.521
Lower fields	22 (59%)	12 (55%)	1.021 (0.42 to 2.44)	0.47 (0.56)	1.605 (0.53 to 4.84)	0.401
Upper fields	15 (41%)	10 (45%)	1.240 (0.48 to 3.22)	0.64 (0.60)	1.899 (0.58 to 6.19)	0.288
Lung sides with lesions	(n=58)	(n=33)				0.693
Left	20 (34%)	10 (31%)	0.910 (0.32 to 2.62)	−0.65 (0.70)	0.523 (0.13 to 2.06)	0.353
Bilateral	17 (30%)	11 (33%)	1.181 (0.41 to 3.38)	0.08 (0.65)	1.087 (0.30 to 3.90)	0.898
Right	21 (36%)	12 (36%)	1.0		1.0	0.705

Twenty-four cases of smear negatives were excluded from analysis because of missed culture test results.

TB, tuberculosis.

Figure 3

Receiver operating characteristic curve for the prediction of culture negatives and culture positives among smear-negative tuberculosis (TB) cases (n=146).

Discussion

The main research finding is that culture-negative PTB cannot be discriminated from culture-positive PTB among smear-negative cases. In addition, a positive sputum smear test and chest radiography compatible with TB remain critical elements in the prediction of culture-positive PTB among patients with clinical suspicion.

The high rates of culture-unconfirmed TB (mostly smear negatives) in more than 40% of patients diagnosed with PTB globally, as reported in 2017 by the WHO,25 are underlined by this study. Female sex (OR 0.33; 95% CI 0.15 to 0.71) was the only independent negative predictor of culture-positive PTB among smear-negative cases, with poor predictive value when compared with combinations of all predictors (AUC from 0.64 to 0.69). Consequently, a lower frequency of culture positivity attributed to less severe lung lesions in women than men has been reported.26–28 So far, no undisputed explanation has been forwarded for this finding, thus supporting our hypothesis that culture-negative PTB may present itself with no differences in clinical and radiographic abnormalities compared with those with culture-positive PTB among smear-negative cases. Our findings emphasise that in settings like ours with higher TB prevalence, there is a low threshold for starting antituberculous therapy, especially in patients with radiographic lesions compatible with TB, despite negative culture results. Hence, predictive models based on clinical variables will not be useful to discriminate patients with culture-negative PTB from patients with culture-positive PTB among smear-negative cases.

Positive sputum smear test (OR 172; 95% CI 23.23 to 1273.54) and chest radiography lesions compatible with TB (OR 10.40; 95% CI 5.86 to 18.45) remain critical elements in the prediction of culture-positive PTB (AUC O.84) among patients with clinical suspicion. Not surprisingly, combining all predictors in the model compared with only the independent significant variables did not really improve its performance to identify culture-positive TB (AUC 0.84–0.87), highlighting the reliability of smear microscopy as a proxy for culture in the classification of TB cases.29 Therefore, in countries with a high prevalence of TB, the specificity of smear microscopy may be superior to that of culture. This may be true even for the diagnosis of TB, since AFB demonstrated in direct sputum smears would then almost invariably represent mycobacteria tuberculous, even in areas with a high burden of HIV.30 By contrast, in countries with a low prevalence of TB, culture (or alternative techniques for species identification) will often be indispensable to the differentiation of TB from other mycobacterial diseases; considering that the study by Nguyen et al4 and Apers et al7 using combined clinical and bacteriological case definitions reported approximately 15%–20% of patients as culture-negative PTB.4 7 Therefore, as prevalence falls, clinicians will be less likely to suspect TB, and will be less likely to be experts in recognising TB, so that even a late culture result will be useful.

Inconsistent with a previous study, we reported cavities appearing on the upper than lower lobes (OR 1.97; 1.05–3.68) and on both sides of the lung (OR 5.582; 2.85–10.91) as independent predictive variables of culture-positive PTB among clinical suspects.31 Most importantly, the results of the univariate analysis revealed that patients previously treated with anti-TB drugs (OR 2.72; 95% CI 1.57 to 4.69) were more likely to have culture-confirmed TB than never treated patients. Upper lobe cavitary TB is the hallmark of postprimary TB and is the site of very high mycobacterial burden. This fits with more recent studies, where the higher bacillary burden was found within the cavities as judged by the time to positivity in liquid culture.32 33 Factors affecting the appearance of the radiograph are likely to be multifactorial and to include host parameters such as ethnicity, age, comorbidities, the bacterial load and degree of disease progression. The interactions of the factors affecting the inflammatory response of an individual to PTB infection need to be prospectively explored.

We must consider a few methodological issues when interpreting the results of our studies. First, our approach is different from previous studies that based their analysis on combining only independent significant predictive variables into a decision model.19 20 Information from a single predictor is often insufficient to provide reliable estimates of diagnostic probabilities or risks.21 22 However, we did not follow-up predictive variables (or clusters of variables) of culture-negative PTB, progressing over time that may be useful in diagnostic decision-making. Therefore, as the condition evolves, clinicians may rely more on the assimilation of information gained over a period of time (‘dynamic evidence’; eg, the addition of new features, the persistence or changes in the characteristics of previous problems) rather than the traditional static information we obtained in our study at one point. Second, it is clear from our study that we did not evaluate how often and to what extent uncertain diagnostic outcome is shared with patients during consultations. Failure to communicate uncertainty effectively can lead to patients failing to return until they are approaching death.34

Conclusion

Our finding suggests that predictive models based on clinical variables will not be useful to discriminate patients with culture-negative PTB from patients with culture-positive PTB among patients with smear-negative cases.