Development of a nomogram for predicting treatment default under facility-based directly observed therapy short-course in a region with a high tuberculosis burden.

BACKGROUND: Poor adherence to tuberculosis (TB) treatment is a substantial barrier to global TB control. The aim of this study was to construct a nomogram for predicting the probability of TB treatment default.
METHODS: A total of 1185 TB patients who had received treatment between 2010 and 2011 in Peru were analyzed in this study. Patient demographics, social, and medical information were recorded. Predictors were selected by least absolute shrinkage and selection operator (LASSO) regression analysis, and a nomogram for predicting TB treatment default was constructed by using multivariable logistic regression analysis. Bootstrapping method was applied for internal validation. Calibration and clinical utility of the nomogram was also evaluated.
RESULTS: The incidence of TB treatment default among the study patients was 11.6% (138/1185). Six predictors (secondary education status, alcohol use, illegal drug use, body mass index, multidrug-resistant tuberculosis, and human immunodeficiency virus serostatus) were selected through the LASSO regression analysis. A nomogram was developed based on the six predictors and it yielded an area under the curve (AUC) value of 0.797 [95% confidence interval (CI), 0.755-0.839]. In the internal validation, the AUC achieved 0.805 (95% CI, 0.759-0.844). Additionally, the nomogram was well-calibrated, and it showed clinical utility in decision curve analysis.
CONCLUSION: A nomogram was constructed that incorporates six characteristics of the TB patients, which provides a good reference for predicting TB treatment default.

Introduction

Tuberculosis (TB) remains a heavy burden on global society and economy. Alarmingly, although TB is usually curable with medication, there were still 1.2 million TB deaths among human immunodeficiency virus (HIV)-negative people and an additional 208,000 deaths among HIV-positive people globally in 2019. With regard to TB medication regimens, current treatment periods are long. A 2-month intensive phase followed by a 4-month continuation phase is commonly recommended for drug-susceptible TB treatment, and, for multidrug-resistant TB (MDR-TB), a longer period of time is usually required.[1,2] It has been recognized that poor adherence to treatment can lead to increased risk of transmission, drug resistance, and death. TB treatment default has been a substantial barrier to global TB control. Therefore, in order to decrease the occurrence of treatment default, it is critical to identify patients with poor adherence and then strengthen management and treatment supervision for these patients.

Several risk factors for TB treatment default have been proposed, including being male, alcohol abuse, illegal drug use, underweight, multidrug-resistant TB, previous treatment default, drug side effects, unemployment, and uncompleted secondary education.[3-7] The potential causes of TB treatment default may be multifaceted. There are many models for predicting TB treatment outcomes; however, few of them were built adhere to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement.[9-13] Since adherence is crucial in the treatment of diseases, such as TB, that require long-term medication, a method that can estimate patient treatment adherence would be very helpful.

In the current study, based on a completed prospective cohort study, demographic, social, and medical information of TB patients were analyzed retrospectively to develop a nomogram for predicting TB treatment default.

Research methods and design

Study population and ethics

In this study, a total of 1185 adult patients with first episode of smear-positive TB were enrolled between 2010 and 2011 in a semi-urban district of northeastern Lima, Peru, which is classified by the World Health Organization (WHO) as a high TB burden country. The majority of these patients were managed at public health facilities (33 community clinics and one hospital). All the patients had a definitive treatment outcome (treatment completion or treatment default). This study was a secondary analysis and it was approved by the Ethics Committee of Jinhua Municipal Central Hospital (No. 2019-133-001). The Ethics Committee waived the need for informed consent because all participant information was anonymous in this secondary analysis.

Variables collection

For this study, the following characteristics of the participants were collected: gender, age (year) (⩽21, 22–26, 27–37, and ⩾38), marital status (married, divorced, single, or widowed), poverty status (yes/no), completed secondary education (yes/no), prison history (yes/no), smoking (never, former, or current), alcohol use (at least weekly or not), illegal drug use (yes/no), rehabilitation history (yes/no), MDR-TB (yes/no), body mass index (BMI) (normal: 18.5–24.9; underweight: <18.5, and overweight: ⩾25.0), HIV infection status (negative, positive, or test not done), coexisting diabetes (yes/no), and other chronic disease status (yes/no). All the patients were treated (free of cost) through facility-based directly observed therapy short-course (DOTS) strategy and the treatment outcome (according to standard WHO definitions) was followed up.[3,15] Treatment default in this study was defined as continuously missed doses ⩾2 months.

Statistical analysis

A multiple imputation method was applied to account for missing data. The baseline characteristics of the patients were summarized. The least absolute shrinkage and selection operator (LASSO) regression method was used for predictor selection. Multivariable logistic regression analysis with a stepwise strategy was used to develop a predictive model and a nomogram. The discriminatory capacity of the nomogram was determined by calculating the area under the curve (AUC). Bootstrapping (resampling times = 500) was performed for the internal validation of the model. Additionally, calibration of the nomogram was evaluated together with the unreliability test. Clinical utility of the nomogram was evaluated using decision curve analysis (DCA). All statistical analyses were conducted with R software (version 3.5.3), and p < 0.05 was considered statistically significant.

Results

Among the 1185 participants, 138 [11.6%, 95% confidence interval (CI), 9.8–13.5%] defaulted from treatment, whereas 1047 (88.4%) patients completed treatment. The median (25–75% interquartile) time for the treatment default was 134 (78–205) days. The characteristics of participants are shown in Table 1. Missing value accounts for 0.27% of the total data.

Table 1.

Baseline characteristics of the study participants.

Variables	TB treatment default		p value
	No (n = 1047)	Yes (n = 138)
Gender, n (%)
Female	441 (42.1)	29 (21.0)	<0.001
Male	606 (57.9)	109 (79.0)
Age (years), n (%)
⩽21	270 (25.8)	39 (28.3)	0.004
22–26	281 (26.8)	40 (29.0)
27–37	238 (22.7)	43 (31.2)
⩾38	258 (24.6)	16 (11.6)
Smoking, n (%)
Never	654 (62.5)	51 (37.0)	<0.001
Current	42 (4.0)	8 (5.8)
Former	351 (33.5)	79 (57.2)
Marital status, n (%)
Married	399 (38.1)	45 (32.6)	0.424
Divorced	73 (7.0)	10 (7.2)
Single	546 (52.1)	81 (58.7)
Widowed	29 (2.8)	2 (1.4)
Poverty status, n (%)
No	876 (83.7)	104 (75.4)	0.015
Yes	171 (16.3)	34 (24.6)
Prison history, n (%)
No	1010 (96.5)	119 (86.2)	<0.001
Yes	37 (3.5)	19 (13.8)
Completed secondary education, n (%)
No	408 (39.0)	83 (60.1)	<0.001
Yes	639 (61.0)	55 (39.9)
Alcohol use at least weekly, n (%)
No	882 (84.2)	77 (55.8)	<0.001
Yes	165 (15.8)	61 (44.2)
Illegal drug use, n (%)
No	923 (88.2)	65 (47.1)	<0.001
Yes	124 (11.8)	73 (52.9)
Rehabilitation history, n (%)
No	993 (94.8)	104 (75.4)	<0.001
Yes	54 (5.2)	34 (24.6)
MDR-TB, n (%)
No	1003 (95.8)	121 (87.7)	<0.001
Yes	44 (4.2)	17 (12.3)
BMI (kg/m2), n (%)
Normal (18.5–24.9)	784 (74.9)	100 (72.5)	0.003
Underweight (⩽18.5)	112 (10.7)	27 (19.6)
Overweight (⩾25.0)	151 (14.4)	11 (8.0)
HIV status, n (%)
Negative	814 (77.7)	78 (56.5)	<0.001
Positive	12 (1.1)	5 (3.6)
Test not done	221 (21.1)	55 (39.9)
Diabetes, n (%)
No	1000 (95.5)	138 (100.0)	0.004
Yes	47 (4.5)	0 (0.0)
Other chronic disease, n (%)
No	1016 (97.0)	136 (98.6)	0.417
Yes	31 (3.0)	2 (1.4)

BMI, body mass index; HIV, human immunodeficiency virus; MDR-TB, multidrug-resistant tuberculosis; TB, tuberculosis.

In LASSO regression analysis (Figure 1), 7 variables with potential predictive value were selected from 15 variables according to non-zero coefficients [the selection criteria of 1 standard error (SE)]. These variables were secondary education status, alcohol use, illegal drug use, rehabilitation history, MDR-TB, BMI, and HIV status.

Figure 1.

Potential predictors screening through the LASSO regression analysis. (a) Tuning parameter (lambda) selection of binomial deviance based on the 1-SE criteria. (b) The coefficients of variables were produced against the log (lambda) sequence. The dotted vertical line was drawn according to the 1-SE criteria, where seven nonzero coefficients were selected.

LASSO, least absolute shrinkage and selection operator; SE, standard error.

To construct a nomogram, the aforementioned seven variables were entered into multivariable logistic regression analysis and a stepwise selection was applied based on the likelihood ratio test with Akaike’s information criterion. Eventually, six predictors (all the variables described above, except for rehabilitation history) were incorporated into the nomogram with a statistical model as shown below: Y = −2.98386–0.49145 × (secondary education status = 1) + 0.83455 × (alcohol use = 1) + 1.60056 × (illegal drug use = 1) + 1.39013 × (MDR−TB = 1) + 0.85935 × (BMI = 1)−0.22707 × (BMI = 2) + 0.74679 × (HIV status = 1) + 0.91240 × (HIV status = 2).

The discriminatory capacity of the nomogram was determined by calculating the AUC. The AUC for the nomogram was 0.797 (95% CI, 0.755–0.839) (Figure 2a), whereas for the internal validation using the bootstrapping method (resampling times = 500) the AUC was 0.805 (95% CI, 0.759–0.844) (Figure 2b).

Figure 2.

The ROC curve and AUC of the model (a) and the internal validation cohort (bootstrap resampling times = 500) (b). The vertical lines represent 95% CI.

AUC, area under the curve; CI, confidence interval; ROC, receiver operating characteristic.

A nomogram that incorporates the six predictors is shown in Figure 3. The probability of TB treatment default can be predicted quantitatively and individually by using this nomogram by calculating the total points of the corresponding point for each predictor.

Figure 3.

Nomogram for predicting the probability of TB treatment default. First of all, find the point for each variable of a patient on the uppermost rule; then add all points together and find the total points; finally, the corresponding predicted probability of TB treatment default could be calculated on the lowest rule.

BMI, body mass index; HIV, human immunodeficiency virus; MDR-TB, multidrug-resistant tuberculosis; TB, tuberculosis.

The proposed nomogram showed good calibration (Figure 4). The unreliability test yielded a p value of 0.964 with an Emax of 0.059 and Eavg of 0.007, suggesting that there was no departure from a perfect fit between the predicted probability and the actual outcome.

Figure 4.

Calibration curve of the nomogram showing the degree of consistency between the predicted probability and the actual outcome. The shaded line represents a perfect prediction by an ideal model, and the dotted line represents the performance of the proposed nomogram. The unreliability test yielded a p value of 0.946, suggesting good calibration between predicted and actual outcomes.

DCA (Figure 5) revealed that, when the risk threshold of treatment default for TB patients was between 3% and 70% based on the predictive model, application of this nomogram to predict the risk of TB treatment default would add more benefit than either the treat-all or the treat-none strategy.

Figure 5.

Decision curve analysis of the nomogram. The red curve represents the model. The decision curve shows that when the risk threshold is between 3% and 70%, application of this nomogram would add more net benefit than either a treat-all or treat-none strategy. For example, assume 100 patients were using the proposed nomogram. If the risk threshold was 20%, then approximately 30 (the corresponding value on the vertical coordinate is approximately 0.3) patients would benefit, without harming the interests of the remaining 70 patients (i.e., when 100 workers use the nomogram to decide whether to take interventions, the added net benefit is approximately 30%).

Discussion

In the present study, a nomogram for predicting the probability of TB treatment default among TB patients was developed. This nomogram incorporates six predictors: secondary education status, alcohol use, illegal drug use, MDR-TB, BMI, and HIV status.

The long duration required for drug treatment is one of the main reasons for the high rate of TB treatment default. Default rates reportedly ranged from 0.5% to 56.6% based on 75 studies conducted in different countries, with a pooled proportion of 14.8%. Therefore, taking measures to improve adherence to treatment is very important in global TB control.[6,7,20] Fortunately, with the adjustment of treatment strategies in recent years, there are declining trends in loss to follow up (LTFU) and increasing trends in the success rate of TB treatment.[21,22] Kibuule et al. reported that since the expansion of facility-based DOTS programme to community-based DOTS programme in Namibia in 2005, the mean annual treatment success rate increased by approximately 20% when comparing the pre-intervention (1996–2005) and the post-intervention (2005–2015) periods.

Several factors affecting TB treatment default, such as alcohol use, illegal drug use, male gender, educational level, and prison or rehabilitation history, have been described in previous articles.[3,5,24] As expected, patients with MDR-TB are more likely to default from treatment than those with drug-susceptible TB on the standard treatment regimens. Several models for predicting TB treatment default have been reported. Rodrigo et al. developed a predictive scoring tool for LTFU based on five characteristics (immigration, living alone or in an institution, previous anti-TB treatment, poor patient understanding, and intravenous drugs use status), yielding an AUC of 0.73 and 0.67 for the derivation and validation groups, respectively. Using the Portuguese TB surveillance database from 2000 to 2012 (TB cases ⩾ 15 years), Costa-Veiga et al. constructed a predictive model (AUC: 0.76) including predictors of TB/HIV co-infection, age ⩾ 64 years, intravenous drugs abuse, other diseases (excluding HIV and diabetes), and retreatment. Additionally, in a case-control study (n = 277) in urban Morocco, Cherkaoui et al. built a scoring system that achieved 82.4% sensitive and 87.6% specific for TB treatment default. However, the application of these models in clinical settings is significantly restricted because of the lack of clinical utility analysis (e.g., DCA), or calibration test, or validation adhere to the TRIPOD statement. Therefore, it is still difficult to effectively estimate the risk of treatment default among TB patients. Since these defaulters have been a substantial barrier to TB control, identifying subjects who are at a high probability of treatment default would be of great value in TB management and treatment.

The proposed nomogram in the present study was constructed based on a prospective study of TB patients, and their demographics, socioeconomic, and medical information were included for analysis. Six potential predictors were selected by LASSO regression analysis and eventually included in the nomogram. All these six predictors could be easily available clinically. The nomogram showed good discriminatory ability (AUC: 0.797) and internal verification results. Moreover, the nomogram showed excellent calibration. More importantly, this nomogram was revealed to have good clinical utility in DCA evaluation. Specifically, DCA demonstrated that when the risk threshold is between 3–70%, application of this nomogram would add net benefit than either the treat-all or the treat-none strategy.

Limitations

Some limitations of this study are worth noting. Firstly, only smear-positive TB patients were included in this study, so the nomogram is suitable only for patients with smear-positive TB. Secondly, there are regional differences in TB epidemiology and socioeconomic status, and differences in TB treatment strategies at different times. This nomogram was constructed based on a study population collected 10 years ago from a TB high-incidence neighborhood of Lima, Peru. Therefore, whether this nomogram can be applied to other regions and the current TB strategies require further verification. Thirdly, the study was also limited by the self-reporting of several variables, and some other potentially affecting factors (e.g., household support, healthcare-related infrastructure, quality of care, availability or change of healthcare practitioners, and treatment regimen change) were not included in the analysis because they were not available in the original data.27 Despite these limitations, this study developed a nomogram, which provides a good reference for other studies predicting TB treatment default.

Conclusions

A nomogram for predicting TB treatment default was developed, which incorporates six characteristics of TB patients. The proposed nomogram showed good discriminatory ability and calibration, as well as good clinical utility. It may be of great value to help adherence management in TB treatment.