Development of a scoring system using a statistical model to predict cure status in patients with cutaneous leishmaniasis.

BACKGROUND: The present study was performed to develop a scoring system for predicting cure status in patients with cutaneous leishmaniasis (CL).
MATERIALS AND METHODS: This study included 199 patients with CL from Skin Diseases and Leishmaniasis Research Center (Isfahan, Iran). Data were collected as longitudinal in each visit of patients. We applied ordinal logistic generalized estimating equation regression to predict score on this correlated data. To evaluate the fitted model, split sample validation method was applied. SPSS software was used for data analysis.
RESULTS: The regression coefficients of the fitted model were used to calculate score for cure status. Based on split-sample validation method, overall correct classification rate was 82%.
CONCLUSION: This study suggested a scoring system predict cure status in CL patients based on clinical characteristics. Using this method, score for a CL patient is easily obtained by physicians or health workers.

INTRODUCTION

Leishmaniasis is an infectious disease caused by the protozoa of Leishmania species, which is transmitted by a female sandfly bite.[12] Leishmaniasis is classified into three groups including cutaneous, mucocutaneous, and visceral. Cutaneous leishmaniasis (CL) is the most common form. About 1.5 million new cases of CL occur per year and more than 90% of them are observed in seven developing countries including Iran, Afghanistan, Syria, Saudi Arabia, Brazil, and Peru.[23456] CL causes lesions on the exposed parts of the body. These lesions are usually painless but can become painful if they become secondarily infected. Most lesions develop during a few weeks of the sandfly bite, but they may also seem to several months later.[2] When the lesions cure, they may leave table and deep scars which can cause mental problems. Therefore, evaluating the severity of CL at each visit of patient is important to select suitable treatment that can reduce the size of the lesions with minimal scarring.

Methods of evaluating the severity of skin diseases are often subjective, which makes a difference in results. Therefore, to keep objectivity in observations, scores are applied to evaluate the severity of skin diseases.[7] This is particularly important for monitoring the response to therapy and for evaluating the efficacy of new drugs. In recent years, scoring systems have been developed for some skin diseases. Agarwal et al. suggested pemphigus area and activity score for the clinical assessment of severity and progression of pemphigus vulgaris.[8] Kimbrough-Green et al. developed melasma area severity index for the assessment of melasma.[9] Ferriman and Gallwey suggested scoring system for hirsutism in women.[10] Valencia et al. reported a score for prognosis of antimonial therapeutic failure in ulcerative CL patients treated with sodium stibogluconate (SSG) using the logistic regression.[3]

To develop a clinically helpful scoring system, it has to keep several criteria: It should utilize readily available and confirmable clinical information, it should have been developed and validated in the population to whom it is to be used, and it should be free from confounding factors.[11]

Since the use of clinical scores provides a valuable tool for clinical management and orients physicians to select the most suitable treatments,[312] the purpose of the present longitudinal study was to develop a scoring system for predicting cure status in CL patients based on influential predictors using a statistical model. The generalized estimating equations (GEEs) approach was applied to this longitudinal data. Evaluation of model was performed using split-sample validation method.

MATERIALS AND METHODS

This study is an analysis of data collected from Skin Diseases and Leishmaniasis Research Center (Isfahan, Iran) in 2011–2012. Dataset includes 199 CL patients. Their information was involved gender, age, morphology of the lesion including flat and other types (papule, nodule, plaque, and others), number of lesions, size of lesions (length of lesion × width), lesions' location including head and neck, body, hands, and legs, type of treatment including systematic, topical, oral and alone visit, visit times of patients during therapy, induration status (grouped in four levels), and cure status which had been defined as four ordered categories and considered as outcome variable. We regarded CL longitudinal data as three-level structure; the Level 1 units were the repeated occasions of measurement, the Level 2 units were lesions of CL patients, and the Level 3 units were CL patients. Hence, we performed a GEE ordinal logistic regression. The GEE approach denotes an extension of the generalized linear model to analyze correlated data.[13] In this approach, the correlation between correlated measurements is modeled by assuming a working correlation matrix.[14] The GEE models make estimates of model coefficients for predictors that are averaged over clusters whereas allowing residuals to correlate within clusters.[15]

Using the fitted model, probability or score of ordered categories for cure status was predicted as:

where J represents number of categories for outcome variable and P(Y ≤j|X) = π1 +π2+....π is cumulative probability from category 1 to category j and probability for category J is calculated as:

πj = 1−(π1 +π2....πj−1)

where

Lj denotes the linear predictor of fitted model for category j, Lj= β0j – (β1 ×1+…+ βkXk), which β0j denotes intercept for category j and β1, β2,…, βk denotes regression coefficients. X1, X2,…, Xk represents predictive variables contained in the fitted model.[16] Thus, a CL patient belongs to the category that has highest probability among all categories. To evaluate the fitted model, split sample validation was used. In this method, dataset was split randomly into two parts; the training set includes a sample of 140 for estimation of the regression coefficients and the test set includes a sample of 59 for evaluating the performance of the score. The predicted categories for test set using regression coefficients resulting training set were compared with observed categories by physicians based on clinical information. High correct classification rate indicates good concordance of the score. Data analyses were performed using a statistical software package (IBM SPSS Statistics version 20, Tokyo, Japan).

RESULTS

Mean age of CL patients was 29.27 years with standard error 1.23% and 68.8% of patients were male. Outcome variable, cure status, was ordinal and includes four categories: No cure, initial cure, partial cure, and complete cure. Predictor variables were gender, age, morphology of the lesion, number of lesions, size of lesion, location, type of treatment, induration status, and times of visit. Table 1 shows results of GEE ordinal logistic regression.

Table 1

Results of ordinal logistic generalized estimating equation for longitudinal cutaneous leishmaniasis data

The regression coefficients of the fitted model were used to calculate probability or score for cure status. Cumulative probability for category j can be calculated as:

where

Lj= β0j− (0.024 gender + 0.009 age − 4.705 induration1 − 4.983 induration2 − 10.744 induration3 − 0.250 location1 + 0.016 location2 + 0.007 location3 + 3.147 morphology − 1.771 treatment1 − 0.778 treatment2 − 0.865 treatment3 + 0.004 number + 0.033 size + 0.039 time)

In this equation, β01= −11.121, β02= −6.686, and β03= 1.407, gender = 1 for males and gender = 0 for females, induration1 = 1 for lesions that take induration at Level 1 and induration1 = 0 otherwise, induration2 = 1 for lesions that take induration at Level 2 and induration2 = 0 otherwise, induration3 = 1 for lesions that take induration at Level 3 and induration3 = 0 otherwise, location1 = 1 for lesions that are at head and neck and location1 = 0 otherwise, location2 = 1 for lesions that are at body and location2 = 0 otherwise, location3 = 1 for lesions that are at hands and location3 = 0 otherwise, morphology = 1 for lesions that are flat and morphology = 0 otherwise. Treatment1 = 1 for those who use treatment of systematic and treatment1 = 0 otherwise, treatment2 = 1 for those who use treatment of topical and treatment2 = 0 otherwise, treatment3 = 1 for those who use treatment of oral and treatment3 = 0 otherwise. Continues variables in equation take their real values.

For example, for a female CL patient at the age of 20 years with 12 lesions, for a lesion in face with induration at Level 3, morphology of others, size of 4 cm2, and used treatment of topical, on the 7th day, her probability for category 1, no cure, is . Cumulative probability for category 2 can be calculated as:

, then probability for category 2, initial cure, becomes P (Y = 2) =0.9895 − 0.5045 = 0.4850.

Cumulative probability for category 3 can be calculated as: then probability for category 3, partial cure, is P (Y = 3) =0.9999 − 0.9895 = 0.0104 and probability for category 4, complete cure, is P (Y = 4) =1 − 0.9999 = 0.0001. Based on calculated probabilities for each category of cure, this CL patient belongs to the category 1, no cure, because it has a higher probability.

Table 2 shows results of classification from split sample validation for the fitted model. The overall correct classification rate for GEE ordinal logistic regression was 0.82. Since values of observed and predicted categories were ordinal, the Spearman correlation coefficient was calculated to determine association between observed and predicted values which was 0.876 (P < 0.001). It shows strong association between the values of predicted and observed categories.

Table 2

A cross-tabulation of the predicted versus true values, from a generalized estimating equation ordinal logistic regression

DISCUSSION

In the present study, we developed a scoring system to predict cure status in CL patients. Ordinal logistic GEE regression was applied to this longitudinal dataset. The significant predictors in ordinal logistic GEE regression were induration status, morphology of the lesion, type of treatment, size of lesion, age, and times of visit. Although gender and location were not significant in the fitted model, they are influential in cure of CL.[17] To adjust effect of variables on cure status, we applied all of mentioned predictor variables in model for calculating score. Based on split-sample validation method, there was good concordance between observed and predicted categories. Complete cure category showed the highest rate of correct classification (82%) and the lowest rate was for initial cure category (63%).

Valencia et al. reported a score for prognosis of antimonial therapeutic failure in ulcerative CL patients treated with SSG. Outcome variable in their cross-sectional study was binary and they calculated probability of treatment failure using a logistic regression. The present study was regarded as longitudinal and outcome variable, cure status in CL patients, was ordinal with four groups. Thus, ordinal logistic GEE regression was applied.

Maxwell et al. developed and validated a scoring system for patients undergoing hip fracture surgery using a logistic regression. They assessed goodness-of-fit of their score to the data using the Hosmer–Lemeshow statistic. They also performed sensitivity analysis using standard receiver operating characteristic (ROC) curve.[11] Although the ROC curve is more informative than the classification table, it is complicated when outcome variable has more than two categories.[16]

Bastuji-Garin et al. suggested a specific severity-of-illness score using logistic regression for cases of toxic epidermal necrolysis. They compared their score with the simplified acute physiology score and a burn scoring system.[18]

A limitation of the present study was that there was no other scoring system for CL with multi-category outcome to compare our score with it. In addition, we obtained a scoring system in a small sample of CL patients from a single center which may influence the results. However, a useful scoring system should be applicable to different centers with similar populations.

CONCLUSION

This study suggested a scoring system predict cure status in CL patients. This predictive score presents useful benefits such as it relies on clinical characteristics and it is easily obtained by physicians or health workers.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.

AUTHORS' CONTRIBUTION

MN, AZ, and MH contributed in the conception and design of the work, performed data collection. MKh and MH contributed in the analysis, drafted the manuscript, performed significant revisions, approved the final version of the manuscript. All authors agreed to all aspects of the work.