Independent validation of a predictive nomogram for risk of reinfection in women with recurrent non-complicated urinary tract infections.

AIMS: Independent external validation of a predictive nomogram for risk of reinfection in women with a history of non-complicated recurrent urinary tract infection (UTI).
METHODS: A retrospective longitudinal study was conducted to validate the LUTIRE nomogram in a Brazilian female cohort. The nomogram was applied to 81 women presenting non-complicated recurring UTI screened at a urological clinic. External validation was performed using the nomogram variables in patients followed up from January 2014 to December 2016 at a urological clinic. Accuracy of the nomogram was obtained by analyzing the predictive capacity observed in the area under the receiver operating characteristic (ROC) curve. A multivariate logistic regression model was used to assess the ability of the nomogram variables to predict the recurrence of UTI over 12 months. The time to recurrence of infection was calculated using a Kaplan-Meier curve and the log-rank test with calculation of the hazard ratio.
RESULTS: The mean age of the study population was 42.8 years; 57 women (70.37%) had recurrence. The independent variables with statistical significance in the multivariate analysis were gram-negative bacteria [odds ratio (OR) 18.38; p = 0.03897] and number of UTIs in the past 12 months (OR 25.11; p = 0.00006). The accuracy of the nomogram for discriminating patients who had UTI recurrence was 82.6% (95% CI = 72.5-90.1).
CONCLUSION: The LUTIRE nomogram showed good accuracy among Brazilian women with recurrent UTI.

Introduction

Approximately 50% of women older than 18 years will present one urinary tract infection (UTI) episode during their lifetime.[1] Of these, approximately 30% will present a second episode in the following 6 months, and approximately 3% will suffer a third episode in the same period.[2] In a primary care setting, 53% of patients older than 55 years reported recurrence of infection after 1 year of observation, whereas in a group of younger patients, the relapse rate was 36%.[3]

UTI recurrence is often associated with a negative impact on quality of life and leads to behavioral changes and alternative treatments in addition to the use of antibiotics. However, there is no high-level evidence to support most behavioral changes in clinical practice.[1-5] Although recurrent UTIs have high direct and indirect costs, there are no tests or questionnaires widely used to assess the risk of recurrence.[1-6]

Cai et al. created the nomogram LUTIRE,[7] which evaluated 17 variables in 768 patients, and, through a multivariate logistic regression model, selected six risk factors for establishing recurrence probability. This nomogram was previously validated in an Italian population and proved useful in determining the risk of ITU recurrence.[7] Notwithstanding, external validation in populations with ethnic, cultural, economic, and religious differences was still need to confirm LUTIRE nomogram accuracy and reproducibility.[8-11]

Our study aimed to perform an independent external validation of the LUTIRE nomogram in a Brazilian female cohort with non-complicated recurrent UTI.

Materials and methods

This was a prospective study approved by the local ethics committee (number 20.040.882). The sample was obtained from a urology department comprising 10 attending physicians who provided care to privately insured patients. Patients were included based on a review of electronic records using code N39.0 of ICD-10 version and a date range of January 2014 through December 2016.

Inclusion criteria were as follows: women aged 18 years or older and younger than 65 years who presented with recurrent non-complicated UTI (two or more episodes in the previous 6 months according to the criteria detailed in Wagenlehner et al.).[12] ITU diagnosis relied on symptoms plus positive urine culture [⩾105 colony-forming units (CFU) per milliliter in midstream urine] with an antimicrobial susceptibility test conducted before the start of treatment. Patients were treated according to the European Association of Urology (EAU) guidelines,[2] and underwent additional tests upon suspicion of complicated UTI. Patients with chronic diseases, such as decompensated diabetes mellitus, chronic kidney disease, liver disease, congenital urinary tract abnormalities, urological cancer, previous pelvic radiotherapy, bladder stones, idiopathic or neurogenic lower urinary tract dysfunction, urolithiasis, chronic urinary retention, or those on intermittent or indwelling catheterization were excluded. Patients who received empirical antibiotics without urine culture were excluded.

Sample size was calculated based on the area under the receiver operating characteristic (ROC) curve (AUC) found for the LUTIRE nomogram external validation group, which was composed of 373 women.[7] In this study, the AUC was 0.85, and the UTI recurrence rate at 12 months was 33.9%. Considering a type I error (α) of 0.05, a type II error (β) of 0.20, value of the null hypothesis of an AUC of 0.50 and AUC of the Italian nomogram of 0.85, the minimum sample size calculated was 21 women in the group with recurrence and 21 women in the group without recurrence. However, considering that the UTI recurrence rate at 12 months in the reference study was 33.9%, the minimum sample size calculated was 21 in the group that had recurrent UTIs and 41 women in the group that did not have recurrent UTIs, comprising 62 patients.

In the study by Cai et al.,[7] the six variables that composed the nomogram to calculate recurrence probability are treatment of asymptomatic bacteriuria in the past year, intestinal function, type of pathogen involved, number of infections in the past 12 months, number of different sexual partners in the past year and hormonal status. In the LUTIRE nomogram, UTI recurrence probability was obtained by matching the value of the sum of the points of each variable. Patients who did not have the six variables necessary to calculate UTI recurrence probability were also excluded from the current study. Figure 1 shows our patient selection flowchart.

Figure 1.

Patient selection flowchart.

ICD, International Classification of Diseases code; UTI, urinary tract infection.

In the current study, to facilitate the calculation of UTI recurrence probability for each patient, a simple digital interface was created, using an Excel® spreadsheet, which contained the score for each variable. The score for each variable was obtained based on the scale of the LUTIRE nomogram. In descending order of relevance, the scores obtained were (a) treatment of asymptomatic bacteriuria in the past year (50 points for previous treatment and 0 for treatment absence); (b) intestinal function (constipation, 50 points; diarrhea, 30 points; and normal bowel function, 0); (c) pathogen type involved (gram negative, 39 points; gram positive, 0 points); (d) number of infections in the last 12 months before inclusion in the study (⩾ 3 episodes, 39 points; 0–2 episodes, 0); (e) number of different sexual partners in the past year (⩾ 3, 35 points; 2, 19 points, 0–1, 0 points); and (f) hormonal status (menopause, 12 points; fertile, 0 points). The total score for each patient was calculated automatically by the spreadsheet (the sum of the scores for each of the six variables). As in the LUTIRE nomogram, the correlation between the total score obtained and the infection recurrence probability was not linear; the urinary infection recurrence probability for each patient was calculated automatically and obtained in the following manner: if the score total was ⩽25, the UTI recurrence probability was 20%; if the total score was between 26 and 154 points, the UTI recurrence probability was obtained using the following formula: [(total score – 26)/2.56)] + 20; if the total score was ⩾155 points, the UTI recurrence probability was obtained using the following formula: [(total score – 154)/5.15)] + 70.

The Bristol scale was used to assess bowel function (constipation: Bristol stool scale type 1 and 2; normal: Bristol stool scale 3–5; diarrhea: Bristol stool scale type 6 and 7).[13,14] Concerning hormone status, premenopausal was defined when the patient had not yet presented menopause or when the patient was receiving oral hormone replacement; menopausal was defined when woman had presented menopause and did not undergo hormone replacement. Uropathogens considered in the urine culture were gram-negative enteral rods, enterococci, Staphylococcus saprophyticus, and group B streptococci.

Clinical visit protocol was not fixed, and depended on the occurrence of UTI symptoms. Clinical data, such as number of infections, absence of relapse, or visits to another service, were retrieved from the medical records. All information was then confirmed by individual telephone contact at the end of follow up. Patients who did not respond to telephone contact and those who self-medicated or treated UTIs without urine test were excluded from the study protocol. Ultimately, 81 women participated in the study.

Primary outcome measure was the occurrence of UTI recurrence at 12-month follow up. Predictive probability of UTI recurrence was calculated using the Excel® spreadsheet, and the accuracy of the LUTIRE nomogram in the current study population was then determined.

Statistical analysis

All statistical analyses were performed considering p < 0.05 and with a confidence interval (CI) of 95%. The statistical software Medcalc for Windows version 9.5.2.0 (Medcalc Software, Mariakerke, Belgium) was used. The Kolmogorov–Smirnov test was used to evaluate whether continuous variables had a normal distribution. Continuous variables with a normal distribution are reported as means and standard deviations, and those without a normal distribution are reported as medians and quartiles. The results of categorical variables are expressed as frequencies and percentages. Differences between the two groups (presence or absence of UTI recurrence) were evaluated using the t test for independent samples for continuous variables that had a normal distribution, using the Mann–Whitney U test for continuous variables that did not show a normal distribution and using the Chi-square test for categorical variables. For categorical dichotomous variables, the odds ratio was calculated.

Sensitivity of recurrence probability obtained by the nomogram was calculated for each cut-off point by dividing the number of patients with recurrence and with a probability of reinfection higher than the cut-off point used by the total number of patients with recurrence. Specificity of UTI recurrence probability was calculated for each cut-off point by dividing the number of women without recurrence and who had a probability of UTI recurrence equal to or less than the cut-off point used by the total number of women who did not have a new UTI episode. The ability of the nomogram to predict UTI recurrence was assessed by sensitivity and specificity. A ROC curve was constructed to graphically demonstrate the sensitivities and specificities of the different probabilities of UTI recurrence calculated by the nomogram. Thus, the best cut-off point and the AUC for the accuracy of the nomogram were determined.

In addition to the global accuracy (AUC), the Brier score was also used to estimate the predictive performance of the nomogram. Brier scores were calculated based on mean square deviations between the predicted results (probability calculated by the nomogram) and those observed (UTI recurrence at 12 months = 1 and no recurrence at 12 months = 0) for each patient. Scores closer to zero indicated better predictions. The Brier score was calculated by the average of the different Brier scores obtained for each patient.

A multivariate logistic regression analysis model (MLRM) was used to evaluate the ability of the six variables of the nomogram to predict UTI recurrence at 12 months. As a variable selection method, stepwise regression was used, considering p < 0.05 significant; the variable was removed from the model when p > 0.20.

Two Kaplan–Meier curves were constructed by the dichotomization of the calculated UTI recurrence probability obtained by the nomogram using the cut-off point provided by the ROC curve, which best discriminated the two groups (greater than 40% and less than or equal to 40%) considering UTI recurrence at 12 months as the classification variable. The outcome was the time to UTI recurrence at 12 months. Patients in whom UTIs did not recur after 12 months were treated as censored data. The comparison between the curves was performed using the log-rank test, and the hazard ratio was calculated.

Results

Of the 81 women who participated in the study, 57 (70.37%) had UTI recurrence in 12 months of follow up. The average time to recurrence was 3 ± 1.87 months (20 days to 10 months).

Nomogram variables are compared between the two groups (UTI recurrence versus non-recurrence) in Table 1. In the univariate analysis, only the variables “bacteria isolated in urine culture” (gram negative or gram positive) and “number of UTI in the past year” were statistically significant as predictors of recurrence at 12 months. The multivariate analysis (Table 2) confirmed the same results, and only these two variables remained in the MLRM.

Table 1.

Baseline characteristics of women who presented and did not present UTI recurrence during a 12-month follow-up period.

Variable	UTI recurrence		Statistical analysis
	Yes = 57(70.37%)	No = 24(29.63%)
Intestinal function			p = 0.1601**
Constipation	26 (45.62%)	60 (25.00%)
Diarrhea	01 (1.75%)	00 (00.00%)
Normal	30 (%)	18 (%)
Intestinal function			p = 0.0663**
Constipation or diarrhea	27 (47.37%)	06 (25.00%)	OR = 2.700
Normal	30 (52.63%)	18 (75.00%)	CI = 0.9352–7.7948
Bacteria isolated in the urine culture			p = 0.0170**
Gram negative			OR = 14.7368
Gram positive			CI = 1.6179–134.2338
Hormonal Status			p = 0.2555**
Post-menopause	23 (40.35%)	13 (54.17%)	OR = 0.5725
Pre-menopause	34 (59.65%)	11 (45.83%)	CI = 0.2188–1.4974
Number of UTIs in the past year			p < 0.0001*
Median	03	02
Q1–Q3	03–04	02–02
Variation	02–08	02–03
Number of UTIs in the past year			p < 0.0001*
Three or more	47 (82.46%)	03 (12.50%)	OR = 32.9000
Up to two	10 (17.54%)	21 (87.50%)	CI = 8.2027–131.9585
Age			p = 0.6413***
Mean ± SD	42.10±12.33	43.54±13.30

CI, 95% confidence interval; OR, odds ratio; UTI urinary tract infection.

Mann–Whitney U test; **Chi-square test; ***t test for independent samples.

Table 2.

Multivariate logistic regression model.

Variable	OR	95% CI	p value
Bacteria isolated in urine culture	18.3887	1.1588–291.8161	0.03897
Number of UTIs in the past year	25.1112	5.1538–122.3495	0.00006

CI, 95% confidence interval; OR, odds ratio; UTI urinary tract infection.

Mean and standard deviation (SD) of the recurrence probability, obtained by the nomogram, are shown in Table 3. A total of 57 women with recurrence had a 12-month recurrence mean calculated probability of approximately 55.89% ± 15.3%, whereas the 24 women without recurrence had a mean calculated probability of approximately 36.44% ± 12.81% (p < 0.0001).

Table 3.

UTI recurrence probability using the nomogram over a 12-month follow-up period.

Variable	UTI recurrence		Statistical analysis
	Yes = 57	No = 24
	(70.37%)	(29.63%)
Probability (%) of UTI recurrence calculated by the nomogram greater than 40			p < 0.0001**
Yes	51 (89.47%)	07 (29.17%)	OR = 20.6429
No	06 (10.53%)	17 (70.83%)	CI = 6.0895–69.9777
Probability (%) of UTI recurrence by the nomogram			p < 0.0001***
Average	55.89	36.44
SD	15.30	12.81
Variation	20.00–64.92		25.07–81.45
Sensitivity of the nomogram to predict UTI recurrence for a probability calculated by the nomogram greater than 40%: 89.47%.
Specificity of the nomogram to predict UTI recurrence for a probability calculated by the nomogram greater than 40%: 70.83%.

CI, 95% confidence interval; OR, odds ratio; SD, standard deviation; UTI urinary tract infection.

Chi-square test; ***t test for independent samples.

The AUC of the calculated likelihood to predict UTI recurrence over a 12-month period was 0.826 (95% CI and 0.725–0.901 range), that is, a high and significant overall accuracy (p < 0.0001) of 82.6% in predicting UTI recurrence over 12 months. Figure 2 illustrates the sensitivity and specificity of predicting UTI recurrence over a 12 month for different calculated odds values.

Figure 2.

ROC curve of the predictive probability of the nomogram for infection recurrence in Brazilian women with recurrent UTI.

ROC, receiver operating characteristic curve; UTI, urinary tract infection.

The calculated probability cut-off point, which best discriminated women with, from those without, recurrence, was 40%, with a sensitivity of 89.47% and specificity of 70.83% (Figure 2 and Table 3). Women who had a calculated recurrence probability greater than 40% had a 20.64-fold higher chance of UTI recurrence at 12 months than those who had a probability less than or equal to this value (Table 3).

The Brier score was 0.197 ± 0.1333, and ranged from 0.0344 to 0.5613, indicating good predictive ability of the nomogram.

Figure 3 illustrates the Kaplan–Meier curve for the UTI recurrence-free time for women who presented a calculated probability greater than 40%, and for those who had a calculated probability less than or equal to 40%. Women who showed a calculated probability of UTI recurrence at 12 months greater than 40% also had a shorter time for UTI recurrence (p < 0.0001). The estimated risk (hazard ratio) for the recurrence-free time calculated probability of 40% was 5.47 (95% CI 2.52–8.32).

Figure 3.

Kaplan–Meier curve for time to UTI recurrence in 12 months dichotomized for a calculated probability obtained from the UTI recurrence nomogram.

UTI, urinary tract infection.

Discussion

The LUTIRE nomogram showed high accuracy when applied to a Brazilian population to discriminate women at higher risk of UTI recurrence. Our results, with an accuracy of 82.6%, are similar to those of Cai et al. in an Italian cohort (85%).[7]

To our knowledge, this is the first study to validate the nomogram proposed by Cai et al.[7] The external validation of the LUTIRE nomogram is a fundamental step for determining its usefulness for clinical practice, raising its classification to level two on the hierarchy scale of predictive systems according to Justice et al.[8]

A relevant point in our study was that the cut-off point for the calculated UTI recurrence probability of 40% was obtained through ROC curve analysis. This cut-off point showed high sensitivity (89.47%) without a substantial loss of specificity (70.83%) when determining women with a higher risk of recurrence. By using this cut-off, it would be possible to provide a better counseling for higher-risk patients, and, consequently, increase adherence to treatment.

In our study protocol, a simpler digital interface allowed the introduction of the score for each variable in an Excel® spreadsheet, which calculated UTI recurrence probability. This spreadsheet can be used in clinical practice, allowing easier calculation of UTI recurrence probability than does the nomogram graphical interface. Thus, this spreadsheet could be used by patients, accessed by a device connected to the internet, to calculate UTI recurrence probability, making it easily understandable by the patient.[10]

Other authors have established models for calculating recurrence probability. Hooton et al. proposed a chance of reinfection model based on the number of sexual relations in the previous week.[15] The relative risk was 1.9 for those patients with two sexual relations in 7 days, and 9 for those with daily relations in 7 days when compared with patients who did not have sex in 7 days.[15] More recently, Garcia-Telo et al. created an internally validated nomogram for predicting the risk of UTI complicated by ESBL (extended spectrum β-lactamase) bacteria.[16] To our knowledge, the Cai et al. nomogram is the only predictive instrument proposed for evaluating non-complicated UTI recurrence risk.[7] The determination of patients with a higher recurrence probability may be useful for the management of this condition, concerning treatment dilemmas, such as increasing bacterial resistance to antimicrobials and the costs involved in treatment.[17-21]

Although most studies consider the number of sexual relations to be the most important risk factor, the number of different sexual partners is also considered by some authors as a risk variable for non-complicated UTI recurrence.[2,4] Identification of the number of different sexual partners has some peculiarities. In the current study population, 88.9% reported only one sexual partner, whereas Cai et al. observed that 42% of patients had one partner.[7] The finding may be explained, in part, by a bias associated with the method of obtaining the information (in this study, an interview by a urologist); however, it is not described how Cai et al. collected this information. The Italian population showed an average age and marital status similar to those of the Brazilian population in this study. Several studies highlight the discrepancy between information provided by women regarding different types of approaches to issues related to sexual habits.[22,23] In general, the interviewees reported fewer sexual partners when approached by male interviewers; the method to most closely reflect reality would be interviews using a digital questionnaire.[22,23] Thus, ideally, the interview to determine the number of different sexual partners could be performed through a digital instrument, ensuring greater privacy in the response obtained.

During the selection of the sample, was observed great difficulty in patient follow up, about performing routine urine tests before and after treatment to define the pathogen involved and to confirm a cure. The difficulty in obtaining a urine culture to confirm a clinical cure was even greater, which is the reason why this criterion was not adopted in our study. However, the absence of urine culture examinations confirming a microbiological cure in some patients did not, in our view, impair the data analysis. Numerous studies confirm the sensitivity of clinical signs as sufficient for the diagnosis of infection and cure, not recommending routine cultures because of a lack of reliability of the results.[24-26]

The recurrence rate of 70.37%, well above the Italian study population (33.9%), may be explained by our clinic being a tertiary reference clinic that specializes in urology.

The characteristics of the study sample are also limitations in this study. The selection of patients from a supplementary health clinic could limit the representativeness of the adult Brazilian female population. Thus, although the number of participants conferred power to the study, larger and different populations are necessary. Despite these limitations, the accuracy of the nomogram in our sample was high and similar to that in the study with an Italian population.

One of the main aspects related to the use of predictive instruments and their implications in clinical practice is their superiority to clinical judgment in deciding the conduct to be adopted.[8,11] As a future continuation of the study, the nomogram could be utilized through an interface easily accessible over the internet, making it comparable with the subjective evaluation of a physician regarding the risk of recurrence or evaluations regarding possible greater adherence to treatment that this instrument offers. Additionally, a prospective longitudinal randomized clinical trial comparing one group who uses the nomogram with a control group in which clinical decisions are based only on clinical evaluations could better define the role of the nomogram in the management of women with recurrent infection.

Conclusion

The LUTIRE nomogram proved to be an instrument with good accuracy for determining the risk of recurrence in a Brazilian adult cohort of women with recurrent UTI. The predictive ability was similar to that for the original Italian population, despite numerous ethnic and cultural differences.