Predicting the use of antibiotics after initial symptomatic treatment of an uncomplicated urinary tract infection: analyses performed after a randomised controlled trial.

OBJECTIVE: To predict antibiotic use after initial treatment with ibuprofen using data from a randomised controlled trial comparing ibuprofen to pivmecillinam in the treatment of women with symptoms of an uncomplicated urinary tract infection (UTI).
SETTING: 16 sites in a primary care setting in Norway, Sweden and Denmark. PARTICIPANTS: Data from 181 non-pregnant women aged 18-60 presenting with symptoms of uncomplicated UTI, initially treated with ibuprofen.
METHODS: Using the least absolute shrinkage and selection operator logistic regression model, we conducted analyses to see if baseline information could help us predict which women could be treated with ibuprofen without risking treatment failure and which women should be recommended antibiotics.
RESULTS: Of the 143 women included in the final analysis, 77 (53.8%) recovered without antibiotics and 66 (46.2 %) were subsequently prescribed antibiotics. In the unadjusted binary logistic regression, the number of days with symptoms before inclusion (<3 days) and feeling moderately unwell or worse (≥4 on a scale of 0-6) were significant predictors for subsequent antibiotic use. In the adjusted model, no predictors were significantly associated with subsequent antibiotic use. The area under the curve of the final model was 0.66 (95% CI: 0.57 to 0.74).
CONCLUSION: We did not find any baseline information that significantly predicted the use of antibiotic treatment. Identifying women who need antibiotic treatment to manage their uncomplicated UTI is still challenging. Larger data sets are needed to develop models that are more accurate. TRIAL REGISTRATION NUMBER: ClinicalTrials.gov (NCT01849926). © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

RCT Entities:   Population Interventions Outcomes

This study aims to predict the use of antibiotics after initial symptomatic treatment in women with symptoms of an uncomplicated urinary tract infection using clinical cure as the main outcome.

The data are obtained from a relatively large randomised controlled trial conducted in an unselected population of women in primary care.

The study population was the subgroup of patients receiving ibuprofen, 143/181 (79%) had complete baseline data and were included in the analyses.

This population might not be large enough to avoid type II error, as the trial was not adequately powered to perform predictor analysis.

Introduction

With years of research and clinical experience, the diagnosis and treatment of uncomplicated urinary tract infections (UTIs) still poses a challenge for clinicians. A positive urine culture has been considered the definition of a ‘true’ UTI, but many women with symptoms of a UTI have negative cultures and still benefit from antibiotic treatment.1 Placebo studies have shown that women with a positive urine culture become symptom free without antibiotic treatment.2

Antibiotic resistance is on the rise, and for the last three decades, researchers have sought to find ways of reducing unnecessary use of antibiotics. The potential for reducing antibiotic use is the largest in self-limiting conditions, such as respiratory tract infections and uncomplicated UTIs, the two most common reasons for prescribing antibiotics in general practice.3 The focus has shifted from finding the easiest way to diagnose and treat a UTI to identifying the patients who actually need antibiotics to recover, and which patients can be recommended a wait-and-see strategy. Both placebo trials and trials comparing non-steroidal anti-inflammatory drugs (NSAIDs) to antibiotics have proven that antibiotic treatment is superior with regards to clinical and bacteriological cures, but they have also shown that a substantial part of the patients do recover without antibiotic treatment, regardless of urine culture results.4–8 In 2010, Little et al compared the effectiveness of five different approaches in the management of UTIs, they concluded that antibiotics targeted with dipstick tests with a delayed prescription as backup or empirical delayed prescription could help reduce antibiotic use.9 The ATAFUTI Trial (Alternative Treatments of Adult Female Urinary Tract Infection: a double blind, placebo controlled, factorial randomised trial of Uva ursi and open pragmatic trial of ibuprofen) published in 2019 concluded that ibuprofen reduced antibiotic consumption without increasing complications.10

The trial by Gagyor et al comparing ibuprofen to fosfomycin in the treatment of uncomplicated UTI showed that two-thirds of the women recovered without antibiotic treatment.6

We conducted a randomised controlled trial comparing ibuprofen to pivmecillinam in the treatment of uncomplicated UTIs, the IMUTI Trial (Ibuprofen versus pivMecillinam for uncomplicated Urinary Tract Infection in women).8 We found that 53% of the women initially treated with ibuprofen recovered without using antibiotics, but pivmecillinam was significantly more efficient than ibuprofen in achieving both clinical and bacteriological cures.

Gagyor et al used data from their trial to develop a prediction model for which women should be recommended immediate antibiotic treatment and who could be recommended initial treatment with ibuprofen.11 This was the first prediction model made with clinical cure as the reference standard, as opposed to previous studies aiming to predict which patients will have a positive urine culture.12–19 Gagyor et al identified five factors predicting the use of subsequent antibiotic treatment in the group initially treated with ibuprofen: a positive test result for nitrite, leucocytes, and erythrocytes, moderate-to-severe urinary urgency or frequency, and impairment of regular daily activities most of the time. Their model was moderately accurate (area under the curve (AUC)=0.73 (95% CI: 0.67 to 0.80)), and they concluded that in order to construct a prediction model that is adequate for general practice, more studies from various independent samples are needed. Hence, we aimed to analyse the baseline information from our trial to identify

which women could be treated with ibuprofen without risk of therapeutic failure and

which women should be recommended antibiotics.

Methods

Context and study design

The analyses in this manuscript are based on data from the IMUTI Trial comparing ibuprofen with pivmecillinam in the treatment of uncomplicated UTI. The trial aimed at comparing effectiveness of the two treatment regimens with clinical cure by day 4 as the main outcome measure. Women were instructed to consult the study site or their local general practitioner’s (GP’s) office/out-of-hours (OOH) service if they did not recover or if they experienced worsening of symptoms/complications. On reconsultation, they were treated according to the study protocol or at the discretion of the GP when reconsulting outside of the study site.

Median time until the patients felt well was 6 days in the ibuprofen group and 3 days in the pivmecillinam group. In the ibuprofen group, 47% of the patients received subsequent antibiotic treatment within 4 weeks versus 11% in the pivmecillinam group.

We performed analyses on data from the ibuprofen group to identify which measures and clinical findings presented at baseline were associated with the use of subsequent antibiotic treatment.

Patient and public involvement

No patients were involved in the design, conduct, reporting or dissemination of our research.

Participants

Non-pregnant women, age 18–60, presenting to the GP or an OOH service with symptoms of an uncomplicated UTI were enrolled in the study. Inclusion criteria were dysuria combined with either increased urinary frequency or urinary urgency or both, with or without visible haematuria.

The main exclusion criteria were: duration of symptoms for more than 7 days, any sign of an upper UTI (fever and upper back pain), vaginal discharge, severe abdominal pain, conditions that could potentially lead to complications (such as diabetes, kidney disease, severe gastritis and previous pyelonephritis), symptoms of a UTI within the last 4 weeks, or use of antibiotics within the last 2 weeks.

Development of the prediction model

To develop a helpful clinical decision aid, we decided to include only predictors present at baseline.

Self-reported symptom burden at inclusion was registered in a patient diary and included dysuria, increased urinary frequency and urgency, visible haematuria, fever, abdominal pain when not passing urine and feeling unwell. Each symptom was graded on a scale from 0 to 6, where 0 was ‘normal/not affected’ and 6 was ‘as bad as it could be’, fever (>38°C) and visible haematuria were yes/no questions. For the analyses in this article, we decided to use the three symptoms represented in the inclusion criteria as individual predictors: dysuria, increased urinary frequency and urinary urgency. We also decided to use ‘feeling unwell’ and ‘abdominal pain when not passing urine’ as separate predictors. Self-reported fever was not used since all patients had their temperature measured at inclusion, and it had to be lower than 38°C for inclusion in the study. Visible haematuria was not included as an independent predictor as we considered the dipstick result for haematuria to be a more sensitive and accurate predictor. Urine culture was not included in the analysis since the results are not available at presentation.

We ended up with five factors of interest: (1) age, (2) number of days with symptoms before inclusion, (3) number of UTIs treated with antibiotics within the last year, (4) self-reported symptom burden at inclusion and (5) urinary dipstick results (leukocytes, nitrite and blood/erythrocytes).

Statistical analysis

We performed univariate and multivariate logistic regression analysis. The predictors were purposefully selected using a cut-off of p≤0.20 from the unadjusted analysis. Age was included in the adjusted model as an essential predictor. We used the least absolute shrinkage and selection operator (LASSO) logistic regression model to create a parsimonious model by selecting a subset of predictors and shrinking the rest to zero.20

The LASSO logistic model selected candidate predictors for subsequent need of antibiotics. Using the estimated regression coefficients (β) of the final model, conditional predicted probabilities for the subsequent use of antibiotics were estimated as the exponent of each woman’s risk score (see online supplementary file 1 for details). The predicted probabilities were estimated for each woman with no missing data points in the selected predictors.

To assess the accuracy of the final fitted prediction model, we plotted sensitivity of the test against (1−specificity) to produce the area under the receiver operating characteristic (ROC) curve. The area under the ROC curve with 95% CI was calculated and used to assess how well the model discriminated between the women who would subsequently need antibiotics from those who did not. If the area under the ROC is equal to 1, then the model perfectly discriminates group 0 from group 1, whereas a value of 0.5 suggests no discrimination.

Several cut-off points of estimated risk scores were used to estimate sensitivity, specificity, the proportion of women that the model predicted would be recommended for antibiotics by their GPs relative to all women and the proportion of women that the model predicted would return to the practices due to treatment failure. We used bootstrap resampling to determine the accuracy of our sample estimates. The LASSO logistic regression model was fitted using the glmnet and pROC packages in R.21

Results

In total, 143 women with UTI in the ibuprofen group were included in the analysis (figure 1). Of these, 77 (53.8%) recovered without antibiotics and 66 (46.2 %) were given subsequent antibiotic treatment. The baseline characteristics of these patients are presented in table 1. The women who subsequently used antibiotics had a higher rate of positive urine dipstick findings, had a greater burden of some symptoms and had experienced symptoms for fewer days compared with the women who recovered without antibiotics.

Figure 1

Flow of participants.

Table 1

Predictors for subsequent antibiotic treatment in the ibuprofen group—standard logistic regression model

No antibiotics (%)	Antibiotics (%)	Unadjusted		*Adjusted
n=77	n=66	OR (95% CI)	P value	OR (95% CI)	P value
Age (ref: <26 years)
≥26	36 (46.8)	35 (53.0)	1.29 (0.67 to 2.48)	0.46	1.31 (0.66 to 2.60)	0.45
Days with symptoms (ref: ≥3)
<3	44 (57.1)	49 (74.2)	2.16 (1.06 to 4.41)	0.03	1.83 (0.86 to 3.88)	0.12
Urine dipstick leukocytes (ref: negative)
Positive	69 (89.6)	63 (95.5)	2.43 (0.62 to 9.58)	0.20	1.50 (0.31 to 7.26)	0.61
Urine dipstick blood (ref: negative)
Positive	55 (71.4)	55 (83.3)	2.00 (0.89 to 4.52)	0.10	1.54 (0.59 to 4.05)	0.38
Feeling unwell (ref: <4)
≥4	23 (28.6)	30 (45.5)	2.08 (1.04 to 4.16)	0.04	1.88 (0.91 to 3.86)	0.09
Urine dipstick nitrate (ref: negative)
Positive	12 (15.6)	15 (22.7)	1.59 (0.69 to 3.70)	0.28
Dysuria (ref: <4)
≥4	51 (66.2)	45 (68.2)	1.09 (0.54 to 2.20)	0.81
Urinary urgency (ref: <4)
≥4	60 (77.9)	51 (77.3)	0.96 (0.44 to 2.12)	0.93
Urinary frequency (ref: <4)
≥4	57 (74.0)	47 (71.2)	0.87 (0.42 to 1.81)	0.71
Antibiotic treatments last year (ref: <3)
≥3	6 (7.8)	8 (12.1)	1.63 (0.54 to 4.97)	0.39
Abdominal pain (ref: <4)
≥4	13 (16.9)	14 (21.2)	1.32 (0.57 to 3.07)	0.51

*Adjusted model based on predictors with p≤0.20 in the unadjusted analysis. Age was included as an essential variable.

Standard logistic regression

In the unadjusted binary logistic regression (table 1), the number of days with symptoms before inclusion (<3 days) and feeling unwell (≥4 on a scale of 0–6) were significant predictors for the subsequent use of antibiotic treatment. However, in a backward stepwise logistic regression (data not shown), only the number of days with symptoms was significantly associated with the use of antibiotics. At the 5% significance level, we did not find statistically significant predictors that were associated with the use of antibiotics in the adjusted model, presented in table 1.

LASSO logistic regression model

In the LASSO logistic regression model presented in table 2, the following predictors were selected: days with symptoms (<3/≥3), urine dipstick blood (negative/positive) and feeling unwell (<4/≥4). Patient age (<26/≥26 years) was forced into the model as a clinically relevant variable. We also explored the potential benefit of including two-way interaction terms. Their inclusion did not influence the predictive ability of the final model; hence, they were not considered. For a given woman, the regression coefficient estimates of the selected predictors were used to compute her predicted probability of receiving an antibiotic later on. Online supplementary file 1 shows how these predicted probabilities were computed. For example, a woman presenting with symptoms for less than 3 days before inclusion, feeling worse than moderately unwell (graded ≥4 on a scale from 0 to 6) at inclusion and with a positive urine dipstick for blood, had a 55.7% chance of needing antibiotics later on compared with a 49.3% predicted probability if she had had symptoms for 3 days or more prior to inclusion.

Table 2

Estimates of regression coefficients, obtained from the penalised method, LASSO

Predictors	Selected	Logistic regression coefficient estimates, LASSO (risk score)
Intercept	–	−0.64
Days with symptoms <3	Yes	0.26
Urine dipstick blood positive	Yes	0.15
Feeling unwell (≥4)	Yes	0.21
Age ≥26	No	0.22
Urine dipstick leukocytes	No
Urine dipstick nitrite	No
Dysuria	No
Urinary urgency	No
Urinary frequency	No
Antibiotic treatments last year	No
Abdominal pain	No

LASSO, least absolute shrinkage and selection operator.

The performance of the model in differentiating women who would subsequently use antibiotics from those who would not is shown in the area under the ROC curve in figure 2, with an AUC of 0.66 (95% CI: 0.57 to 0.74).

Figure 2

Receiver operating characteristic (ROC) curve for the performance of the least absolute shrinkage and selection operator risk score in identifying women who would subsequently use antibiotic treatment. The area under the ROC curve is 0.66 (0.57, 0.74).

Table 3 shows the accuracy of prediction for different risk score cut-off points. The risk score that maximised sensitivity and specificity was 0.45, with sensitivity of 75.8% (95% CI: 63.6 to 85.5) and specificity of 43.6% (95% CI: 32.4 to 55.3). A sensitivity of 75.8% implies that 75.8% of the women presenting with UTI were correctly recommended antibiotics, whereas a specificity of 43.6% means that 43.6% of the women, not recommended antibiotics, were able to manage their condition without antibiotics. At this risk score (>0.45), 65.3% of all the women presenting with UTI would be recommended antibiotics by the GP. However, treatment failure would be observed in 11.1% of all UTI cases forcing these women to return to the practices for treatment. A specificity of 43.6% also suggests that a large proportion of the women recommended antibiotic treatment would have been able to handle their condition without antibiotics.

Table 3

Sensitivity, specificity, predicted proportion of women who received antibiotics and the predicted proportion of women who would return to the practices due to treatment failure

Risk score cut-off point*	Sensitivity, %, (95% CI)	Specificity, %, (95% CI)	Antibiotics prescribed, %, (95% CI)	Returning patients, %, (95% CI)
>0.21	100.0 (94.6 to 100.0)	5.1 (1.4 to 12.6)	97.2 (93.0 to 99.2)	–
>0.30	98.5 (91.8 to 100.0)	16.7 (9.2 to 26.8)	90.3 (84.2 to 94.6)	0.7 (0.0 to 3.8)
>0.39	75.8 (63.6 to 85.5)	42.9 (31.6 to 54.6)	65.3 (56.9 to 73.0)	11.1 (6.5 to 17.4)
>0.45	75.8 (63.6 to 85.5)	43.6 (32.4 to 55.3)	65.3 (56.9 to 73.0)	11.1 (6.5 to 17.4)
>0.48	53.0 (40.3 to 65.4)	64.1 (52.4 to 74.7)	43.8 (35.5 to 52.3)	21.5 (15.1 to 29.1)
>0.57	28.8 (18.3 to 41.3)	83.3 (73.2 to 90.8)	22.2 (15.7 to 29.9)	32.6 (25.1 to 40.9)
>0.65	15.2 (7.5 to 26.1)	91.0 (82.4 to 96.3)	11.8 (7.0 to 18.2)	38.9 (30.9 to 47.4)

Estimates are given with 95% CIs at different cut-off points.

*Based on the sum of the logistic regression coefficients for each predictor.

Complications

In our study population, 12 patients developed a febrile UTI/upper UTI (5/7). We had complete baseline data for 10 of these 12 patients. All 10 had a urine dipstick positive for blood (value ≥3) and leucocytes,1–3 but nitrite negative. The mean age was 29, 6/10 had had symptoms for less than 3 days before inclusion, 9/10 reported a symptom burden of ≥4 for urinary urgency, 8/10 reported urinary frequency ≥4, 5/10 reported feeling unwell ≥4, 5/10 reported dysuria ≥4, only 1 reported abdominal pain ≥4 and none had had more than 3 antibiotic treatments during the last 12 months. Six patients had a prediction score of >0.45 and four had a score of <0.45. This means that if the suggested prediction rule had been used/followed, 60% would have accurately been prescribed antibiotics, whereas 40% would have been treated with ibuprofen and experienced complications.

Discussion

Main results

In our unadjusted analysis, the number of days with symptoms (<3) and degree of feeling unwell were significantly associated with the use of antibiotics, but in further analysis, no single factor was associated with the use of antibiotics. A scoring system based on four relevant predictors resulted in an area under the ROC curve of 0.66 (95% CI: 0.57 to 0.74).

We chose to dichotomise all continuous variables in the prediction model. This was a decision made after initially performing the analyses using continuous variables. Thereafter, we converted the continuous variables to categorical variables by grouping the values into two or three groups. There was little difference in the results. This led to the decision of dichotomising each continuous variable for simplicity. For each variable, we kept the cut-off value that yielded the biggest difference. For the age variable this cut-off value was 26 years, the median age in our study population, which also made sense clinically. Dichotomising continuous variables has been warned against, since this simplification often comes at the cost of creating a prediction model with less statistical power.22 Given the small difference in accuracy in our prediction model, we decided to use dichotomised variables in the final model.

Strengths and limitations

This is a relatively larger trial performed in an unselected population of women with presumably uncomplicated UTI. However, our study was not adequately powered to perform predictor analyses. We only looked at the patients receiving ibuprofen who had complete baseline data. Starting with 181 patients, only 143 patients, 79%, had complete baseline data, this might not be a large enough population to avoid type II error. The estimation of an appropriate sample size for clinical prediction models is challenging because one has to consider the sizes of the training set, cross-validation and the test set. However, there are different ways of calculating such a sample size, some approaches are described by Riley et al in a recent publication.23 Using the fact that we had 11 baseline predictors and an estimated binary outcome of 0.4 (the proportion of patients who would come back for an antibiotic treatment), their method would give an estimated sample size of 604 patients. We ended up with only 4 predictors in our final model, if we include only 4 predictors in the same calculation formula, a sample size of 369 patients is estimated.

Furthermore, our population was young compared with similar study populations,6 24 and a majority of the patients were from OOH services and may differ somewhat from an ordinary general practice population.

Comparison with other studies

Gagyor et al performed similar analyses to ours after their randomised trial.11 They developed a scoring system where using a relevant cut-off score for initiating antibiotic treatment would lead to 58% being treated with antibiotics and 42% with initial symptomatic treatment. This would still lead to some overprescribing, but only 6% of the women would come back for a second consultation potentially leading to an antibiotic prescription. Apart from a positive urine dipstick for erythrocytes and feeling unwell (similar to ‘impairment of regular daily activities’), our results did not confirm the predictors found in the Gagyor Study. We do not have a good explanation for this. The number of patients included in our analyses was smaller than the number of patients in the trial by Gagyor et al, this might make our results less reliable. The mean age in our trial population was lower than the mean age in the trial by Gagyor et al, this might also have lead to a difference in selection of predictors.

Identifying which patients will need antibiotics to treat their uncomplicated UTI and which patients can be recommended symptomatic relief is still challenging. Larger data sets are needed to develop more accurate prediction models eligible for use in primary care. An individual patient data (IPD) meta-analysis based on studies comparing placebo or NSAIDs to antibiotic treatment could help achieve this. Merging data from different trials for analysis purposes are challenging. The baseline data will differ somewhat between the trials, the scales used to measure symptoms and the outcome measures might have some variations. However, we believe that an IPD meta-analysis could provide more information, and give us a large enough sample size to develop a more accurate prediction model. Such an analysis might also enable us to distinguish between subgroups in the study populations, for example younger versus older women, and possibly find different predictors in the different subgroups. Previous studies aiming to predict which patients would need antibiotic treatment to manage their UTI have used a positive urine culture as the gold standard for defining a UTI. Several studies have been conducted to find predictors for a positive urine culture among women presenting with symptoms of a UTI. They have reached an overall conclusion that, apart from clinical symptoms, a urinary dipstick positive for nitrite and/or leukocytes, or positive for leukocytes and blood, is the only predictor with a high positive predictive value.25–27 However, the negative predictive values are relatively poor, hence using only the dipstick as a predictor will underestimate the need for antibiotic treatment. At the same time, it will overestimate the need for antibiotics, since several women with a positive urine culture will get well without antibiotic treatment.

Implications for practice

We were not able to find any baseline information that could significantly and accurately predict the need for subsequent antibiotic treatment. Our prediction model yielded an AUC of 0.66, and in clinical practice, it will not be of much help in identifying which patients should be treated with antibiotics. However, the 95% CI was above 0.5, and thus the selected predictors can potentially indicate the seriousness of the condition, a higher degree of feeling unwell, symptoms for less than 3 days combined with a positive urine dipstick for blood could indicate that initial treatment with ibuprofen will lead to treatment failure. This again indicates that patients experiencing milder symptoms could benefit from a wait-and-see strategy with initial symptomatic treatment.

Delayed prescribing has proven to reduce the use of antibiotics,9 and at least a third of women with symptoms of a UTI will be positive to such an approach.28 A systematic review conducted by Wawrysiuk et al concludes that using shared decision-making and delayed prescribing with advice to take NSAIDs for symptom relief can be a preferred strategy for patients with uncomplicated UTIs, but the patient should receive close follow-up.29 Given the current knowledge, recommending symptomatic relief to patients who are willing to delay taking antibiotics, with a backup prescription at hand, seems to be a safe and efficient strategy to lower antibiotic prescribing for women with uncomplicated UTIs.