Clinical Documentation to Predict Factors Associated with Urinary Incontinence Following Prostatectomy for Prostate Cancer.

BACKGROUND: Advances in data collection provide opportunities to use population samples in identifying risk factors for urinary incontinence (UI), which occurs in up to 71% of men with prostate cancer following prostatectomy. Most studies on patient-centered outcomes use surveys or manual chart abstraction for data collection, which can be costly and difficult to scale. We sought to evaluate rates of and risk factors for UI following prostatectomy using natural language processing on electronic health record (EHR) data.
METHODS: We conducted a retrospective analysis of patients undergoing prostatectomy for prostate cancer between January 2008 and August 2018 using EHR data from an academic medical center. UI incidence for each patient in the cohort was assessed using natural language processing from clinical notes generated pre- and postoperatively. Multivariable logistic regression was used to evaluate potential risk factors for postoperative UI at various time points within 2 years following surgery.
RESULTS: We identified 3792 patients who underwent prostatectomy for prostate cancer. We found a significant association between preoperative UI and UI in the first (odds ratio [OR], 2.30; 95% confidence interval [CI], 1.24-4.28) and second (OR 2.24, 95% CI 1.04-4.83) years following surgery. Preoperative body mass index was also associated with UI in the second postoperative year (OR 1.11, 95% CI 1.02-1.21).
CONCLUSION: We show that a natural language processing approach using clinical narratives can be used to assess risk for UI in prostate cancer patients. Unstructured clinical narrative text can help advance future population-level research in patient-centered outcomes and quality of care.

Introduction

Prostate cancer is diagnosed in 180,000 men every year, and one in seven men is affected in his lifetime.1 Potentially curative interventions such as surgery may have significant treatment-related morbidity that negatively impacts quality of life. Urinary incontinence (UI) is common, and up to 71% of all men may require the use of absorbent pads following radical prostatectomy.2 Because ten-year survival rates in prostate cancer exceed 99%, reduction in treatment-related morbidity can have profound and long-lasting effects. Recent efforts to identify risk factors for UI following treatment aim to facilitate informed treatment decisions and increase patient engagement in medical decision-making.2–5

Prior work has found that factors associated with UI following treatment include treatment modality and pre-treatment urinary function.4–6 Much of this research was conducted using validated health-related quality of life instruments such as the EPIC-26,7 which can be time-consuming and costly to administer, are difficult to scale and administer in routine clinic workflow, can lack generalizability due to limited sample size, and can be subject to ascertainment bias.

The increased adoption of electronic health records (EHR) offers new opportunities to examine risk factors for UI following prostate cancer treatment.8 Previous studies have shown that detail regarding a patient’s experience of UI is often documented in free-text, unstructured EHR elements such as clinician notes, which can be parsed with data mining methods such as natural language processing (NLP) to extract the incidence of the outcome.9 As structured data (e.g., billing codes) analysis alone may not sufficiently capture outcomes like UI that are not routinely coded, data mining methods may improve the robustness of research on these patient-centered outcomes (PCOs) and on larger patient populations, which could yield more generalizable findings.10

Previously, we have described an NLP approach for identifying UI following prostatectomy that showed high concordance with patient-reported outcomes captured using EPIC-26.11 In this study, we use the developed NLP pipeline to evaluate rates of UI following prostatectomy for prostate cancer and to explore risk factors for postoperative UI, including preoperative urinary function. This evidence and approach can be used to guide shared decision-making and advance patient-reported outcomes research.

Methods

Data Sources

Data were extracted from a prostate cancer clinical data warehouse described elsewhere.12 In brief, prostate cancer patients were identified using diagnosis codes (ICD-9: 185, 233.4; and ICD-10: C61) from EPIC Clarity, which was installed in 2008. Patients were linked to the California Cancer Registry (CCR), a state-wide population-based cancer surveillance program that provides tumor characteristics and additional information on treatments outside of our cancer center. Captured variables included patient demographics, diagnoses, procedures, cancer-specific characteristics, and all clinical narrative text.

Ethics Statement

This study was reviewed and approved by Stanford University’s Institutional Review Board. No patients were involved in setting the research question, study design, or outcome measures. Protected Health Information is not reported in this study and we confirm patient data confidentiality.

Study Population

We conducted a retrospective analysis of patients undergoing prostatectomy between January 2008 and August 2018. Patients were included if they had been previously diagnosed with prostate cancer and had undergone prostatectomy as identified by CPT and by ICD-9 and -10 procedure codes (). Procedure codes were inclusive of subtotal and radical procedures, perineal and retropubic techniques, and open and percutaneous approaches. To prevent confounding, patients were excluded if they underwent radiation therapy in addition to surgery.

Electronic Health Record Processing

We assessed UI for each patient using an NLP pipeline that annotates EHR free-text notes as previously reported.13 This open-source pipeline cleans EHR free-text notes and extracts sentences containing at least one of 61 unique terms indicative of urinary incontinence, a dictionary that was created by a group of urology professionals at our institution (). The pipeline does not require manually labeled text for training the NLP model. We created the training data heuristically by exploiting the preexisting domain knowledge (dictionary) to address the issue of inter-rater variability (as low as 0.62) observed during the annotation procedure. In order to learn the language space for context-aware vectorization of the words, the pipeline uses a neural word embedding model that was trained on 528,362 clinical notes including progress notes, discharge summaries, telephone call notes, and radiology reports. The final note classification model achieved an average f1-score of 0.86 on hold-out test data (117 notes). A more detailed analysis of NLP performance can be found elsewhere.13

Explanatory Variables

Preoperative UI symptoms in the 4 months prior to the date of the surgery were studied as a primary explanatory variable given previous findings connecting pre-treatment urinary function with postoperative UI. Other covariates of interest were chosen based on previous association with PCOs following surgery for prostate cancer. Demographic variables such as age at diagnosis, race, and preoperative BMI have been found to affect urinary function and quality of life after treatment for prostate cancer.14,15 Relative comorbidity burden was expressed using the Charlson Comorbidity Index, which encompasses 19 categories of comorbidity with a single score that reflects association with mortality.16 Comorbidities such as coronary artery disease, hypertension, and diabetes mellitus have also been associated with anatomic factors that increase the risk of postoperative UI, such as stricture of the anastomosis between the bladder neck and the urethra.17 Additional covariates included insurance type and tumor characteristics such as stage and Gleason grade group.

Outcome Variable

Using the NLP pipeline, we assessed whether UI was present at three-month timepoints within the first 2 years following surgery. The presence of UI was defined as NLP-identified documentation of symptoms within 45 days on either side of each time point. Using the pipeline, UI can be affirmed, negated, or inconclusive. Composite time points were additionally created for the first and second years after treatment. Missing or inconclusive values were handled using a modified last-observation-carried-forward approach that is executed only if the last observation negates UI. This imputation method assumes that clinicians do not document the absence of UI following a prior denial of symptoms.

Statistical Analyses

Summary statistics for the cohort are reported as frequencies for categorical variables and medians and interquartile ranges for continuous variables. Univariable analyses assessed UI frequency among evaluable patients at each time point, stratified by baseline UI status. The number of evaluable patients at each time point consists of those with affirmed or negated UI and does not include patients for whom symptoms cannot be determined. Error was expressed as a 95% confidence interval at each time point. Logistic regression was used to evaluate the effect of factors such as baseline UI status on UI risk in the first and the second years following surgery to account for effect on early and late symptoms. The strength of association between covariates and outcome was represented as an odds ratio (OR) with 95% confidence intervals (CI) and a significance level of p = 0.05. All analyses were conducted using R (version 3.4.1; R Foundation for Statistical Computing, Vienna, Austria).

Results

Cohort Characteristics

We identified 4367 patients in our prostate cancer cohort who were diagnosed with prostate cancer and underwent surgery between January 2008 and August 2018. Of these 4367 individuals, 565 (12.9%) were deemed ineligible as they had received multiple modalities of treatment, and 10 (0.2%) were excluded due to ambiguous source data. Overall, 3792 (86.8%) of 4367 patients were included in the final study cohort (Table 1). At diagnosis, 71.6% of patients in the final study cohort had stage I or II disease, 22.1% of patients had stage III or IV disease, and remaining patients had disease of unknown stage. Preoperative diagnosis and staging approach were performed per provider preference and were not captured in the database.

Table 1

Characteristics of the Final Cohort of Patients Undergoing Surgery as Primary Treatment for Prostate Cancer

Total	3792
Age at first treatment (median [IQR])	63.00 [58.00, 68.00]
Diagnosis year (%)
2005	260 (6.9)
2006	325 (8.6)
2007	332 (8.8)
2008	319 (8.4)
2009	325 (8.6)
2010	354 (9.3)
2011	330 (8.7)
2012	294 (7.8)
2013	238 (6.3)
2014	247 (6.5)
2015	266 (7.0)
2016	234 (6.2)
2017	209 (5.5)
2018	59 (1.6)
Race (%)
White	323 (8.5)
Asian	161 (4.2)
Black	323 (8.5)
Hispanic	72 (1.9)
Other	2913 (76.8)
BMI (median [IQR])	27.07 [24.68, 29.83]
Insurance type (%)
Medicaid	70 (1.8)
Medicare	1657 (43.7)
Private	1720 (45.4)
Unknown	345 (9.1)
Stage (%)
I	249 (6.6)
II	2463 (65.0)
III	745 (19.6)
IV	93 (2.5)
NA	242 (6.4)
Grade group (%)
1: Gleason ≤ 6	1102 (29.1)
2: Gleason 3+4=7	1654 (43.6)
3: Gleason 4+3=7	590 (15.6)
4: Gleason 8	146 (3.9)
5: Gleason 9–10	191 (5.0)
NA	109 (2.9)
Charlson (median [IQR])	2.00 [2.00, 3.00]
Baseline UI (%)	154 (4.1)

In the 4 months prior to treatment, 154 of 474 (32.5%) evaluable surgery patients had documented UI (Figure 1). Relative to the baseline period, UI frequency at 3 months following treatment increased to 250 of 424 evaluable (59.0%) surgery patients (p < 0.001 vs baseline).

Figure 1

Prevalence of urinary incontinence: at baseline (pre); at 3, 6, 9, 12, 15, 18, 21, and 24 months following surgery; and at composite time points for the first year (YR1, 3- through 12-month time points) and second year (15- through 24-month time points) following surgery.

UI and Pre-Treatment Function

82 of 113 evaluable surgery patients (72.6%) with baseline UI symptoms had documented UI in the first year following treatment, compared to 91 of 178 (51.1%) evaluable surgery patients with no baseline UI symptoms (p < 0.001) (Figure 2). In the second year following treatment, UI was noted for 28 of 69 (40.6%) evaluable surgery patients with preoperative UI compared to 35 of 134 (26.1%) evaluable surgery patients without baseline UI (p = 0.051). Univariable logistic regression demonstrated a statistically significant association between preoperative UI status and UI in the first year (OR 2.53, 95% CI 1.52–4.20, p < 0.001) and second year (OR 1.93, 95% CI 1.04–3.58, p = 0.036) following surgery.

Figure 2

Prevalence of urinary incontinence (UI), stratified by pre-treatment/baseline UI symptoms: at 3, 6, 9, 12, 15, 18, 21, and 24 months following surgery; and at composite time points for the first year (YR1, 3- through 12-month time points) and second year (15- through 24-month time points) following surgery.

Risk Factors for Post-Treatment UI

Multivariable logistic regression was performed to assess the association between UI in the first and second years following treatment with explanatory variables including age, ethnicity, BMI, insurance type, disease stage and grade, Charlson Comorbidity Index, and baseline UI status (Table 2).

Table 2

Multivariable Regression of Risk Factors for UI in the First and Second Years Following Treatment

	Year 1 After Treatment				Year 2 After Treatment
Covariate	OR	95% CI Low	95% CI High	p	OR	95% CI Low	95% CI High	p
Age at first treatment	1.00	0.95	1.05	0.977	1.00	0.94	1.07	0.902
Race
White	Reference				Reference
Asian	0.50	0.16	1.57	0.235	1.53	0.39	5.95	0.544
Black	6.26	0.67	58.13	0.107	7.33	0.82	65.99	0.075
Hispanic	1.46	0.49	4.36	0.494	1.12	0.28	4.48	0.871
Other	0.34	0.05	2.27	0.267	0.21	0.02	2.32	0.202
Insurance type
Medicaid	Reference				Reference
Medicare	0.47	0.09	2.41	0.365	7.06	0.59	84.30	0.123
Private	0.26	0.05	1.41	0.119	4.60	0.37	57.85	0.238
Unknown	0.43	0.06	3.16	0.405	1.65	0.06	45.01	0.766
BMI	0.99	0.93	1.07	0.864	1.11	1.02	1.21	0.019
Stage1
I	Reference				Reference
II	0.53	0.11	2.56	0.431	1.29	0.15	11.09	0.815
III or IVa	0.57	0.11	2.99	0.504	0.79	0.08	7.60	0.838
Grade group
1: Gleason ≤ 6	Reference				Reference
2: Gleason 3+4=7	1.39	0.66	2.96	0.387	1.84	0.71	4.76	0.209
3: Gleason 4+3=7	1.85	0.69	4.91	0.219	3.26	0.98	10.84	0.054
4: Gleason 8	0.44	0.08	2.44	0.349	1.86	0.23	15.14	0.561
5: Gleason 9–10	0.72	0.10	5.23	0.746	1.81	0.14	24.18	0.653
Charlson score	1.03	0.87	1.23	0.711	1.02	0.78	1.32	0.903
Pre-treatment UI	2.30	1.24	4.28	0.008	2.24	1.04	4.83	0.039

Notes: aStage III and stage IV patients were consolidated in multivariable regression, as there were few observations for stage IV.

After controlling for these variables, we found a statistically significant association between UI in the first year following treatment with baseline UI (OR 2.30, 95% CI 1.24–4.28, p = 0.008). We also found a statistically significant association between UI in the second year following treatment with baseline UI (OR 2.24, 95% CI 1.04–4.83, p = 0.039) and BMI (OR 1.11, 95% CI 1.02–1.21, p = 0.019). We did not find statistically significant independent associations between UI in the first or second year following treatment with age at treatment, race, insurance type, higher disease stage, Gleason grade group, or Charlson score.

Discussion

In this analysis of patients who underwent prostatectomy for prostate cancer, we found that the presence of UI prior to surgery is an independent predictor of increased risk of UI in both the first and the second years following surgery. Our measured prevalence of UI is similar to that reported in studies that have assessed UI prevalence using instruments such as the EPIC-26 and the Prostate Cancer Symptom Indices.6,18 Our work complements these results using a different data source and suggests that use of novel EHR data mining approaches produces findings consistent with those obtained from survey instruments.

In previous work, Chen et al found that, among patients undergoing radical prostatectomy, patients with better urinary functioning at baseline were less likely to have problematic UI in the years following treatment.6 The study investigators—who classified the severity of UI as normal (no symptoms), intermediate (symptoms are present but not distressful), and poor (symptoms are distressful)—reported that a higher percentage of the 72 patients with intermediate baseline urinary function had intermediate or poor UI at 24 months (84.7%) compared to the 288 patients with normal baseline urinary function (65.7%). The prevalence of poor UI was similar among patients with normal (15.8%) and intermediate (17.3%) baseline function, however. We report prevalence of UI at 24 months as 25.0% and 17.2% at 24 months for patients with and without baseline UI, which is more in line with “poor” UI as defined by Chen et al and may suggest that symptoms captured in our data mining approach—and perhaps, by extension, free-text clinical notes—are more severe or distressing to the patient. Barocas et al similarly found using EPIC-26 among a sample of 1523 postoperative patients that those with excellent baseline UI domain scores had a higher mean domain score at 3 years following radical prostatectomy compared to patients with poor baseline domain scores.18

We also found a statistically significant association between increasing BMI and UI in the second year following treatment. The role of BMI in urinary continence is controversial. Wiltz et al reported that men with a BMI > 30 more frequently had perioperative complications, including prolonged operative time and increased blood loss, and were more likely to have worse urinary symptom scores at 3 and 9 months following robotic prostatectomy.19 On the other hand, Nilsson et al found no association of BMI with UI prevalence in a series of 1411 men following radical prostatectomy.20 We found a relatively weak association between BMI and UI risk in the second year following surgery, and it is possible that differences in patient populations, surgical techniques, or chance might explain differences in findings between studies.

We notably did not find a statistically significant association between UI and age at the time of surgery, despite previous reports demonstrating age to be a strong predictor for return of continence following radical prostatectomy.17 Nilsson et al have described an exponential relationship between age at surgery and UI prevalence after prostatectomy, although only showed a statistically higher risk in the ≥70 age group compared to the ≤54 age group.20 Black race has similarly been associated with worse quality-of-life following treatment for prostate cancer,15 and we detected a trend, albeit not significant, suggesting a positive association with postoperative UI.

One explanation for the different results produced by our analysis is data missingness. Many patients in our cohort lacked documentation of UI-related symptoms during the 2-year postoperative follow-up period, decreasing the robustness with which we could assess UI frequency, conduct multivariable analyses, and identify novel risk factors for postoperative UI. This highlights the inherent challenge of using the EHR as a data source, as outcomes can be assessed only insofar as they are sufficiently documented in the medical record. While we have previously shown concordance of UI reporting between patients and clinicians for postoperative UI,11 other studies have suggested that urologist and patient assessment of symptoms can often disagree.21 Continued efforts to incorporate machine learning approaches in outcomes analyses will, therefore, require concurrent focus on ensuring appropriate input data quality.

An additional limitation is the accuracy with which UI was identified among the EHR unstructured text by the construction of the NLP pipeline itself. Banerjee et al showed that this NLP pipeline had 89% precision and 84% recall in a validation data set, with f1-score of 0.86. Imperfect precision may lead to non-differential misclassification of exposure that could bias association analyses toward the null hypothesis. Imperfect recall may contribute to data missingness. Regardless of performance, the model will understate the true prevalence of UI if either clinicians or patients do not report all symptoms, or if the manner in which UI is documented is not captured by the design of the pipeline.13 Second, this study was performed at a single institution, which limits the generalizability and the power of this analysis. Future studies in other healthcare settings and institutions could allow us to assess the reproducibility of NLP-derived findings given potential variability in physician documentation and patient population. Regardless, our algorithm was able to identify associations of UI with preoperative urinary status and BMI that have been reported using surveys and manual chart reviews.

Our results highlight the opportunity to generate both real-time and large-scale outcomes analysis from routinely collected EHR data.22 Even with potential limitations of mining the EHR, our methods assess patient-centered outcomes previously measurable only through time- and resource-intensive approaches such as EPIC-26 or chart review.7 Electronic patient-reported outcomes systems can allow for continuous monitoring and integration of PCO data into clinical practice and may ultimately facilitate the creation of decision tools to communicate individualized postoperative UI risk with patients.23 NLP and related approaches, therefore, have the potential to supplement existing instruments such as the EPIC-26 in assessing the risk of UI and other PCOs following prostate cancer treatment and could be used in large EHR systems to estimate PCOs at the population level.

Conclusion

Mining clinical narrative text of an EHR can provide population-level information on important patient-centered outcomes such as urinary incontinence after prostate surgery. By linking mined and structured data, we can produce rich datasets that are useful in assessing risk factors for outcomes such as urinary incontinence in prostate cancer care, which ultimately may help in evaluating and improving the quality of care in urologic oncology as well as other clinical settings.