External validation of a magnetic resonance imaging-based algorithm for prediction of side-specific extracapsular extension in prostate cancer.

INTRODUCTION: Recently developed algorithm for prediction of side-specific extracapsular extension (ECE) of prostate cancer required validation before being recommended to use. The algorithm assumed that ECE on a particular side was not likely with same side maximum tumor diameter (MTD) <15 mm AND cancerous tissue in ipsilateral biopsy <15% AND PSA <20 ng/mL (both sides condition). The aim of the study was to validate this predictive tool in patients from another department.
MATERIAL AND METHODS: Data of 154 consecutive patients (308 prostatic lateral lobes) were used for validation. Predictive factors chosen in the development set of patients were assessed together with other preoperative parameters using logistic regression to check for their significance. Sensitivity, specificity, negative and positive predictive values were calculated for bootstrapped risk-stratified validation dataset.
RESULTS: Validation cohort did not differ significantly from development cohort regarding PSA, PSA density, Gleason score (GS), MTD, age, ECE and seminal vesicle invasion rate. In bootstrapped data set (n = 200 random sampling) algorithm revealed 70.2% sensitivity (95% confidence interval (CI) 58.8-83.0%), 49.9% specificity (95%CI: 42.0-57.7%), 83.9% negative predictive value (NPV; 95%CI: 76.1-91.4%) and 31.1% positive predictive value (PPV; 95%CI: 19.6-39.7%). When limiting analysis to high-risk patients (Gleason score >7) the algorithm improved its performance: sensitivity 91%, specificity 47%, PPV 53%, NPV 89%.
CONCLUSIONS: Analyzed algorithm is useful for identifying prostate lobes without ECE and deciding on ipsilateral nerve-sparing technique during radical prostatectomy, especially in patients with GS >7. Due to significant number of false positives in case of: MTD ≥15 mm OR cancer in biopsy ≥15% OR PSA ≥20 ng/mL additional evaluation is necessary to aid decision-making. Copyright by Polish Urological Association.

INTRODUCTION

Radical prostatectomy constitutes mainstay curative treatment in men with organ-confined prostate cancer [1] as well as in selected patients with locally-advanced disease [2]. Functional outcomes of the surgery, including postoperative continence and potency, might be substantially improved utilizing nerve-sparing [3, 4]. However this approach should be limited to patients in whom tumor does not extend through prostate capsule [5]. Extracapsular extension (ECE) is an independent risk factor of positive surgical margins and might affect long-term biochemical status [4]. Increased understanding of the effect of ECE on oncological outcomes and the increasing feasibility of unilateral nerve-sparing yielded several predictive tools, that can be used preoperatively to avoid overqualification to nerve-sparing in men suffering from locally advanced disease [6, 7, 8]. We have previously described development and internal validation of logistic-regression-based binary algorithm for safe, side-selective initial qualification to nerve sparing in patients staged preoperatively with mpMRI [9]. The aim of this study was to externally validate our predictive model to assess, whether the results may be generalized to a broader population of patients, who undergo radical prostatectomy with an intention to spare neuro-vascular bundles.

MATERIAL AND METHODS

Patients selection and data collection

Both development and validation study were approved by relevant ethics committees. A development cohort involved 88 patients (176 lobes) [9], whereas validation cohort involved 154 patients (308 lobes) with prostate cancer who underwent laparoscopic radical prostatectomy with preoperative staging using 3T mpMRI in another department. The images were interpreted by a single experienced radiologist specialized in genitourinary tract diagnostics, who described suspicious prostatic lesions using PIRADSv2 system, according to ESUR recommendations [10]. This resulted in 308 records, because every lobe was considered a separate case, according to the methodology used in the development cohort from the original publication [9]. Patients selection in terms of exclusion and inclusion criteria were as previously described for development cohort [9]. The following clinical variables were extracted for the validation cohort: age, serum prostate specific antigen (PSA), prostate volume, PSA density (PSAD). An extent of tumor infiltration was assessed for each prostate lobe separately using the following variables: number of positive cores, percentage of positive cores, percentage of cancer in total biopsy specimen and Gleason score. If no cancer was found in the lobe at biopsy, all biopsy-derived variables for this side were counted as zero. Consequently, for each side of RP specimen, Gleason score, surgical margins status and pathological stage were reported separately. The analysis of prostate specimens was performed according to the International Society of Urological Pathology guidelines (2014) and the TNM classification was used.

Local staging in the validation cohort was performed using Achieva 3.0-T MRI TX (Philips, Amsterdam, The Netherlands) with dual RF transmitter and 32 independent receiving channels with a multichannel phased-array coil. An endorectal coil was not used. As in the development cohort, examinations included T2-weighted imaging, diffusion-weighted imaging (DWI), and dynamic contrast-enhanced imaging (DCE) carried out according to the European Society of Urological Radiology guidelines. The MRI protocol included: axial T2-weighted turbo spin echo sequence, axial diffusion-weighted imaging spin echo sequence with apparent diffusion coefficient map, axial dynamic contrast-enhanced imaging, axial T1-weighted spin echo with selective fat suppression sequence, axial T1-weighted turbo field echo sequence, coronal and saggital T2-weighted turbo spin echo sequence in all cases and magnetic resonance spectroscopic imaging in selected cases. In each case mpMRI was performed more than 4 weeks after prostate biopsy.

Examinations in the validation cohort were evaluated by a single radiologist, who was not blinded to the clinical characteristics. The Prostate Imaging Reporting And Data System (PIRADS) version 2 was used to assess lesions. Using this system every suspicious lesion identified in the gland was scored from 1 to 5 based on specific radiologic signs found in mpMRI sequences. Subsequently, an overall score, analogously to Likert scale, was given for each lesion. MTD (maximum tumor diameter) was defined as the largest diameter of a lesion defined PIRADS 3-5. Indications for prostate biopsy included: PSA elevation (>4 ng/mL) and/or abnormal DRE. Tru-Cut prostate biopsy (TRUS core-Bx) was performed systemically and guided with transrectal ultrasound. In case of suspicious lesion visible in TRUS or prebiopsy MRI, an additional targeted core was taken. Histopathological evaluation of biopsy cores and postprostatectomy specimen was as described before [11].

Ethical approval and informed consent

All procedures performed during the study were in accordance with the ethical standards of the institutional and national research committee and with the 1964 Helsinki declaration and its later amendments. Ethical board of Medical University of Warsaw has accepted the retrospective design of the study (approval number AKBE/46/17). Informed consent was obtained from all individual participants included in the study.

Statistical analysis

Continuous variables are presented as medians with corresponding interquartile ranges (IQR). Univariate and multivariate logistic regression models were utilized to confirm the variables from primary model as independent predictors of extracapsular extension. Variables were categorized based on cut-offs identified in development cohort [9]. Area under receiver operating characteristic curve (AUC), sensitivity, specificity, positive (PPV) and negative predictive value (NPV) of the algorithm and its components were identified for bootstrapped cohort. Analyses were performed using SAS 9.4 software (SAS Institute, Cary, USA) and R program (version 3.5.3, the R foundation for Statistical Computing, http://www.r-project.org) with foreign, rms, pROC and ResourceSelection packages was used to perform statistical analysis. The threshold for significance was set at p <0.05.

RESULTS

The development cohort [9] included 88 patients (176 lobes) whereas validation cohort included 154 patients (308 lobes). Mean age in the validation cohort was 63.6 (IQR = 8) and mean PSA was 10.7 ng/mL (IQR = 5.8). In a total of 71 (23.1%) and 21 (6.8%) postprostatectomy lobes specimens ECE and SVI were present respectively. Validation cohort included patients with smaller mean prostate volume (39.9 mL vs 45.5 mL, p = 0.003) but was not significantly different from development cohort regarding age (63.6 vs 63.3 years, p = 0.8), PSA 10.7 vs 10.5 ng/mL (p = 0.9), PSA density (0.30 vs 0.25 ng/ml^2, p = 0.16), total cancerous tissue in biopsy cores in each lobe (22.4% vs 18.1%, p = 0.5), MTD in each lobe (8.9 mm vs 10.1 mm, p = 0.19), ECE prevalence (23.1% lobes vs 30.1%, p = 0.1), SVI prevalence (6.8% lobes vs 8.7%, p = 0.47), biopsy Gleason score ≥7 (32.1% lobes vs 34.1%, p = 0.69) or biopsy Gleason score ≥ 8 prevalence (6.5% lobes vs 8.5%, p = 0.46). In summary, both cohorts did not differ significantly regarding commonly accepted risk factors for ECE.

Preoperative characteristics of validation cohort are summarized in Table 1. Preoperative characteristics of the development cohort can be found in previous manuscript [9].

Table 1

Baseline preoperative characteristics of validation cohort

PSA	<10 ng/mL	107 (69.9%)
	10–20 ng/mL	32 (20.9%)
	≥20 ng/mL	14 (9.15%)
ISUP 2014*	Grade group I	133 (57.33%)
	Grade group II	59 (25.43%)
	Grade group III	20 (8.62%)
	Grade group IV	10 (4.31%)
	Grade group V	10 (4.31%)
PSA density	<0.15	42 (30%)
	0.15-0.30	58 (41.43%)
	>0.30	40 (28.6%)

ISUP 2014 GG – International Society of Uropathology 2014 Gleason Groups in biopsy; PSA – prostate specific antigen [ng/mL]; PSAD – prostate specific antigen density [ng/mL4]

side-specific data

Logistic regression

To confirm primary model components as independent predictors of ECE, logistic regression was implemented. Uni- and multivariate analysis is presented in Table 2.

Table 2

Univariate and multivariate logistic regression for prediction of extracapsular extension

Variable	Univariate		Multivariate
	OR (95% CI)	p	OR (95% CI)	p
ISUP 2014 GG1 GG2 GG3 GG4 GG5	12.3 (1.2–4.7)1.7 (0.6–4.6)1.8 (0.6–5.0)3.9 (1.4–11.0)	NSNSNSNS
Age	0.99 (0.97–1.03)	NS
Pvol	1 (0.99–1.01)	NS
MTD [mm]	1.07 (1.04–1.09)	<0.0001
MTD ≥15 mm	4.1 (2.6–6.5)	<0.0001	1.6 (0.8–3.3)	0.19
PSA	1.06 (1.04–1.09)	<0.0001
PSA ≥20 ng/mL	7.9 (4.0–15.5)	<0.0001	3.9 (1.6–9.4)	0.003
PSAD	4.4 (2.1–9.5)	0.0001
Total % of cancer in cores	1.02 (1.02–1.03)	<0.0001
Total % of cancer in cores ≥15%	3.8 (2.4–5.9)	<0.0001	2.0 (1.6–3.6)	0.027

ISUP 2014 GG – International Society of Uropathology 2014 Gleason Groups in biopsy; Pvol – prostate volume in mpMRI [mL]; MTD – maximum tumor diameter of PIRADS ≥3 lesion [mm]; PSA – prostate specific antigen [ng/mL]; PSAD – prostate specific antigen density [ng/mL4]; OR – odds ratio; CI – confidence interval

Multivariate model was constructed from previously implemented predictors (PSA >20 ng/mL, total percentage of cancerous tissue in biopsy cores >15% and MTD >15 mm). The effects of the predictors were lower in the validation sample comparing to the development sample. PSA and percentage of biopsy cancer were confirmed as significant independent predictors and MTD as predictor presenting tendency to significance with 1.9 OR of ECE.

Finally, we merged samples to calculate more stable estimation of the effects of the predictors for the entire cohort. Coefficients β were smaller than expected in the validation cohort, which suggested that we tried to validate an overfitted model, but overfitting of the development model was excluded in the original study. The final model form a combined dataset (development + validation) revealed good performance. The regression coefficients in the final model are a compromise between the estimates in the development and validation sample (Table 3).

Table 3

Effects of predictors and discrimination indexes in development, validation and combined model

Model	Development		Validation		Combined
	OR	p-value	OR	p-value	OR	p-value
PSA ≥20 ng/mL	14.3	0.0013	3.9	0.003	5.0	<0.0001
% of cancer in cores ≥15%	7.5	<0.0001	2.0	0.027	3.1	<0.0001
MTD ≥15 mm	7.0	<0.0001	1.6	0.19	3.1	<0.0001
Discrimination indexes
C*	0.858		0.641		0.744
R^2*	0.499		0.098		0.241

C-index – counted from area under the ROC curve; reproduces diagnostic accuracy (0-1). R^2 – provides a measure of the proportion of the variance; the larger it is, the more the variance of the dependent variable is explained by the regression model (0-1); OR – odds ratio; MTD – maximum tumor diame-ter; PSA – prostate specific antigen

For validation of clinical implementation of the model sensitivity, specificity, positive and negative predictive values were calculated for bootstrapped risk-stratified validation cohort (Table 4).

Table 4

Sensitivity, specificity, PPV and NPV (bootstrap n = 200)

Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
Development cohort
91% (83–92)	74% (65–98)	54% (44–65)	94% (89–99)
Validation cohort* (n = 308)
70% (58–83)	50% (42–58)	31% (20–40)	84% (76–91)
Validation cohort (GS = 6)** (n = 152)
60 (40–76)	51 (44–59)	19 (13–27)	86 (78–93)
Validation cohort (biopsy GS = 7)*** (n = 126)
71 (61–81)	49 (42–56)	38 (31–45)	79 (71–86)
Validation cohort (biopsy GS >7)**** (n = 30)
91 (82–97)	47 (37–59)	53 (41–63)	89 (80–96)

Area under curve:

0.65

0.60

0.62

0.81

DISCUSSION

In the present study we externally validated previously developed side-specific algorithm predicting extracapsular extension in patients who underwent preprostatectomy mpMRI [9]. Validated model utilizes D’Amico definition of PSA high-risk (≥20 ng/mL) and side-selective variables that describe cancer volume: maximum tumor diameter of suspicious lesion measured in mpMRI and total percentage of cancerous tissue in biopsy cores. Based on nonlinear associations detected in development cohort [9], predefined cut-offs were used to binarize the model. We confirmed fairly good performance of the nomogram in the validation cohort from the external center.

The idea of introducing binary model was oriented on frequently unclear risk cut-offs of continuous models and clinical utility of simple risk grouping like presented in D’Amico groups [12], that remain crucial preoperative risk assessment despite constant development of more accurate multivariable models. However, the proposed nomogram was a combination of the nonlinear predictors and, as such, this formula may not be easily reproduced to calculate outcome predictions for new patients. This underlines the need of meticulous validation before broader use. Since outcomes of multiparametric MRI software analysis have been proved to correlate with D’Amico scoring [13], supplementing risk groups with objective MRI-derived measurements seems to be clinically reasonable. Although we observe constant advances in standardization of preoperative staging mpMRI [11, 14], its role in reducing positive margins, especially considering shift of surgical feasibility [15], remains to be confirmed [16] keeping risk modeling the issue of scientific interest.

The idea of replacing experience-based physician opinion with mathematical models has well established scientific explanation [17]. Staging model invented by Partin et al. has become integral part of preprostatectomy track and has been consequently updated among with stage migration due to PSA screening and mpMRI implementation [6]. The main limitation of Partins straightforward, extensively validated risk calculator is lack of side-specificity. Considering extended lymphadenectomy side-specificity is of less significance [18]. To correctly perform the nerve-sparing surgery a planning method needs a change from ‘patient-approach’ to ‘lobe-approach’. The side-specific prediction has been proposed in multiple regression-based models. First introduced by Ohori et al included almost all side-specific biopsy volumetric markers as well as grading and PSA [19] yielding high AUC of 0.773 in external validation [20]. Recently, Sayyid et al have proposed a similar novel nomogram based on clinical side-specific variables [7]. External validation of the tool yielded high AUC of 0.74 and excellent calibration. However, this model is based on multiple subjective variables (TRUS lesion, DRE staging) and neglects preoperative mpMRI assessment, therefore its clinical implementation might differ depending on department. MRI as independent adjunct to clinical data has been introduced by Feng et al. [21], by supplementing previous Ohori model with staging mpMRI. Sensitivity and negative predictive value of the updated model have reached 85% and 95% respectively, whereas AUC increased from 0.86 to 0.94. Noteworthy, validation probe was modest (112 patients), so this encouraging results should be interpreted with caution due to potential calibration issues. Recently, Alessi et al. released a nomogram based on the scales proposed by ESUR in the interpretation of MRI [22]. They have proven, that PIRADS v2 assessment categories of 3 or less rule out the presence of ECE with great certainty among all PCa-risk groups. However, due to markedly lower specificity of ESUR scoring systems among intermediate to high PCa-risk group it can be assumed that many of these men would be incorrectly disqualified from NVB-sparing surgery, had to rely only on MRI parameters. Such nomograms also have a serious limitation, as they do not assess a specific risk of ECE for each side of the gland. The results obtained in this way, however encouraging, may not translate into clinical practice. An update of the MSKCC nomogram with MRI data provided little, if any, incremental value to risk assessment of ECE improving AUC by only 0.03 [23]. Therefore, MRI results should always be interpreted together with clinical parameters.

In contrast to majority of previously continuous models (6–8,18–21), the tool we validated in present study generates direct decision instead of percentage risk as outcome. Obviously, such approach further reduces role of physician judgement. On the other hand supporting decision with the algorithm does not exclude supplemental use of more accurate tool like nomograms in second step. In fact, by risk grouping based on volumetric derivates and PSA we intended to determine situation in which extracapsular extension can be safely ruled out with variables that remain objective regardless of local experience and preoperative preparation policy. Volume of cancerous tissue in biopsy seems to meet this condition at first place. Since mpMRI determined maximum tumor diameter seems to be more objective and less experience dependent than complete staging assessment and simultaneously should be included in every PIRADS report [11], we believe this variable can be easily used even in departments with limited experience with MRI. Understanding that NPV and accuracy of the tool might be comparable with outcomes of sole staging mpMRI achieved in highly experienced centers [13, 22, 23], we believe utility of the tool might be mostly visible when gaining experience with resonance imaging of the prostate. Based on initial studies, MRI was considered to have no incremental value over standard staging approach in low-risk patients and was not indicated in that setting. However, recent metaanalysis revealed that, in spite of its low sensitivity in detecting EPE in the low-risk PCa, it provides valuable information about feasibility of nerve-sparing surgery [16]. Moreover, current EAU guidelines recommend to use prebiopsy MRI for staging purposes.

To provide genuine validation and define best working environment for the tool, an external cohort of the department with expertise in prostate cancer treatment was used. Both cohorts were similar in terms of endpoint prevalence and relevant predictors widely considered to be related to ECE. To sustain certainty that proposed categorical variables can predict ECE independently, logistic regression was performed in validation cohort. We confirmed that total percentage of cancerous tissue in biopsy >15%, MTD >15 mm and PSA >20 ng/mL are ECE predictors in a model validated in Gleason score-stratified external cohort. Given that definitions of predictors were exactly the same and disease advancement was similar, we manage to obtain a fairly good performance at external validation. The discriminative ability dropped at development with c statistic from around 0.86 to 0.64, although Somers’ Dxy rank correlation indicated, that the model still improved the prediction of ECE in the validation cohort by nearly 30% (Table 3). Worse outcomes at validation may be explained by relatively small size of the development cohort, or that patients were originally selected from a single center. Presumably, validation in several external sites would create an opportunity to update this simple formula and enable its border application.

The final model was calculated from a combined dataset (development and validation) and revealed good performance (Table 3), similarly to the native model. The calculation of final model derived from a full sample led to more stable estimation of the effect of the predictors and prevented waste of relevant information. This combination of data assumes that the two samples represent a similar population, which is indeed the case (Table 1). Although model managed to safely rule out ECE in entire cohort (NPV 84%), it tended to overestimate the risk, which resulted in high rate of false positives and unsatisfactory specificity as well as PPV. Due to substantial deterioration of sensitivity especially in low risk patients area under ROC curve has not exceeded 0.7 in entire cohort. It is however worth noting, that in patients with high-risk Gleason grade in biopsy (>7), better discrimination resulted not only in further improvement of sensitivity and NPV, but also substantial reduction of false positives (specificity and PPV reaching 60%), which elevated AUC to 0.8. This highlights the impact of different ECE prevalence among distinct PCa-risk groups on diagnostic accuracy. Since positive surgical margins are of major concern in high-risk patients [24] and do not always require change of postoperative track in low risk patients [12] we interpret this results as primary validation outcome that supports its clinical use.

The validation results suggest that the algorithm lacks specificity and might additionally underestimate risk in Gleason score ≤7. Thus it cannot be recommended for routine use as a sole preoperative tool assessing ECE risk in every patient. In low- or intermediate-risk patients the algorithm should be used for initial assessment and supplemented with one of the validated nomograms [7, 20, 21], especially in case of men highly motivated to nerve-sparing and unclear contraindications to this approach. In patients with high-risk organ-confined prostate cancer the algorithm can be strongly recommended at least as initial screening and adjunct to nomograms, since its predictive yield seems clearly proven in this group (Figure 1).

Figure 1

The use of an algorithm in making decisions about the use of the nerve-sparing technique. The negative and positive predictive values calculated for the validation group were used to determine the extracapsular extension risk. 159 of 308 prostate lobes fulfilled criteria for ECE (-) in the entire validation group. For patients with Gleason score >7 in biopsy the values are 89% and 53% respectively.

nsRP – nerve-sparing radical prostatectomy; MTD – maximum tumor diameter; mpMRI – multiparametric magentic resonance of prostate; PSA – prostate specific antigen; bx – biopsy; ECE – extracapsular extension; CI – confidence interval

The study has several limitations that should be signalized. Although cohorts have not differed significantly considering preoperative assessment as well as risk groups composition, there were some less substantial differences that can impact the validation outcomes. In validation cohort magnetic resonance imaging was performed on 3-T system not 1.5-T like in development cohort. Moreover, validating MRI was performed not only for staging purposes, but also for targeting biopsy in some individuals. That might influence percentage of cancerous tissue in biopsy cores and require additional model calibration with possible change of cut-off. However, clinical use of mpMRI is now wider than in development period and majority of departments use it now also in a prebiopsy setting (25), we believe that this inconsistency might in fact allow validation cohort to reproduce current clinical environment even better. Consequently, validation cohort was collected after ISUP 2014 whereas model was primary developed with ISUP 2005 grading. Surprisingly, validation and development regression models have not found Gleason grade significant anyway. Finally, although ECE prevalence and Gleason composition is similar to those observed in similar series [7, 19, 20, 21], retrospective design might be an obvious source of selection bias. Therefore, authors encourage do perform fully independent validation by other investigators at other centers.

CONCLUSIONS

This external validation confirms the good performance of our model using PSA, biopsy and MRI parameters to predict ipsilateral ECE. Its simplicity and user friendly format provides an easy tool in initial screening of men undergoing radical prostatectomy with the intention of preserving NVB.

CONFLICTS OF INTEREST

The authors declare no conflicts of interest.