Effect of training and individual operator's expertise on prostate cancer detection through prostate biopsy: Implications for the current quantitative training evaluation system.

PURPOSE: This study was conducted to evaluate the relevance of training and experience to gaining expertise in prostate biopsy based on an assessment of outcomes from the performance of urology residents.
MATERIALS AND METHODS: We retrospectively reviewed the medical records of 10,299 patients who underwent prostate biopsy by 50 operators under a unified urology residency program. The number of prostate biopsies performed by an operator for each patient was used as an indicator of operator experience. Residents were grouped into quartiles according to cancer detection rates in the first 50 and the last 50 procedures.
RESULTS: Among 10,299 patients (median age, 67.5 years; median prostate-specific antigen [PSA], 7.04 ng/mL), the overall prostate cancer detection rate and that for patients with PSA <10.0 ng/mL were 37.0% and 25.9%, respectively. Operator experience was a significant predictor for cancer detection in patients with PSA <10.0 ng/mL. Cancer detection rates and the proportion of more advanced prostate cancers were higher in the last 50 cases than in the first 50 cases. Detection rates varied significantly among operator; residents with higher detection rates at training initiation showed even higher detection rates after additional training.
CONCLUSIONS: Training that adds to the cumulative experience of a trainee appears to play a meaningful role in improving cancer detection rates. The level of skill required to achieve mastery for independent practice may be assessed from the accuracy results of prostate biopsy procedures, and trainees with poor rates will require more technical training to improve precision. © The Korean Urological Association, 2021.

INTRODUCTION

Modern medical training requires experienced practitioners to transfer complex knowledge and skills to learners [1]. Repetitive practicing of technical skills is a prerequisite to achieving an expertise level that is sufficient for independent practice. Thus, technical training with actual hands-on experience and performing a large volume of procedures, which contributes to increased knowledge about the procedure, as well as patient safety, are essential for adequate training of residents [1]. In several countries, urology residents are required to manage a certain number of clinical cases by utilizing a variety of standard procedures in order to successfully complete their training programs [2].

Numerous studies of prostate biopsy have focused on cancer detection rates and associated clinical variables [34567]. Several studies have evaluated the association of operator experience with the outcome of a prostate biopsy [189]. However, the characteristic features of the learning curve for prostate biopsy remains debatable. Training quality, individual differences in experience, and an operator's natural skill can all play key roles in training during a urology residency. However, to the best of our knowledge, no study has investigated the association of these factors with the outcome of a prostate biopsy.

Therefore, in this study, we analyzed the cancer detection rate of prostate biopsies performed by urology residents over a 10-year period to identify the effects of experience on cancer detection rates. Additionally, we compared clinical outcomes to identify factors that could improve residency training. We investigated the extent to which the associations among experience, training quality, and individual differences in proficiency could make a difference to prostate biopsy outcomes.

MATERIALS AND METHODS

1. Patient selections and data collection

This multicenter retrospective study was approved by Institutional Review Board of the Yonsei University Health System-Gangnam Severance Hospital (approval number: 3-2020-0179). The requirement for informed consent was waived as this study was based on retrospective, anonymous patient data and did not involve patient intervention or the use of human tissue samples. We reviewed the results of prostate biopsy for 10,632 patients who underwent a biopsy procedure performed by 50 urology residents who began their residency between 2006 and 2015 at Sinchon Severance Hospital (n=6,301), Gangnam Severance Hospital (n=2,483), and National Health Insurance Service Ilsan Hospital (n=1,848). To determine the operator's experience, we used the number of prostate biopsies performed by the operator based on data from patient medical records. As a transrectal ultrasound system, Sinchon and Gangnam Severance Hospital used ProFocus Ultraview (BK Medical, Peabody, MA, USA), and the other hospital used Accuvix XQ (Samsung Medison, Hongcheon, Korea).

Patients with incomplete data (n=333) were excluded. The remaining 10,299 patient biopsy results were included in the final analysis dataset. Patients taking 5-alpha-reductase inhibitors (5-ARIs) or who underwent transurethral operation were also included in the data analysis without any distinction. Patients who underwent pre-biopsy magnetic resonance imaging were not included in the patient group, because target or cognitive biopsies are performed by a specialist, not residents.

The patient characteristics analyzed in this study included age, serum prostate-specific antigen (PSA) level, prostate volume, history of prostate biopsy, and biopsy-based Gleason scores. In patients taking 5-ARIs, PSA was adjusted by 2 times and prostate volume by 1.4 times in accordance with studies that have reported a 50% decrease in PSA and 30% decrease in prostate volume when taking 5-ARIs for 6 months or longer [1011]. Patients were categorized into three groups based on their PSA levels (<10.0, 10.0–20.0, and ≥20.0 ng/mL). Clinically significant prostate cancer was defined as a Gleason score ≥7 (3+4).

2. Resident program

The urology residency program at the participating centers is integrated into a large academic medical center. All residents at the three institutions (Sinchon Severance Hospital, Gangnam Severance Hospital, and National Health Insurance Service Ilsan Hospital) worked in rotational postings during the unified training programs. The residency program graduates one to six residents annually.

3. Teaching program for prostate biopsy

The institutional training program policy recommends that 12 core prostate biopsy procedures are to be performed by residents. In 2011, a program of enhanced training for prostate biopsy was initiated that requires all residents to perform transrectal ultrasounds in the first 6 months, under the mentorship of senior residents or a professor, before they can independently perform a prostate biopsy. After qualifying in the technical proficiency assessment for the transrectal ultrasound, the resident performs five to 10 prostate biopsies under mentorship of a senior urologist or a professor.

4. Statistical analysis

All values are expressed as a number (%) or mean±standard deviation. Continuous variables are expressed as median (interquartile range). Intergroup differences in continuously distributed variables were assessed using the Mann–Whitney U-test. Multivariate regression analyses were conducted to identify significant predictors of cancer diagnosis, and these included variables that had a p-value of <0.05 in univariate analyses. Receiver operating characteristic (ROC) curves and area under the ROC curves (AUCs) were used to obtain cut-off values. These optimal cut-off values were based on predefined values and were determined according to a sensitivity analysis using the Youden Index (sensitivity+specificity-1). AUC values were compared using the DeLong method for statistically significant differences in AUCs. All residents were grouped into quartiles based on cancer detection rates for patients with PSA <10.0 ng/mL in the first 50 procedures and the last 50 procedures. Individual differences in expertise were analyzed using the chi-square test for trend. All reported p-values are two-sided, and statistical significance was set at p<0.05. Statistical analyses were performed using the Statistical Package for Social Sciences (SPSS), version 25.0, for Windows (IBM Corp., Armonk, NY, USA) and R, version 3.1.3 (R Foundation, Vienna, Austria).

RESULTS

1. Demographic data of the study participants

The baseline characteristics of the 10,299 patients (median age, 67.5 years; median PSA, 7.04 ng/mL) are shown in Table 1. Overall, each resident performed a median of 195.5 procedures during their 4-year training period. A total of 3,810 patients (37.0%) were diagnosed with prostate cancer. The cancer detection rates of the groups with PSA <10.0 ng/mL (n=7,000, 68.0%), 10.0–20.0 ng/mL (n=1,708, 16.6%), and ≥20.0 ng/mL (n=1,591, 15.4%) were 25.9%, 42.5%, and 80.1%, respectively.

Table 1

Characteristics of prostate biopsy patients

Variable		Total	PSA <10.0 ng/mL	PSA 10.0–20.0 ng/mL	PSA ≥20.0 ng/mL	p-value
No. of patients		10,299 (100.0)	7,000 (68.0)	1,708 (16.6)	1,591 (15.4)
Age (y)		67.5 (61.0–73.1)	66.1 (59.8–71.8)	69.4 (63.1–74.4)	71.4 (65.9–76.8)	<0.001
PSA (ng/mL)		7.04 (5.04–12.33)	5.64 (4.50–7.17)	12.95 (11.15–15.60)	48.91 (27.70–129.32)	<0.001
Prostate volume (cm3)		39.8 (30.0–53.7)	39.3 (29.7–52.5)	41.5 (31.1–56.9)	40.0 (30.1–54.4)	<0.001
PSA density (ng/mL/cm3)		0.18 (0.12–0.33)	0.14 (0.10–0.20)	0.32 (0.22–0.45)	1.19 (0.70–2.84)	<0.001
Previous prostate biopsy history		877 (8.5)	586 (8.4)	197 (11.5)	94 (5.9)	<0.001
Diagnosis of prostate cancer		3,810 (37.0)	1,810 (25.9)	726 (42.5)	1,274 (80.1)	<0.001
Gleason score						<0.001
	≤6	1,290 (33.9)	938 (51.8)	246 (33.9)	106 (8.3)
	≥7	2,520 (66.1)	872 (48.2)	480 (66.1)	1,168 (91.7)

Values are presented as number (%) or median (interquartile range).

PSA, prostate-specific antigen.

2. Prediction of cancer diagnosis

The results of the multivariate analysis for cancer detection indicated that age (odds ratio [OR], 1.05; 95% confidence interval [CI], 1.048–1.060; p<0.001), PSA (OR, 1.05; 95% CI, 1.049–1.059; p<0.001), prostate volume (OR, 0.99; 95% CI, 0.985–0.990; p<0.001), history of prostate biopsy (OR, 0.74; 95% CI, 0.618–0.876; p=0.001), and experience with >150 cases (OR, 1.36; 95% CI, 1.231–1.504; p<0.001) were significant predictors (Table 2). When the AUC values of experience with >150 cases in addition to the conventional variables (age, PSA, prostate volume, and history of prostate biopsy) and that of conventional variables were compared, no significant differences were found (OR, 0.756; 95% CI, 0.746–0.767 vs. OR, 0.754; 95% CI, 0.743–0.764; p=0.136).

Table 2

Results of multivariable analyses of predictive indicators of a diagnosis of prostate cancer

Total	Total		PSA <10.0 ng/mL		PSA 10.0–20.0 ng/mL		PSA ≥20.0 ng/mL
	OR (95% CI)	p-value	OR (95% CI)	p-value	OR (95% CI)	p-value	OR (95% CI)	p-value
Age (y)	1.05 (1.048–1.060)	<0.001	1.06 (1.049–1.064)	<0.001	1.04 (1.028–1.055)	<0.001	1.03 (1.016–1.053)	<0.001
PSA (ng/mL)	1.05 (1.049–1.059)	<0.001	1.14 (1.103–1.175)	<0.001	1.11 (1.076–1.163)	<0.001	1.02 (1.017–1.029)	<0.001
Prostate volume (cm3)	0.99 (0.985–0.990)	<0.001	0.99 (0.986–0.992)	<0.001	0.99 (0.983–0.992)	<0.001	0.98 (0.976–0.987)	<0.001
History of prostate biopsy	0.74 (0.618–0.876)	0.001	-	-	0.46 (0.322–0.662)	<0.001	0.27 (0.160–0.458)	<0.001
Experience >150 case	1.36 (1.231–1.504)	<0.001	1.34 (1.185–1.510)	<0.001	1.50 (1.196–1.891)	<0.001	1.45 (1.047–1.996)	0.025

OR, odds ratio; CI, confidence interval; PSA, prostate-specific antigen; -, not available.

In the subgroups stratified according to PSA levels, experience with >150 prostate biopsy cases was a significant predictor for cancer detection in multivariate analysis (Table 2). Notably, in the patient group with PSA <10.0 ng/mL, a significant difference in AUC values was found between operators with experience in >150 cases in addition to conventional variables and that of conventional variables (OR, 0.662; 95% CI, 0.646–0.677 vs. OR, 0.656; 95% CI, 0.640–0.672; p=0.001) (Fig. 1). However, no significant differences in the AUCs were found in the comparisons of groups with PSA levels of 10.0–20.0 ng/mL or ≥20.0 ng/mL.

Fig. 1

Comparison between AUC of experience in addition to conventional variables (age, PSA, prostate volume) and that of conventional variables for predicting prostate cancer in patients with PSA <10.0 ng/mL. AUC, area under the receiver operating characteristic curve; PSA, prostate-specific antigen; OR, odds ratio; CI, confidence interval.

3. Comparison of cancer detection rates between the first 50 and the last 50 cases

Baseline characteristics of the first 50 cases and the last 50 cases for the 50 residents are shown in Table 3. The last 50 cases had significantly lower age, lower PSA levels, and lower PSA density than the first 50 cases. However, the cancer detection rates and the proportion of clinically significant prostate cancer were higher in the last 50 cases than in the first 50 cases (p=0.025 and p=0.025, respectively).

Table 3

Comparison of patient characteristics between the first 50 cases and the last 50 cases of the 50 urology residents

	Characteristic	First 50 cases	Last 50 cases	p-value
No. of patients		2,356	2,310
Age (y)		67.5 (60.9–72.9)	66.7 (60.0–72.3)	0.012
PSA (ng/mL)		7.26 (5.11–12.60)	6.85 (4.96–11.66)	0.003
Prostate volume (cm3)		39.2 (29.9–54.0)	39.7 (30.0–52.6)	0.959
PSA density (ng/mL/cm3)		0.19 (0.12–0.34)	0.17 (0.11–0.31)	0.007
History of prostate biopsy		179 (7.6)	180 (7.8)	0.803
Diagnosis of prostate cancer		814 (34.6)	823 (35.6)	0.025
Gleason Score				0.025
	≤6	297 (36.5)	257 (31.2)
	≥7	517 (63.5)	566 (68.8)

Values are presented as number only, median (interquartile range), or number (%).

PSA, prostate-specific antigen.

4. Association of supplementary training with expertise and with cancer detection rates

This study compared outcomes in patients with PSA <10.0 ng/mL to determine the effect of skills-strengthening supplementary training. With the adjunctive benefit of additional training, 28 residents showed a significantly higher cancer detection rate in the first 50 cases and the last 50 cases than the 22 residents who graduated before the supplementary training program was initiated (median 24.2% vs. 17.5%, p=0.015; median 25.0% vs. 19.5%, p=0.015, respectively) (Table 4). Notably, cancer detection rates varied significantly among operators. Median cancer detection rates in the first 50 cases and the last 50 cases among the 50 residents were 23.2% (range, 4.5%–55.2%) and 23.5% (range, 9.1%–53.3%), respectively. Thirteen (26.5%) residents had a detection rate >30.0%.

Table 4

Comparison of the prostate cancer detection rate in patients with PSA <10.0 ng/mL as an effect of supplementary training

Variable	2006–2010	2011–2015	p-value
No. of residents	22	28
Diagnosis of prostate cancer in the first 50 cases (%)	17.5 (14.2–25.7)	24.2 (20.5–31.3)	0.015
Diagnosis of prostate cancer in the last 50 cases (%)	19.5 (15.3–25.2)	25.0 (21.4–37.0)	0.015

Values are presented as number only or median (interquartile range).

PSA, prostate-specific antigen.

5. Association of individual differences in expertise and cancer detection rates

Table 5 shows a linear association between the first 50 cases and the last 50 cases. Expertise identified in the early phase of residency training tended to remain consistent in the late phase (p<0.001). In sub-analysis of the effect of supplementary training, residents with higher detection prostate cancer rates at the beginning of their program showed higher outcomes after supplementary training, as expected (p=0.006).

Table 5

Evaluation of the first and last 50 cases showed a linear association based on the quartile of cancer detection rates for patients with PSA <10.0 ng/mL

First 50 cases	Last 50 cases
	Q1	Q2	Q3	Q4	Total
Q1	6	4	2	0	12
Q2	4	3	2	4	13
Q3	2	4	5	2	13
Q4	0	2	4	6	12
Total	12	13	13	12	50

PSA, prostate-specific antigen; Q, quartile.

DISCUSSION

Understanding the learning curves for clinical procedures is important to optimize treatment outcomes [1213141516]. Several studies have investigated the learning curve for prostate biopsy performed by trainees [117]. However, optimal cut-off values for the number of biopsies that needs to be performed to improve cancer detection rates have remained unclear. In a study of 770 patients who underwent prostate biopsy by 24 residents, no differences were found in the cancer detection rates between the first and the sixth month of training [1]. Karam et al. [9] reported that there was no learning curve based on an analysis of 170 patients who underwent biopsy by an unreported number of residents. Hori et al. [17] suggest that 50 cases for a non-physician may result in as effective a performance of prostate biopsy as would be exhibited by an experienced urologist. However, we found that experience in performing prostate biopsy was a significant factor for accurate cancer detection in 10,299 prostate biopsies performed by 50 residents (median, 195.5 cases per trainee). Additionally, an optimal cut-off value of 150 cases was required to improve proficiency in cancer detection for patients with PSA <10.0 ng/mL. This result suggests that a new trainee should perform prostate biopsy for patients with PSA levels ≥10.0 ng/mL, similar to the recommendation based on a previous study [17]. These differences in results among the cited studies could be due to differences in the number of cases per trainee in each study.

Generally, age, PSA, and prostate volume are considered risk factors for prostate cancer [18]. Meanwhile, operator skill proficiency, training quality, and individual differences in expertise may also be considered as external contributory factors to cancer detection rates. This study indicated that more experienced operators show a higher cancer detection rate, as well as a higher proportion of clinically significant prostate cancers (Table 3). However, identifying specific factors that can improve outcomes with increasing experience will likely be difficult. Indeed, we assume that more experienced operators have independently developed professional knowledge and skill sets that enable them to identify suspicious prostate cancer lesions by identifying hypoechogenicity, irregularity, microcalcification, and vascularity, factors that may improve their cancer detection rate [19].

The cancer detection rate of men with a PSA level 4 to 10 ng/mL is approximately 30% to 35% [20]. In the group of patients with a PSA <10 ng/mL (median PSA 5.64 ng/mL) in this study, the detection rate of 25.9% was lower than that reported in previous studies. In the first 5 years (2006–2010) of the study, the detection rate in the first 50 and the last 50 cases for patients with PSA <10.0 ng/mL were 17.5% and 19.5%, respectively. Based on these results, our institutions decided to strengthen the training for prostate biopsy. As expected, the cancer detection rate significantly increased (the first 50 cases yielded 24.2% and the last 50 cases yielded 25.0% positive results; Table 4). Therefore, we concluded that improving training quality and skill proficiency can increase average cancer detection rates in prostate biopsy.

Of interest, this study found that 26.5% of operators detected prostate cancer in >30% of patients with PSA <10.0 ng/mL for the first 50 cases. The detection rates varied significantly among operators (first 50 cases, range 4.5%–55.2%; last 50 cases, range 9.1%–53.3%), similar to results from a previous study. Lawrentschuk et al. [8] reported that unknown differences in expertise or technique were present among individual operators and that there was significant variation in their cancer detection rates (range 43.8%–52.4%). Therefore, we hypothesize that unexplainable factors other than learning curve and average skill proficiency of prostate biopsy performance may have an effect on cancer detection rates. While operators with better results in the first 50 cases showed consistently better outcomes in the last 50 cases (Table 5), residents with higher detection rates at the beginning of residency subsequently showed higher outcomes only after completing their supplementary training.

This study holds implications for both trainees and trainers: 1) Operator experience is a significant factor that affects cancer detection in patients with PSA <10.0 ng/mL. Therefore, trainees should receive training practice on patients with PSA ≥10.0 ng/mL, since diagnostic outcomes are similar for experienced operators. 2) Training quality for trainees has an important role in improving prostate cancer detection; without high-quality intensive training, trainees may not be able to develop sufficient individual expertise. 3) Individual capabilities were evident in the early phase of resident training. Therefore, trainees with poor outcomes in early resident training need more focused technical training. Urology residency training involving prostate biopsy should be organized with a qualitative evaluation system factoring in individual capabilities, instead of conventional quantitative evaluation systems based on current standards that define a good training program.

This study has some limitations. First, we did not specifically show how to modify supplementary training for proficiency in prostate biopsy. We assumed that learning transrectal ultrasound for 6 months after beginning resident training and before participating in prostate biopsy, combined with technical training and additional hands-on experience, possibly improved the cancer detection rates. Second, the characteristics of individual differences in prostate biopsy were uncategorized. We suspect that there may have been the possibility of subtle differences in medial deviation of the biopsy needle and under-sampling at the prostatic apex. Third, the learning curve may vary with individuals. An optimal cut-off value that indicates improvement in cancer detection might be less than 150 cases in a trainee who demonstrates high cancer detection acuity early in residency training. Finally, this paper was unable to separately classify and analyze patients who underwent a transurethral operation or were taking medications, including 5-ARIs, that could affect the prostate. Excluding those patients would help to derive more accurate results, but since this paper focused on resident education, there was a limit to excluding them. If these factors were excluded, we deemed that there would have been a special effect on the analysis of the resident's experience scale. If follow-up research proceeds, it would be better to organize patient groups with this in mind.

CONCLUSIONS

Resident urology trainees should gain clinical experience with patients of PSA levels ≥10.0 ng/mL. We recommend that at least 50 cases of biopsies for patients with PSA level <10 ng/mL should be performed under the mentorship of a trained urologist. To achieve the level of mastery required for independent practice, trainees with poor prostate biopsy performance early in their programs require more support and focused technical training.