Evaluation of the urology residency training program in Saudi Arabia: A cross-sectional study.

PURPOSE: This study evaluates the satisfaction of urology residents with the Saudi Board of Urology (SBU) Training Program and identifies areas of weakness and strength to improve the educational environment, surgical competency, and overall satisfaction of urology residents with the program.
METHODS: We administered an electronic self-made questionnaire that included two sections. One comprised demographic data (age, gender, weight, height, marital status, level of training, city of training, and center of training), while the other concerned SBU evaluation (satisfaction with different aspects of training, such as ways of assessment, mentors' feedback, surgical competency, research, and strengths and weaknesses of SBU).
RESULTS: The overall satisfaction of urology residency program was 28.8% while 44.2% of residents had a neutral response. The highest level of satisfaction with clinical and surgical practice was among graduates (56.9%) and Riyadh residents (45.1%). Furthermore, good work/life balance received the lowest level of satisfaction (5.2%) among senior residents, while good clinical experience received the highest level (62.7%) among the graduates. Residents reported a high exposure in endourology and pediatric urology, while transplant, reconstructive, and neurourology had the lowest exposure. Forty-two percent of respondents undertook research during their residency training, but most respondents (54%) did not publish any research papers during their training. Sixty-two percent of graduates felt that their training program did not prepare them adequately to perform well on the board examinations.
CONCLUSION: Our results confirmed that satisfaction of residents with the urology program process is variable according to the city of training. Having high satisfaction level in some cities reflects the improvement of urology training program after restructuring. We identified new areas in need of improvement, namely lack of mentorship, clear and formal assessment process, and variation of training process between central and peripheral programs. Copyright:

INTRODUCTION

The Saudi Board of Urology (SBU) Training Program aims to train and produce proficient and skilled urologists who are self-sufficient. The educational environment is an essential element for learning, accomplishment, and overall satisfaction during the residency training programs. Furthermore, a trainee improving performance is directly correlated with a positive learning environment.[12] Nevertheless, residency training is a complex process that is influenced by multiple factors, ranging from program structure, accessibility to learning resources, hands-on surgical training, support from peers and mentors, methods of evaluation, and even the role of chief resident, which can impact the SBU program's outcome and the overall educational environment.[34]

Supervised by the Saudi Commission for Health Specialties (SCFHS), the SBU Training Program is a 5-year program established in 1994 that involves approximately 35 centers distributed in 18 cities inside and outside the Saudi Arabia. The 1st year of training includes a 9-month rotation in general surgery where residents grasp the basics of surgery, and a 3-month rotation in the intensive care unit that gives the trainee the minimum required competency to deal with critically ill surgical patients. The urology resident then goes into a 3-month rotation over all the subspecialties in urology (uro-oncology, endourology, andrology and infertility, reconstruction, minimally invasive, neurourology, female urology, and pediatric urology) along with a 3-month rotation in selective specialties (plastic surgery, pathology, emergency medicine, vascular surgery, and urogynecology).[5]

In 2015, Binsaleh et al. conducted the first evaluation of SBU using the postgraduate hospital educational environment measure; however, the respondents’ rate was poor (53%), which warranted further evaluation of the program.[6] Therefore, in 2015, Binsaleh et al. conducted yet another cross-sectional study evaluating urology residents’ perception toward the surgical theater educational environment, but with another poor response rate of 45.8%, the need for a well-established understanding of the educational environment was essential.[7] In 2016, Al Otaibi[8] showed that out of 72 hospitals in Saudi Arabia, renal transplants were being performed by urologists in six centers, while two other centers were using surgeons to perform the transplants. The urology residency training programs in Saudi Arabia adopted the CanMEds Competency Framework in 2014.[9] This restructuring aimed to shift the educational focus from the process to the outcomes, standardize and assure quality of training across the nation, and make residency programs “trainee-centered” rather than “trainer-centered.”[9] Such paradigm shift was expected to improve the overall satisfaction rate among the residents.

Therefore, we aim to evaluate the satisfaction of urology residents with SBU and identify areas of weakness and strength in order to improve the educational environment, surgical competency, and overall satisfaction of urology residents.

METHODS

This cross-sectional survey study was conducted between June and August 2019. All urology residents received an online version of the survey by E-mail using the platform www.surveymonkey.com (SurveyMonkey, Portland, OR, USA). Further, SBU graduates between 2014 and 2018 were asked to participate in the survey as well. Informed consent was obtained from all participants. A total of 51 questions were included in the questionnaires, which were E-mailed to all residents registered in the urology training program; the identities of the participants whose data were collected were kept anonymous. Open questions were included in the questionnaire. The study protocol was approved (approval number 2009-0090E) by the Central Institutional Review Board of the Ministry of Health in June 2019. The electronic questionnaire included two sections. The first section concerned demographic data, including age, gender, weight, height, marital status, level of training, city of training, and center of training; the second section concerned the evaluation of SBU itself, including satisfaction with different aspects of training, ways of assessment, mentors’ feedback, surgical competency, research, strengths, and weaknesses of SBU. Satisfaction level was measured by a five-point Likert scale. Descriptive statistics were reported as median, mean, and standard deviation. Data were collected in a Spreadsheet and subsequently analyzed using SPSS (version 17 Inc., Chicago, IL, USA). P ≤ 0.05 was considered statistically significant.

RESULTS

The questionnaire was distributed among 247 registered residents, of whom 215 (87.04%) completed the questionnaire. The mean age of participants was 30.27 years, 95.3% were males, and 42.3% were trained in Riyadh. Junior residents comprised 43.7% of respondents, 27% were senior residents, and 29.3% were graduates. Table 1 shows the demographic data of the participants.

Table 1

Demographic details of study participants

Characteristics of study respondents	n (%)
Sex
Male	205 (95.3)
Female	10 (4.7)
Body mass index
Normal	69 (32.09)
Overweight	84 (39.06)
Obese	61 (28.37)
Not available	1 (0.48)
Marital status
Single	71 (33)
Married	138 (64.2)
Divorced	6 (2.8)
Raising children
Yes	105 (48.8)
No	110 (51.2)
Region of training
Central region (Riyadh)	91 (42.3)
Eastern region	32 (14.9)
Makkah	5 (2.3)
Jeddah	27 (12.6)
Al Madinah	8 (3.7)
Taif	11 (5.1)
Asir/Southern region	27 (12.6)
Other (please specify)	14 (6.5)
Training level
Junior residents (PGY-1, PGY-2, PGY-3)	94 (43.7)
Senior residents (PGY-4, PGY-5)	58 (27)
Graduated	63 (29.3)

The satisfaction level with the residency training is shown in Table 2. Overall, 28.8% of respondents were satisfied with their training program, 44.2% were neutral, and 27% were dissatisfied. The administrative domains that yielded a high satisfaction rate were program director support (41.4%) and role of chief residents (34%). On the other hand, administrative domains with low satisfaction rate were evaluation system (25.1%) and distribution of rotations (29.8%). For the program's ability to prepare residents for clinical practices, the level of satisfaction was 42.80%, while in the case of preparing for surgical practice, the level was 29.4%. It appears that the highest overall level of satisfaction with the urology residency program was among graduates (31.4%) compared to senior and junior residents, and among respondents from Riyadh (45.1%) compared to other regions [Figure 1]. Regarding assessment, 33% of the respondents agreed that they had clear ways of assessments during the residency training program, while 67% disagreed.

Table 2

Satisfaction levels about urology residency training program in multiple domains

Item	Satisfied (%)	Neutral (%)	Dissatisfied (%)
What is your overall level of satisfaction with your urology residency program?	28.8	44.2	27
What is your level of satisfaction with your Urology residency program administration in the following?
Implemented teaching curriculum	32.1	47.9	20
Distribution of rotations	29.8	30.2	40
Evaluation system	25.1	42.3	32.6
Program director support	41.4	25.1	33.5
Surgical exposure	31.2	31.6	37.2
Role of chief residents	34	30	35
What is your level of satisfaction with your urology training program in preparing you for practice in the following areas?
Clinical practice	42	35.8	21.4
Surgical practice	29.4	38.8	31.8
Overall, what do you feel are the strengths of your residency program?
Faculty committed to teaching	25.6	42.3	32.1
Good clinic experience	44.2	42.8	13
Good surgical skills experience	30.7	43.7	25.6
Support for resident well being	18.2	34.6	47.2
Access to modern technology	30.2	30.2	39.5
High faculty to resident ratio	14.6	52.1	33.3
Abundant research opportunities	14.9	29.3	55.8
Good work/life balance	15.3	38.6	46
Assessment methods	Yes (%)		No (%)
Did you have clear ways of assessments during the residency training program	33		67

Figure 1

Overall level of satisfaction with urology residency program according to the city of training

Participants’ opinion regarding their clinical and surgical exposure was variable in different urological subspecialties [Figure 2]. Endourology, pediatric urology, and uro-oncology had the highest exposure (86.5%, 65.5%, and 53.1%, respectively), while transplant, reconstructive, and neurourology had the lowest exposure (19.5%, 27.9%, and 31.6%, respectively). Participants in the western region of Saudi Arabia (Makkah, Taif, and Al-Madinah) had the lowest level of exposure to most of the subspecialties when compared to participants in other regions.

Figure 2

Reported adequacy of clinical and surgical exposure among residents and graduates in different subspecialties

Table 3 presents the volume of exposure in common urological procedures reported by all respondents. Most senior residents and graduates have performed more than 10 cases in their residency training, with exception to microscopic varicocelectomy. Respondents in the certain regions (i.e., Makkah and Al-Madinah) reported <10 cases performed for all the procedures with exception to flexible ureteroscopy.

Table 3

Performed cases by the end of your training (till now for residents) as a primary surgeon

Procedure	Volume	Training level (%)			All respondents (%)
		Junior residents (PGY-1, PGY-2, PGY-3)	Senior residents (PGY-4, PGY-5)	Graduate
Flexible URS	None	44.7	1.7	3.2	20.9
	1-5	27.7	10.3	7.9	17.2
	6-10	6.4	17.2	11.1	10.7
	11-20	10.6	20.7	15.9	14.9
	More than 20	10.6	50.0	61.9	36.3
Microscopic varicocelectomy	None	57.4	29.3	30.2	41.9
	1-5	24.5	31.0	17.5	24.2
	6-10	6.4	6.9	6.3	6.5
	11-20	3.2	12.1	14.3	8.8
	More than 20	8.5	20.7	31.7	18.6
Hydrocelectomy	None	25.5	0.0	1.6	11.6
	1-5	55.3	24.1	4.8	32.1
	6-10	11.7	36.2	19.0	20.5
	11-20	6.4	25.9	23.8	16.7
	More than 20	1.1	13.8	50.8	19.1
TURP	None	59.6	1.7	1.6	27.0
	1-5	26.6	22.4	7.9	20.0
	6-10	11.7	29.3	19.0	18.6
	11-20	2.1	25.9	30.2	16.7
	More than 20	0.0	20.7	41.3	17.7
Orchidopexy	None	38.3	10.3	3.2	20.5
	1-5	38.3	27.6	22.2	30.7
	6-10	10.6	20.7	19.0	15.8
	11-20	6.4	19.0	22.2	14.4
	More than 20	6.4	22.4	33.3	18.6

TURP: Transurethral resection of the prostate, URS: Ureterorenoscopy

Regarding conducting research, most respondents (42.1%) undertook research during their residency training as an optional project. However, most respondents (54%) did not publish any research papers, 27.9% published one paper, 7.9% published two papers, and 10.2% published three or more papers during their residency training. Research activity during residency training among respondents is presented in Table 4.

Table 4

Research activity during residency training among respondents

Question	n (%)
Did you undertake research during your residency training?
Yes (mandatory)	45 (20.9)
Yes (optional)	90 (41.9)
No	80 (37.2)
How many papers did you publish during your residency training?
None	116 (54)
1	60 (27.9)
2	17 (7.9)
≥3	22 (10.2)
Which of the following facilitations were available for you?
Mentor support	80 (37.2)
Biostatical support	45 (20.9)
Secretary support	32 (14.9)
Research department	38 (17.7)

One part of the survey was directed toward graduates regarding their preparation for the final board examination [Table 5]. Among them, 62% felt that their training program did not prepare them adequately to perform well on the board examinations. Fifty-seven percent of graduates thought that continuing clinical and on-call duties while studying have negatively affected their performance on the board examinations.

Table 5

Preparation for the final board examination among graduates (n=63)

Question	n (%)
My residency training program had a protected study time prior to the board examinations
Agree	20 (31.7)
Neutral	17 (27)
Disagree	26 (41.3)
Did you feel your training program prepared you adequately to perform well on the board examinations?
Yes	24 (38)
No	39 (62)
What factors negatively affected your performance on the board examinations?
Continuing clinic duties while studying	36 (57)
Continuing call duties while studying	36 (57)
Inadequate clinical exposure during residency	19 (30)
Inadequate teaching during residency	30 (47.6)

DISCUSSION

The SCFHS and the SBU had undergone significant restructuring recently to standardize and assure the quality of training across the nation. Moreover, the number of accepted residents for training annually and the training centers were increased to keep up with the growing number of medical schools and their graduates. Performing our study at this specific time was important to assure that the quality of residency training was not affected by the enormous growth of number of residents.

Most participants comprised young, married, childless, male, junior residents in Riyadh city [Table 1], which shows that the urology training programs remain unattractive to female medical students, an overcrowded junior year clustered in one city, and shows the impact of this training program on family and parenthood status. This is in agreement with multiple studies that have shown similar age and sex distribution, along with marital and parenthood status, both nationally and internationally, with no noticeable change in demographics 5 years later.[1011] These demographic characteristics can be both a result and a cause of dissatisfaction among the trainees.[121314]

Regarding the overall satisfaction about the urology training program, 28.8% were satisfied while 44.2% had a neutral response. This level of satisfaction is considered low compared to other international residency programs (e.g., Italy, 54.9%).[15] As evident in Figure 1, there is a significant discrepancy in satisfaction level between different cities and regions in Saudi Arabia. The high level of satisfaction achieved by training centers in Riyadh region reflects that the restructuring of urology training program improved the quality of training. Residents in training centers of the western region (i.e., Makkah, Taif, and Al-Madinah) reported low satisfaction with all aspects of training. This discrepancy is attributed to various factors, including difference in clinical and surgical exposure between large referral centers and those in rural areas, failure of implementing the new restructured curriculum by academic affairs in training centers, and absences of proper feedback in between residents and their trainers. This indicates the need for the following: providing more mentorship support to juniors and improving the quality of training during these early years; invest in a national health-care system that provides access to patients in peripheral centers with quality assurance rather than relying on referral of cases to centers in Riyadh; convert the training program into a joint program, where residents rotate in a scheduled rather than arbitrary manner to help standardize the quality of training programs nationally; and integrate adjuncts of training, such as high-fidelity simulation and animal laboratories, nationally. These measures have been shown to improve the quality of training and trainees’ satisfaction.[1617181920]

Although the Scientific Committee for urology training program in SCFHS has issued a new curriculum that detailed all the duties, clinical and surgical requirements, and assessment methods, most participants agreed that the assessment process was vague. This deficit might be a result of residents’ lack of enlightenment about the curriculum as it is their responsibility to be familiar with all aspects of their training. Academic affairs in governmental training centers, program directors, and trainers also share part of the responsibility to educate residents about the objectives of training and assessment methods. This reflects the importance of a clear, structured, regular assessment process in improving the quality of training and satisfaction, which have been shown in multiple studies, and have received little attention from program directors, for multiple reasons, including time constraint on expert assessors and the disruption of continuity of care.[21] However, these concerns can be circumvented by recruiting physician assistants and technicians to help assess technical skills of trainees, and by adopting a more time-flexible assessment process such as through distance and e-learning platforms, and integrating assessment process in clinical care such as formal assessment during morning reports, handovers, grand rounds, and mortality and morbidity meetings.[2223]

Most residents and graduates felt satisfied by their clinical and surgical exposures in endourology, pediatric urology, and uro-oncology. However, most of them felt that their exposure in the rest of urological subspecialties was inadequate. This observation is attributed to the fact that urolithiasis comprises the bulk of surgical practice in most of the training centers in Saudi Arabia. More than half of senior residents have performed <10 cases of common urological procedures performed by the general urologist such as microscopic varicocelectomy, hydrocelectomy, and circumcision. Surprisingly, 30.2% of graduates reported no microscopic varicocelectomy cases performed during their residency training. When comparing residents’ exposure in Saudi Arabia to other urology residents worldwide,[1524] we noticed that hands-on training is suboptimal in the SBU. Moreover, a significant discrepancy was also noticed in the amount of surgical exposure in-between residents and graduates in different region of the country where training centers in the western region (i.e., Makkah, Taif, and Al-Madinah) providing less chance of clinical and surgical exposure to residents. Although the SCFHS follows a very strict policy and procedures in approving and accrediting each training center and a full year interhospital rotation is granted to each resident during his 4th year of residency to cover the defect of his own training center, these efforts did not translate in improving residents’ satisfaction about clinical training and surgical exposure.

Research activities and number of publications of residents were suboptimal. Only 46% of respondents have published at least one research in a peer-review journal during their training. Hellenthal et al. investigated the manuscript publication by urology residents in the United States and Canada and reported that 81% of residents submitted at least one manuscript for publication and 66% of residents published at least one manuscript during their residency.[25] Lately, the SCFHS made research projects and publications a mandatory aspect of residency evaluation.

Limitations of this study are the inherent biases of the design (subjectivity, recall bias, and selection bias). The strengths of this study include the high response rate making it more representative of the targeted population (i.e., residents in training) at all levels of training. In addition, to the best of our knowledge, this is the first evaluation study post the 2014 restructuring (adoption of the CanMEds Competency Framework). Future studies should use more robust study design and objective instruments to measure the quality of urology training programs process and outcomes in a longitudinal fashion and should focus on standardizing and assuring the quality of training.

CONCLUSION

Satisfaction of residents with the urology program process is variable according to the city of training. Having high satisfaction level in some cities reflects the improvement of urology training program after restructuring. We have identified new areas in need of improvement, namely lack of mentorship, clear and formal assessment process, and variation of training process between central and peripheral programs. These insights should influence an action plan by the program directors’ committee.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.