Assessment of Postgraduate Online Medical Education During the COVID-19 Pandemic in Saudi Arabia: A Cross-Sectional Study.

Purpose: The COVID-19 pandemic has limited the traditional way of teaching due to contact restrictions and the trainees being the front-line providers of patient care in certain specialties. During the pandemic, many academic institutes have adopted various methods for utilizing online learning as an alternative to traditional teaching. Numerous studies reported the impact of these changes on medical education with varying results. As such, comprehensive assessments are necessary to evaluate the outcomes of this rapid transformation. The aim of this study was to provide qualitative and quantitative assessments of post-graduate online medical education during the COVID-19 pandemic in Saudi Arabia. Participants and
Methods: In this cross-sectional study, an online questionnaire was distributed among postgraduate trainers and trainees in Riyadh second health cluster. The questionnaire was used to assess the experiences, perception, coping, satisfaction and preferences of medical trainers and trainees towards online education during the COVID-19 pandemic.
Results: A total of 207 participants were involved in this study. While the sociodemographics differed between trainers and trainees, age was significantly associated with negative pre-pandemic online learning experiences. Stress was reported among both groups and was significantly correlated with the pre-pandemic computer and internet competency. Coping was reported to be easier by trainers compared to trainees. The overall perception of online learning was positive in 73% of the respondents. Perception significantly correlated with age, stress, coping and satisfaction (P < 0.0001). The majority of trainees were interested in a hybrid mode learning, combining traditional teaching with online education.
Conclusion: There is a significant difference between trainers and trainees with regard to their experience of online education. Further studies are required to assess how to effectively implement online education in postgraduate training programs and identify strategies to overcome the reported deficiencies.

Introduction

Online learning (eLearning) was gradually incorporated into medical education over the past 20 years, which has paralleled the increased use of eLearning across all workforce sectors.1 A review published in 2006 concluded that eLearning would be “one of the most important developments in the delivery of postgraduate medical education.”1 The authors of that review article, and many others who shared similar views in the early internet era, could not have known how that prediction would be tested. However, as the world came to be immersed in the SARS-CoV-2 (COVID-19) pandemic in 2020, eLearning surely became important in the delivery of postgraduate medical education.

The COVID-19 pandemic caused a massive change worldwide; affecting all areas of workforce including education. This state of emergency has led to many modifications within the healthcare system, such as cancelling elective surgical procedures, reducing the volume of acute-care surgeries, closing all outpatient clinics, limiting the presence of trainees on service and calling-off departmental educational activities.2–6 This resulted in significant interruptions of clinical rotations. In addition, traditional in-person academic activities such as face-to-face teaching and simulation labs were halted; examinations, courses and conferences were postponed on an international level.7,8 Almost overnight, online learning transitioned from its status as a developing option to becoming mandatory if education were to continue.

The rapid transition from traditional face-to-face to eLearning has transformed the way medical education was delivered and posed many challenges to trainers and trainees involved.4,9 Synchronous and Asynchronous eLearning modalities have been utilized by several institutions during this period. Numerous platforms were utilized for delivering academic content; the most frequently used were ZOOM and Microsoft-Teams.10,11 Innovative teaching modalities took place, including the “flipped-classroom” method, where learners were provided with didactic materials and pre-recorded videos prior to the educational session.11–13 Other strategies were implemented to accommodate for teaching clinical skills and ensuring the continuity of clinical education, achieved through video-recorded surgical procedures as well as providing telehealth patient consultations.11 With regard to evaluation, the most commonly reported assessment method during this period was in the form of multiple-choice questions; other studies reported the conversion of the standard Objective Structured Clinical Examinations (OSCE) to an online version.11,14

Numerous obstacles and challenges have been reported as a result of this massive transition. Trainers, teachers, and educators were required to rapidly adapt to digital technologies; trainees encountered difficulties with poor bandwidth connectivity; accessibility and time management issues were evident, as well as communication challenges due to the lack of non-verbal language.11,15

The resulting global experience with online medical education is being shared primarily as information gathered from user surveys. Although quantitative data are essential, detailed qualitative data are as necessary today as they were in the early studies, to allow comprehensive and reliable investigations of this complex intervention comprising “multiple human components (teachers, learners, etc.) interacting in a nonlinear fashion to produce outcomes which are highly context dependent.”16

Accordingly, we distributed an online survey to postgraduate medical learners and teachers in Riyadh, Saudi Arabia. These data are expected to supplement the expanding total of literature, adding to the reported experiences and possibly contributing to the development of strategies that can resolve specific issues, gaps, and deficiencies in online postgraduate medical education. The aim of this study is to provide qualitative and quantitative assessments of postgraduate online medical education during the COVID-19 pandemic amongst trainers and trainees in Saudi Arabia.

Research Objectives

The research had three main objectives. The first was to describe the experiences, coping, perceptions, satisfaction and preferences for online learning by medical trainers and trainees. The second was to determine how the experiences correlated with perceptions, satisfaction, and preferences. The third and foremost objective was to test the null hypothesis: no difference between trainers and trainees on various aspects of online learning.

Materials and Methods

In this cross-sectional study, a questionnaire was developed through a review of recent publications on online learning during the Covid-19 pandemic in addition to the experiences of the researchers involved. At first, the questionnaires used in previous similar studies were carefully reviewed by the authors. Then, more questions were added based on the experiences of the researchers involved— whom were either active trainers, learning managers or decision makers during the pandemic. The researchers discussed the items of the questionnaire for relevance and finalized only 43 of them.

The questionnaire was pre-tested on a pilot number of potential respondents, with care being taken to exclude them from the main survey. The questionnaire comprised several sections addressing the sociodemographics of participants and measures of experiences, perception, satisfaction, and preferences. Table 1 shows the list of variables used in the research and their corresponding questions. Nominal variables were recorded as multiple-choice questions, while ordinal variables were scored based on a 4-point or 5-point Likert Scale.

Table 1

Variables of the Study

Category	Items
Socio-demographics	S1. Age (NOM)	S4. Size of household (NOM)
	S2. Gender (NOM)	S5. Specialty (NOM)
	S3. Marital status (NOM)
Transition from traditional to online learning	Q1. Pre-pandemic computer and internet competency (ORD)	Q13. Stress associated with sudden transition from traditional to online learning (ORD)
	Q2. Pre-pandemic online learning experience (NOM)	Q14. Coping with online learning during the pandemic (ORD)
	Q12. Guidelines before the start of online learning activities? (NOM)
Changes in institutional policies, procedures, and support during the pandemic	Q9. Extent of redesigning teaching courses for online learning during the pandemic (ORD)	Q11. Institutional support to online learning during the pandemic (ORD)
	Q10. Extent of policy and procedure change to accommodate online learning during the pandemic (ORD)
Online experiences during the pandemic	Q3. Online learning software during the pandemic (NOM)	Q6. Site of online learning activities during the pandemic (NOM)
	Q4. Online learning equipment during the pandemic (NOM)	Q7. Online learning activities during the pandemic (NOM)
	Q5. Hours of online learning activities per week during the pandemic (NOM)	Q8. Online assessment activities during the pandemic (NOM)
Perceptions of and satisfaction with online learning	Q15. Overall perception of online learning during the pandemic (ORD)	Q17. Difficulties experienced in online learning during the pandemic (NOM)
	Q16. Satisfaction with online learning during the pandemic (ORD)	Q19. Unique opportunities and challenges presented by online learning (NOM)
Comparison of online to traditional face to face learning	Q20. Online learning better for understanding questions (ORD)	Q27. Online learning is associated with higher examination stress (ORD)
	Q21. Online learning is better for teaching skills (ORD)	Q28. Online learning is better in economizing resources (ORD)
	Q22. Online learning is better for teacher-learner communication (ORD)	Q29. Online learning is more fun (ORD)
	Q23. Online learning is better for effective time use (ORD)	Q30. Online learning is academically more stimulating (ORD)
	Q24. Online learning is better for reducing academic stress (ORD)-	Q31. Online learning gives more free time to learners (ORD)
	Q25. Online learning is better in overall satisfaction with learning (ORD)	Q32. Online learning gives more free time to teachers (instructors) (ORD)
	Q26. Online learning is better for learner-to-leaner communication (ORD)
Preferences for online learning	Q33. Time of day preferable for online learning activities (NOM)	Q36. Method of assessment for which online learning is most effective (NOM)
	Q34. Appropriate duration of an online learning activity (NOM)	Q37. Preference for the post pandemic period (NOM)
	Q35. Type of learning activity for which online learning is most effective (NOM)	Q. 38 Aspects of online learning to continue after the pandemic (NOM)

Abbreviations: ORD, Ordinal variable; NOM, Nominal variable.

Institutional Review Board (IRB) approval was granted on 1 July 2020 from the local IRB at King Fahad Medical City—under category “Exempt” based on Good Clinical Practice (GCP) Guidelines. The questionnaire included a cover-letter describing the purpose of the study along with a statement of informed consent for research participation—which was developed in accordance with the local IRB guidelines. No participant identifiers were collected as part of our survey and the responses were anonymous. The involved researchers maintained adherence to GCP guidelines throughout the duration of the study.

The questionnaire was distributed by e-mail on August 21, 2020, to 1200 trainers and trainees of academic medical centers within the Riyadh 2nd Health Cluster, which included King Fahad Medical City, Prince Muhammad Ibn Abdulaziz Hospital, Al Yamamah Hospital, King Salman Dialysis Center and three specialized dental centers. The sample size was computed using the Raosoft online formula as 205; based on a total population of trainees and trainers of 1200, alpha error of 0.05, 95% confidence level, and a 2:8 distribution based on the estimated trainee-to-trainer ratio. A follow-up was made 1 week after the initial e-mail via a reminder e-mail and/or phone call. Data received were checked and edited for consistency and accuracy. Open-ended items were coded, except for 2 questions (Q18, Q38) which were analyzed qualitatively. The variable “position” was defined as either “trainee” or “trainer.”— “Trainee” was coded to include residents R1-R4, interns, fellows and pre-scholars; “Trainer” was coded to include consultants, assistant consultants, and program directors.

Statistical analysis was carried out using SPSS v.26 to cross-tabulate frequencies of the variables and test for association using the chi square statistic, with significance being set at 0.05 using 2-sided asymptotic p values. Both the Spearman and the Kendall tau correlation coefficients were computed with a critical value of significance of 0.05 and 2-sided p values. The main objective of the analysis was to determine and explain the differences between trainers and trainees. Where indicated, multivariate logistic regression models were used to test for association while controlling for confounding to determine independent associations.

Results

Study Sample Socio-Demographics (S1–5)

A total of 1200 emails were sent out, of which 207 were returned giving a response proportion of 17%. Table 2 shows the sample characteristics. There were significant differences between trainees and trainers in age, gender, household size and specialty, but only age showed independent association after running a logistic regression model containing all the socio-demographic variables.

Table 2

Sample Characteristics

		Trainee	Trainer	Total	P value
Variable	Categories	NumberCol %	NumberCol %	NumberCol %
S1. Age (NOM)	20–25	2818.2%	23.8%	3014.5%
	26–30	8454.5%	00%	8440.6%
	31–35	2717.5%	1120.8%	3818.4%
	36–40	74.5%	1732.1%	2411.6%
	Above 40	85.2%	2343.4%	3115.0%	P=0.000
	Subtotal	154100.0%	53100.0%	207100.0%
S2. Gender (NOM)	Male	7649.4%	3566.0%	11153.6%
	Female	7850.6%	1834.0%	9646.4%
	Subtotal	154100.0%	53100.0%	207100.0%	P=0.036
S3. Marital status (NOM)	Married	6944.8%	4686.8%	11555.6%
	Not married	8555.2%	713.2%	9244.4%
	Subtotal	154100.0%	53100.0%	207100.0%	P=0.000
S4. Size of household (NOM)	0–3	5032.5%	611.3%	5627.1%
	3–5	6340.9%	3056.6%	9344.9%
	6–9	3422.1%	1426.4%	4823.2%
	9+	74.5%	35.7%	104.8%
	Subtotal	154100.0%	53100.0%	207100.0%	P=0.028
S5. Specialty (NOM)	Medical	5233.8%	1120.8%	6330.4%
	Pediatric	3120.1%	1222.6%	4320.8%
	Surgical	2113.6%	59.4%	2612.6%
	Emergency	159.7%	35.7%	188.7%
	Rehabilitation	138.4%	59.4%	188.7%
	Ob/Gyn	138.4%	47.5%	178.2%
	Radiology	63.9%	35.7%	94.3%
	ICU	21.3%	35.7%	52.4%
	Dental	10.6%	713.2%	83.9%
	Subtotal	154100.0%	53100.0%	207100.0%	P=0.003

Abbreviation: NOM, nominal variable.

Transition from Traditional to Online Learning and Changes in Institutional Policies, Procedures, and Support (Q1, 2, 9–14)

Table 3 shows reported transitions and changes due to the pandemic. A high proportion, 82.1%, rated their pre-pandemic computer and internet experience as high or medium and there was no significant difference between trainees and trainers. One-quarter, 25.6%, had no online learning experience before the pandemic, with a higher proportion among trainees. There was no significant correlation between their pre-pandemic computer and internet competency with pre-pandemic online learning experience. Age was not correlated with the pre-pandemic computer and internet competency but was significantly negatively correlated with pre-pandemic online learning experience (rS= −0.257, P < 0.000).

Table 3

Transition from Traditional to Online Learning and Changes in Institutional Policies, Procedures, and Support

		Trainee	Trainer	Total	P value
	Category	NumberCol%	NumberCol %	NumberCol %
Q1. Pre-endemic computer and internet competency (ORD)	High	6944.8%	2649.1%	9545.9%
	Medium	5837.7%	1732.1%	7536.2%
	Fair	2013.0%	917.0%	2914.0%
	Very low	63.9%	00.0%	62.9%
	None	10.6%	11.9%	21.0%
	Subtotal	154100.0%	53100.0%	207100.0%	0.458
Q2. Pre-endemic online learning experience (NOM)	0–1 years	5435.1%	2343.4%	7737.2%
	2–3 years	2918.8%	1018.9%	3918.8%
	4–5 years	117.1%	47.5%	157.2%
	5+ years	138.4%	1018.9%	2311.1%
	None at all	4730.5%	611.3%	5325.6%
	Subtotal	154100.0%	53100.0%	207100.0%	0.036
Q9. Extent of redesigning teaching courses for online learning during the pandemic (ORD)	Drastic change	6542.2%	2852.8%	9344.9%
	Minor change	5737.0%	1935.8%	7636.7%
	No change	2214.3%	35.7%	2512.1%
	Not applicable	106.5%	35.7%	136.3%
	Subtotal	154100.0%	53100.0%	207100.0%	0.324
Q10. Extent of policy and procedure change to accommodate online learning during the pandemic (ORD)	Drastic change	1610.4%	59.4%	2110.1%
	Major change	5837.7%	2445.3%	8239.6%
	Minor change	3724.0%	1732.1%	5426.1%
	No change at all	2013.0%	00.0%	209.7%
	Did not know	2314.9%	713.2%	3014.5%
	Subtotal	154100.0%	53100.0%	207100.0%	0.073
Q11. Institutional support to online learning during the pandemic (ORD)	High support	5334.4%	1528.3%	6832.9%
	Moderate support	5737.0%	2037.7%	7737.2%
	Minimal support	2818.2%	1222.6%	4019.3%
	No support at all	1610.4%	611.3%	2210.6%
	Subtotal	154100.0%	53100.0%	207100.0%	0.827
Q12. Guidelines before the start of online learning activities? (NOM)	Yes, adequate	5535.7%	1324.5%	6832.9%
	Yes, inadequate	4227.3%	917.0%	5124.6%
	None at all	5737.0%	3158.5%	8842.5%
	Subtotal	154100.0%	53100.0%	207100.0%	0.024
Q13. Stress associated with sudden transition from traditional to online learning (ORD)	Extreme stress	95.8%	23.8%	115.3%
	Major stress	2314.9%	1120.8%	3416.4%
	Minor stress	6039.0%	2954.7%	8943.0%
	No stress	6240.3%	1120.8%	7335.3%
	Subtotal	154100.0%	53100.0%	207100.0%	0.054
Q14. Coping with online learning during the pandemic (ORD)	Very well	6441.6%	1732.1%	8139.1%
	Well	5636.4%	3056.6%	8641.5%
	Not sure	1811.7%	47.5%	2210.6%
	Bad	106.5%	11.9%	115.3%
	Very bad	63.9%	11.9%	73.4%
	Subtotal	154100.0%	53100.0%	207100.0%	0.113

Abbreviations: ORD, ordinal variable; NOM, nominal variable.

There was a positive correlation between the reports of ‘redesigning teaching courses’ and “change of procedures to accommodate online learning” (rS= 0.360, P < 0.0001). Major or drastic changes were reported in both by 44.9% and 39.6% of respondents, respectively. The trainees differed from trainers in reporting provision of guidelines before the start of online learning activities (P < 0.024). About 27.3% of trainees reported inadequacy, while a higher proportion of trainers, 58.5%, reported absence of guidelines. About 70.1% of respondents reported a high level of institutional support for online learning with no significant difference between trainees and trainers.

A high proportion of respondents, 80.6%, reported coping well or very well with online learning, with no significant differences between trainers and trainees. Coping was negatively correlated with age (rS= −0.151, P < 0.030), positively correlated with the pre-pandemic computer and internet competency (rS= 0.202, P < 0.004) but not correlated to pre-pandemic online learning experience (rS= −0.094, P < 0.177).

The transition from traditional to online learning was associated with stress. About 24.6% of trainers reported extreme or major stress compared to 20.7% among trainees, and there was overall significant difference in stress between trainers and trainees. Stress was correlated with the pre-pandemic computer and internet competency (rS=−0.162, P < 0.020), but not with prior experience of online learning or with age; as (rs= 0.035, P = 0.562) and (rs= 0.045, P = 0.453), respectively.

Online Learning During the Pandemic: Experiences, Perceptions and Satisfaction (Q3–8, 15–17, 19)

Zoom was the preferred software by 94.2% of respondents. All modern equipment were used equally by trainees and trainers; laptops, smartphones and tablets, with desktop computers being the least (9.2%). More than half of the respondents, 53.6%, reported spending 4 hours or more per day on online learning activities, but there was no significant difference between both groups (P < 0.224). The majority of online learning activities, 92.3%, took place at home during quarantine period, with no significant differences between trainees and trainers.

The 2 most popular learning activities were lectures and seminars/webinars accounting for a total 87.9% of all activities, but trainees reported more lectures while trainers reported more seminars/webinars. Case presentations/discussions was the most popular form of assessment at 49.3%, followed by short oral examinations 15.5% and online OSCE 12.6%; the rest of the assessment methods were rarely used. It is noteworthy that there were significant differences: trainees reported more case presentations/discussions, 55.8%, while trainers reported more short oral examinations 26.4% and online OSCE 20.8%.

Overall perception of online learning was very positive or positive, totaling 73%, with no significant differences between trainees and trainers. Overall perception was correlated with age (rS= −0.0213, P < 0.002), with stress (rS= −0.359, P < 0.00), with coping (rS= 0.672, P < 0.00) and with satisfaction (rS= −0.835, P < 0.000).

On a Likert scale of 1–5, 71.1% of respondents were either satisfied or highly satisfied and 3.9% were very unsatisfied, with no differences between trainees and trainers. Satisfaction was correlated with age (rs = −0.136, P < 0.020), Pre-pandemic computer and internet competency (rs = 0.146, P < 0.016), stress (rS= −0.363, P<0.0001), and coping (rS= −665, P < 0.0001) but was negatively correlated to overall perception (rS= −0.835, P < 0.000).

While the response to the item on difficulties of online learning was 39.6%, 15.0% reported technical problems as the most common problem, with trainers reporting 22.6% and trainees reporting 12.3%. A question on challenges and opportunities presented by online learning revealed that 39.6% agreed that online learning presented challenges and opportunities while 15% thought it did not; however, this item had a non-response of 42.5%.

Comparison of Online to Traditional Face to Face Learning (Q20–32)

Respondents were asked to compare online to traditional learning on 12 items using a 5 point the Likert scale. These items were highly correlated. In total, the proportion, of “strongly agree” varied between 19.8–47.8% among trainees and 8.2–35.3% among trainers. Significant differences between trainees and trainers were observed in 6 out of 12 variables. Learners were more satisfied with learner to teacher communication 35% vs 30.9%; teacher to learner communication 39% vs 17%; reduction in academic stress 72.1% vs 62.3%; overall satisfaction 57.1% vs 39.7%, and academic stimulation 66.9% vs 75.5%. Trainers agreed that online learning had less stress 62.3% vs 72.1% and that it gave more time to teachers 75.5% vs 66.9%.

A wide diversity of subjective responses was given to the open-ended item asking for what was missed from traditional learning. The most common factors being interaction and associated items, like body language and engagement. Other subjective responses related to classroom dynamics such as good teaching, attention, understanding, explanation and concentration. Others related to the need for human contact such as commitment, passion and activity. The rest of the responses were more objective and measurable such as clinical practice, supervision, time, communication, verbal feedback, and workshops. Some responders said nothing was missing.

Preferences Regarding Online Learning (Q33–38)

The reports on preferred time for online learning activities showed variability with no significant differences between trainees and trainers. The highest was evening hours after work 30.4% closely followed by afternoon working hours 29.5%, morning working hours 22.2%, and night after work 15.0%. Weekends were the least popular 2.9%. Most respondents, 86%, preferred the duration of online learning activities of not more than 2 hours with no significant difference between trainees and trainers. Lectures and case discussions were preferred by 95% of both trainees and trainers as the most effective online learning activities. Respondents considered multiple choice questions 54.6% and problem-solving questions 28.5% as the most effective assessment methods. With regards future preferences, 64.3% preferred combining traditional and online methods but a respectable proportion of 25.5% preferred continuing eLearning as the sole method of education. A negligible proportion of 9.7% wanted discontinuing online learning and returning to traditional methods.

Many respondents did not answer the item about what aspects of online education should continue post pandemic. Lectures and case presentations/discussion were the most mentioned, but these did not come with online learning. Five preferred returning to the traditional methods. One wanted “everything in online learning to continue post-pandemic”.

Discussion

Differences Between Trainers and Trainees

Online medical education is not new; however, the rapid switch to exclusive online learning worldwide required institutions to take a deep dive into what was previously considered a complementary educational tool. Although most publications are related to medical school education, several considerations and issues are common among all educational levels.11,17 Our research adds to the increasing number of reports documenting issues and perceptions in response to the transition to online learning during the COVID-19 pandemic.

The major null hypothesis of this study states that there was no difference between trainees and trainers on all variables relating to online learning within our study. Knowing the differences is important for tailoring future online activities to suit the abilities and expectations of trainees and trainers alike. Gender differences were not significant in our study; however, a generation gap was obvious from the data. The trainers being older differed in being more married, having larger households and specialties. Besides socio-demographic variables, trainees and trainers had significant differences on 11 variables that can be grouped as transition from traditional to online learning, online assessment activities, and comparison to traditional learning.

Transition to eLearning, Stress, Coping, Perception and Satisfaction

In the transition to online learning, trainees had more years of prior experience with online learning, which reflects the generation gap. The trainees, being younger, are well versed in computer technology and internet use compared to older trainers.

Several variables were correlated with age as the underlying determining factor. Previous experience of online learning by trainees (younger in age) has made their transition to pandemic online learning easier. Younger age explains better coping with online learning through its positive correlation with pre-pandemic computer and internet competency. The younger trainees had lower overall perception, which may be related to having less experience in learning methods and outcomes in general.

A recent literature review summarized barriers and solutions to developing and implementing online learning programs for medical students and postgraduate trainees in global settings; however, they were not complicated by the challenges of a worldwide pandemic.17 Time and infrastructure issues were 2 barriers during such comparatively relaxed environment in studies reported from 2006 through 2015. A pre-pandemic United States national survey of 214 internal medicine residency program directors reported that synchronous online learning was used by 40% and asynchronous learning by 72% of programs.18 The asynchronous programs were considered to be more accommodating of resident schedules and duty hour restrictions. However, even in those non-urgent settings, faculty development was considered to be less than adequate (30%) or non-existent (56%) by 86% of respondents. In our survey, 74% of trainers reported being provided with few (18%) or no (56%) guidance prior to implementing online learning. Trainees in our study were more prepared; however, barely one-third (33%) had received adequate guidelines before embarking on their online learning activities. Conversely, a survey study at the College of Pharmacy (COP) at King Saud bin Abdulaziz University for Health Sciences in Saudi Arabia reported that almost two-thirds of the students believed that the COP was well prepared for the complete transition to online learning during the COVID-19 pandemic.19 In our study, the trainees were more aware of guidelines before the start of online learning—they must have sought sources other than the trainers. Most likely, they looked for or demanded the guidelines.

As with any transition from the familiar to the new, the introduction of online learning was associated with stress. The trainers experienced more stress with eLearning, which is explained by their shorter experience with online learning before the start of the pandemic. A focus group study of 60 undergraduate medical students’ perceptions of online learning was carried out in Qassim region of Saudi Arabia during April and May 2020. The study reported that, similar to observations in most studies of online learning, technical issues were common barriers.15 The authors emphasized that providing technology training courses to teachers is essential. Deficiencies in these skills could contribute to stress during the transition period. In our study, almost one-fourth (24%) of teachers experienced major or excessive stress while transitioning to an online learning setting; while only 4% had a negative coping experience once learning activities were underway.

Our results indicated that perception of online learning was positively correlated with coping, which is logical, and was negatively related to stress, which is understandable. Its negative correlation with overall satisfaction is explainable by the clash between the ideal of a high perception and the actual satisfaction from the reality of the experience. Indeed, as online teaching methods are being regarded as an efficient tool for learning, the quality of eLearning was expected to be comparable to traditional methods; and learning outcomes were not expected to be compromised.19 Most reports of experiences after the transition to online learning in medical education are also related to medical school education.20 For example, a survey study involving pre-clinical students was carried out at the University of California at San Diego during March and April 2020. They reported that, in general, students believed the quality of instruction and their ability to participate were negatively affected by remote learning.21 However, the short interval that has transpired since the start of the pandemic did not make-way for examining online learning outcomes achieved by students, with either positive or negative attitude towards it. Furthermore, in November 2020, 30 residents in a Mexican general surgery residency program (PG1-PG5) participated in a study surveying their experience after transitioning to online learning since April 2020.22 Although the academic and organizational level was considered higher than that provided by traditional learning, by 47% and 67% of participants, most (57%) were neutral about whether there was a concomitant increase in their academic performance; and whether the changes had been more useful for their training (53%) compared with their previous training. A systematic review of 29 qualifying articles examined the impact of COVID-19 on all aspects of surgical training, including the transition to online learning.23 Acknowledging decreased hands-on surgical experience; patient exposure was ubiquitous, which was in some cases accommodated partially by simulations and telemedicine. Although two studies reported 65% and 82% of trainees had favorable opinions of their online learning; in one study, 65% of trainees believed that even their theoretical training had been negatively affected.

Online Learning and Assessment Activities

Trainees differed from trainers in preferences of learning activities. Trainees had a higher proportion of lectures, while trainers had a higher proportion of seminars/webinars. There were notable differences between trainees and trainers regarding assessment activities; trainees mentioned highest participation in case presentations/discussions, while trainers mentioned more short oral examinations and online OSCE. The differences in describing the activities that took place are difficult to explain, since they experienced the same learning activities. There is a possibility that the question item was not understood by one or both groups; they may have indicated their preferences instead of their observations or the preferences biased the observations. We have no data on the distribution of respondents by hospital. It is possible that trainers who responded to the questionnaire were from hospitals that practiced online learning differently from other hospitals to which most trainees belonged.

OSCE assessment was not commonly reported by our study participants. The fact that 15% of teachers and 9% of students reported it as an assessment method and did not recommend it for the future, suggesting that they may have had a negative experience. Although OSCE has been in use for many years, its effectiveness and role continues to be examined. Some pre-pandemic studies on OSCE assessments for medical students reported higher stress and difficulty levels compared to traditional assessments.24 However, positive experiences of OSCE use in medical school have also been reported. A small survey study in a teaching hospital in Dammam, Saudi Arabia, reported that 63% of students and 80% of faculty believed that OSCE provided a fair assessment of clinical skills; and approximately two-thirds agreed that it was an enjoyable experience.25 Postgraduate students have also reported positive experiences with OSCE. In 2012, a survey of 66 internal medicine residents’ perceptions of OSCE was examined in Saudi Arabia, after implementation of OSCE as part of the final clerkship exam in 2008.26 On a 5-point Likert scale with 5 indicating strong agreement, the mean score was 4.5 for items asking if the exam was well administered, well structured and if staff guidance was helpful. The mean agreement score for the items asking whether OSCE was stressful was 3.5 and was 2.3 for intimidating. The authors concluded that the overall perception of the residents towards OSCE was favorable and encouraging.

Prior to the COVID-19 pandemic, a small number of postgraduate training programs reported their experiences with virtual OSCEs.27 Subsequently, when the UCL medical school in London canceled face-to-face assessments in response to COVID-19, an online 18-station timed OSCE was convened.27 Assessments were similar to those used in traditional OSCE, including clinical communication skills, written communication, practical skills, examination skills, and professionalism. The authors shared 12 practical tips compiled from their experience and from the literature that can help in the design and delivery of online OSCE. The Harvard School of Dental Medicine developed an online OSCE during the COVID-19 pandemic using the Zoom eLearning platform, because it featured breakout rooms where private mini-sessions could be created by the host.14 Students signed-in and were allocated to their breakout rooms; then progressed through the rooms when the allotted time had passed. Most students thought the online OSCE was as successful as traditional OSCEs, and all students believed they were able to completely demonstrate their knowledge. Examiners also had positive impressions of the online OSCE and emphasized the importance of calibrations and run-throughs prior to launch. Technical issues were the only difficulties encountered. The authors believed there was value in moving the in-person assessments online in the post-pandemic era.

Comparison to Traditional Learning

Trainees differed from trainers on 6 out of 11 items comparing online to traditional learning. Trainees were more apt to agree that online learning was better for teacher-learner communication, reducing academic stress, overall satisfaction and academic stimulation, whereas the trainers disagreed. These disagreements are explainable by the phenomenon of the generation gap considering the experience with computers and the internet, which is higher in the younger trainees. Trainers agreed more than trainees that online learning gave more time to teachers. This is understandable because online learning saves transient time between events such as movement to and within the hospital.

A recent meta-analysis of studies reported from 2000 to 2017, which compared online with offline undergraduate medical education, reported that knowledge and skills were significantly improved with online learning.28 Although not all studies in the review signify that online learning was more effective, none concluded that online learning was less effective than traditional learning. A single-center US study after the COVID-19 pandemic began included 81 emergency medicine and internal medicine residents. They reported that participants preferred in class interactions with peers (85%) and lecturers (80%); with 62% reporting decreased engagement with lecturers during online conferences.10 Residents were significantly more engaged in other tasks during online conferences compared with in-class attendance. In our study, 35% of participants disagreed that online learning was academically more stimulating compared with traditional learning settings.

Changes in communication opportunities may contribute to decreased acceptance or effectiveness of online learning. An Egyptian survey (N = 78) and focus group (N = 25) study examined faculty perceptions of medical school responses to the COVID-19 pandemic. They reported that communication issues between faculty and students led to student detachment.29 Almost two-thirds (63%) of our teachers disagreed that online learning enhanced teacher-learner communication, and 57% did not agree that it improved learner-teacher communication. Although our students expressed more positivity, only 39% believed learner-teacher communications were improved in the online setting, and 35% believed teacher-learner communication was improved.

A survey of 538 clinical years (fourth through sixth year) medical students was performed in all medical schools across Jordan; less than 2 months after a state of emergency was proclaimed in response to the COVID-19 pandemic in 2020.30 Over half of the participants (62%) reported poor interaction with instructors as a drawback, with only 14% reporting better interaction in an online learning setting. More students would prefer a hybrid approach in the future whether they were satisfied (22%), neutral (24%), or dissatisfied (29%) with the online learning; and a return to traditional learning was preferred by more students dissatisfied with online learning (15%) compared with neutral (4%) and satisfied (1%) students. The majority of our teachers and students advocated for hybrid (65%) or online only (26%) education in the post-pandemic period; accordingly, efforts to maximize the benefits provided by online education are warranted.

A cross-sectional study in India included 55 postgraduate surgery residents who were without previous exposure to online teaching; the study was performed 1 month after transitioning to online didactic training during the COVID-19 lockdown.31 The transition included an orientation program for all teachers and residents. Individual items on the quality of online teaching did not indicate perceived superiority of either online or traditional learning; however, the authors considered the overall quality perception of online teaching to be favorable. The participants in our study were almost exclusively involved in theoretical learning and teaching. Transitioning to online education in this capacity does not require the capabilities, infrastructure, and inputs that are required to provide online clinical training. Overall, one-half of our study participants disagreed that online learning was better for teaching skills, with a higher percentage of teachers sharing this view (63%) compared with students (45%). Similarly, a survey study of medical schools was conducted in Libya in mid 2020; stating that over half (55%) of the participants disagreed or strongly disagreed that online learning can be used for teaching clinical aspects of medical sciences.32

Recommendations

A review of 14 studies on adaptive processes to the COVID pandemic in undergraduate and residency programs concluded that re-modulating the educational approach provided positive opportunities for personal and professional growth.11 However, the authors acknowledged that these qualitative narrative studies did not systematically analyze the characteristics and results of the changes that were introduced. They also believed that many of the enhancements that were described would require economic inputs that are not achievable in many parts of the world. Another study also suggested that the shift toward online education may have a lasting positive impact.33

The historical use of online videos for teaching procedural skills to postgraduate medical learners was exemplified when a systematic review was able to identify 20 qualifying articles published between 2009 and 2019.34 In the pre-pandemic period, the use of online videos was considered complementary to the more traditional teaching models. Virtual lectures could allow expert educators to disseminate beneficial knowledge to programs that do not have that level of expertise. Within an institution, cross-disciplinary education may be facilitated. It is considered that on demand virtual asynchronous lessons can be an inexpensive way to improve both access and content quality.35 These authors also believed that cross-institutional virtual collaborations can be part of a low-cost time conservative strategy; providing specialized training that otherwise may not be available in the learners’ institutions.

Several recommendations have emerged as medical training had to be adjusted for the limitations of COVID-19 on a global scale. Telehealth initiatives have been launched and/or expanded in several settings. When permitted by their institutions, residents can benefit from participating in virtual patient visits, reviewing charts and engaging in patient counseling under the supervision of the attending physician.36 Studies for examining the benefit of incorporating telemedicine into resident curricula are warranted. A few programs have responded to the pandemic by creating virtual video-based clinical training.37 A US undergraduate surgical education curriculum developed in response to the pandemic, emphasized the importance of using an interactive live-streaming platform for surgical experience, as well as patient-facing telehealth visits.38 The authors concluded that their virtual surgical education could be expanded for use in the post-pandemic era. Accordingly, institutions could benefit from cooperating on the development of valid strategies to incorporate clinical training into their postgraduate educational programs. Cleveland Clinic in Abu Dhabi constructed a 3-level pandemic response approach for developing and facilitating interventions determined to be necessary to maintain residency training.39 Online didactic education was supplemented by converting rounds to virtual platforms. However, their detailed framework description awaits assessment of its effectiveness and resident perceptions and satisfaction.

It is evident that currently published studies must be examined for their contributions to online learning development and implementation strategies in the future. Surveys such as our study can provide the basis for undertaking additional studies; to identify and adopt creative methods for effective online learning delivery and assessment. Tracking and identifying student skill gaps has become even more essential in a setting with reduced clinical learning opportunities. Innovation opportunities triggered by the pandemic should be taken advantage of; virtual and augmented reality technologies may be particularly important for teaching practical skills such as emergency interventions and surgical techniques.

In summary, our study adds to the baseline provided by numerous other studies of online learning experiences during the first few months of the COVID-19 pandemic. These early publications should be followed with additional studies, where details of program design and modifications in response to these initial observations are shared and realistic assessments of program effectiveness are performed. The availability of adequate qualitative details can contribute to collaborative participation in developing standardized strategies that can overcome the challenges of the pandemic and increase the quality of medical education in the future.

Limitations of the Study

The actual study sample of 207 was adequate according to calculations, but the researchers had been ambitious sending out 1200 questionnaires to capture as much diversity as possible. The relatively low response proportion of 17% is explained by the general digital fatigue in the society, because people receive many research questionnaires through the social media. The sampling could have been more representative if it was stratified to account for differences between training hospitals and academic centers. The study could have been enriched by additional qualitative research on some variables to obtain more in-depth understanding of online learning, which is a new phenomenon.

Conclusion

The main finding of the study is the difference between trainees and trainers in their experiences with online learning. This is explained by the generation gap in the acquisition and use of modern technology between the older trainers and the younger trainees. Overall, there was high proportions of coping, perceptions and satisfaction with online learning. The majority of the respondents also preferred continuing online learning combined with traditional methods in the post-pandemic period.