Is Online Video-Based Education an Effective Method to Teach Basic Surgical Skills to Students and Surgical Trainees? A Systematic Review and Meta-analysis.

BACKGROUND: Online education has been increasingly utilized over the past decades. The COVID-19 pandemic accelerated the transition of conventional face-to-face curricula to online platforms, with limited evidence for its teaching efficacy. This systematic review aims to assess the effectiveness of online video-based education compared with standard conventional education in teaching basic surgical skills to surgical trainees and students undergoing medical training.
METHODS: We performed a literature search in Embase, Medline, Cochrane CENTRAL and Scopus from inception until February 2022. Studies included were randomised controlled trials (RCTs) and observational studies. We included randomised controlled trials only for meta-analysis. The primary outcome was surgical skill proficiency. The secondary outcomes were participant perception, confidence and satisfaction. Two authors independently assessed the search results for eligibility, extracted the data and assessed the risk of bias using the Cochrane Risk of Bias tool 2. Where appropriate, we performed random effects meta-analyses of the pooled study data to calculate a standardized mean difference.
RESULTS: A total of 11 studies met the inclusion criteria totaling 715 participants; 603 were included in qualitative analysis and 380 in meta-analysis. All included studies were assessed as having a low risk of bias. The majority of studies found no significant difference between conventional and video-based education in teaching basic surgical skills, three studies found video-based education was superior and one study found conventional education was superior. There was no statistically significant difference in skill proficiency between the two groups (standardized mean difference of -0.02 (95% CI: -0.34, 0.30); p=0.90). Video-based education results in an equivalent improvement in confidence and satisfaction rates. Additional benefits of video-based education include convenience, accessibility and efficiency.
CONCLUSIONS: Basic surgical skills can be taught as effectively through online video-based education as conventional teaching methods. Online education should be utilized as an adjunct to medical curricula beyond the COVID-19 era.

INTRODUCTION

Basic surgical skills (BSS) are technical hands-on skills, considered to be fundamental competencies in early medical education. In a typical medical school curriculum, Basic surgical skills includes gowning and gloving, the handling of instruments, knot-tying and suturing. Competency in these basic surgical skills is required of all graduating medical students regardless of intent to pursue a surgical specialty. Therefore, the education of these skills is fundamental to students’ success as junior doctors. Traditionally, these skills have been taught to medical students either via textbook or face-to-face with observation by a trained physician followed by practice in skills laboratories and hospital settings.

Over the past decades, advances have been made to explore online and remote learning in multiple domains, as online education has the potential to create flexible learning, create more opportunities for those in rural and lower socioeconomic areas, reduce load on faculty and improve costs for educational institutions. As medical education involves life-long learning, methods to overcome restrictions to learning, such as limited training hours available and increased need to learn with the development of technology, are deemed advantageous. The COVID-19 pandemic accelerated the introduction of online education into the medical curriculum and rapidly pushed the majority of learning to an online format from the start of 2019. Many institutions have kept online learning integrated in curricula going forward, with the view to keep the best of both face-to-face and online learning.

The efficacy of online video-based teaching has been researched over the past decades. A person usually retains 10% to 15% of content that is read, 10% to 20% of content that is heard, and 20% to 30% of what is seen. When audio and visual materials are presented in combination, however, 40% to 50% of knowledge is retained. There is evidence for the efficacy of video-based learning in teaching advanced surgical skills including laparoscopic and robotic skills. , It has also been suggested video-based teaching of surgical skills results in significant knowledge gain, , , , , rapid skill progression and high student satisfaction.

There are not yet any systematic reviews looking specifically at the utility of online video-based education in teaching basic surgical skills. This identifies an opportunity for our systematic review with meta-analysis addressing the following research question: is online video-based education an effective method to teach basic surgical skills to students and surgical trainees? This review will provide more robust evidence for the use of online educational tools to teach practical skills, with the view to optimize future medical curricula in this unprecedented time.

METHODS

Search

This systematic review was completed in accordance with the PRISMA guidelines. A literature search of the electronic databases Medline, Embase, Cochrane CENTRAL and Scopus was performed, retrieving records from inception until February 2022. Our search term included the following keywords: ‘medical student’, ‘student’, ‘trainee’, ‘resident’, ‘video’, ‘computer based’, ‘online education’, ‘electronic learning’, ‘virtual learning’ and ‘surgery’ (see Appendix 1 for complete search strategy). The search was limited to English studies and studies published from 2002-2022 (inclusive). Primary studies including both randomized controlled trials and observational studies were included for qualitative analysis. Only randomized controlled trials were included in the meta-analysis. Studies investigating advanced or specialized procedures (such as laparoscopic or robotic surgery) were excluded. Studies utilizing technological methods other than video-based learning, such as Virtual Reality (VR), Holography Augmented Reality (AR) and simulation, were excluded. Studies investigating complete surgical curriculum education rather than technical skill training were also excluded. Studies investigating the perceptions of participants towards video-based learning (rather than the efficacy of video-based learning) were excluded. Studies lacking a control group that underwent a traditional, face-to-face education format were excluded. In addition to the articles searched by keywords, the reference list of each article fitting our criteria was scanned to ensure the inclusion of all relevant studies. This review was registered with PROSPERO (ID: CRD42022308116).

Data extraction

Two independent reviewers extracted study data from 11 eligible studies into a pre-determined template for the following parameters:

Baseline characteristics: year published, study design, participant type, number of participants, surgical procedure being studied, duration of education, gender, age, handedness, post-graduate qualifications, previous use of video learning for learning surgical skills, previous suturing training and plan to enter surgical field

Outcome(s): timepoint(s), outcome(s) assessed, blinding of participants and skill assessors

Data synthesis

Of the eight studies that were suitable for meta-analysis, six included all necessary summary statistics. The remaining two studies underwent data conversion by two independent investigators. For the data extracted from Xeroulis et al., the standard deviation was calculated from the standard error of the mean provided. For the data extracted from Tejos et al., the mean was calculated using the median and standard deviation scores provided. Cochrane's RevMan 5 software was utilized to perform a meta-analysis using pooled study data. A random-effects model was used to mitigate bias and account for potential heterogeneity between the included studies. Results were considered statistically significant if p<0.05 and confidence intervals reported at 95%. The I2 statistic was calculated to assess heterogeneity between studies. A qualitative summary of results was also done to enable a more complete comparison of studies.

Risk of bias assessment

The Cochrane Risk of Bias Tool 2 (RoB 2) for randomized trials was utilized to assess risk of bias in all included studies. It assesses five domains: random sequence generation, deviations from the intended intervention(s), incomplete outcome data, outcome measurement and selective reporting. For the two included studies that were case-control studies, relevant fields were omitted to not skew the overall risk of bias score. Publication bias was assessed by generating and examining a funnel plot.

RESULTS

The initial search yielded 987 studies. After removing 144 duplicates, 843 studies were title and abstract screened for inclusion, leaving 46 studies to be assessed. Full text screening resulted in the inclusion of 11 eligible studies. Details of the eligibility process are summarized in Figure 1 .

FIGURE 1

PRISMA flow diagram for systematic review.

Study characteristics

Study characteristics are summarised in Table Nine of the studies are randomized control trials and two case control studies. The studies were undertaken across a range of geographical locations, including the United States of America (n=3), Canada (n=2), Hong Kong, the United Kingdom, Myanmar, Uganda, Brazil and Chile.

TABLE

Study Characteristics

Study	Study Design	Participants (n)	Practical Skill	Study Groups	Assessment-Time Point (TP)-Outcome Measure (OM)	Results	Risk of Bias
Co et al. 202119Hong Kong	Prospective case control study	Medical students (62)	Basic surgical skills (linear incision, suturing, knot tying)	1. Web based surgical skills learning2. Face-to-face learning	TP: 3 weeks after interventionOM: Modified OSATS	There was no significant difference between the case and control group in the clinical competency assessment (p=1).	Low
Nathan et al. 202124UK	Randomized controlled trial	Medical students (72)	Basic surgical skills (3x interrupted sutures, knot tying)	1. Virtual classroom training (VCT)2. Face-to-face (FTF)3. Non-interactive computer based learning (CBL)	TP: immediately before and after interventionOM: OSATS	All groups produced a significant positive improvement in proficiency from baseline to post-intervention.VCT was non-inferior to FTF.VCT was superior to CBL.FFT was superior to CBL.	Low
Chien et al. 201523USA	Randomized controlled trial	Medical students (36)	Laceration repair	1. Self-directed video-based learning (VBL)2. Live workshop learning (LWL)	TP: 7 and 77 days after interventionOM: Suture task checklist	There was no difference in suturing proficiency between the VBL and LWL group at day 7 (p=0.549) and day 77 (p=0.8979).	Low
Lwin et al. 201726Myanmar	Randomized controlled trial	Medical students (50)	Basic surgical skills (suturing, knot tying)	1. Self-directed interactive video-based learning (VBL)2. Instructor-led teaching (ILT)	TP: immediately after interventionOM: OSATS	Mean OSATS scores increased significantly from pre- to post-intervention in both groups (p<0.001).There was not a significant difference in post-intervention scores between the VBL and ILT groups.	Low
Autry et al. 201327Uganda	Case control study	Interns (18)	Basic surgical skills (knot tying)	1. Video teaching session2. Face-to-face instruction	TP: before and 2 weeks after interventionOM: OSATS	Score improvement of 50%+ was achieved in 75% of the video teaching group compared to 14% of the control group (p=0.04).	Low
Bochenska et al. 201828USA	Randomized controlled trial	Medical students (50)	Basic surgical skills (knot tying)	1. Expert video (EV)2. Standard curriculum (no video) (SC)	TP: day 2 of clerkship (education session on day 1) and end of week 4OM: modified OSATS	There was a significant increase in student performance on knot-tying for both groups from pre- and post-intervention (EV: p=0.004, SC: p<0.001).	Low
de Sena et al. 201322Brazil	Randomized controlled trial	Medical students (50)	Limberg rhomboid flap	1. Computer-assisted learning (CAL- laptop with multimedia)2. Text-based education (standard print article)	TP: immediately after intervention OM: OSATS	The computer-assisted learning (CAL) group had superior performance to the text-based education group as confirmed by checklist scores (p<0.002), overall global assessment (p=0.017) and post-test results (p<0.001).	Low
Tejos et al. 202018Chile	Randomized controlled trial	Medical students (130)	Basic surgical skills (suturing)	1. Video-guided learning2. Peer feedback3. Expert feedback	TP: immediately before intervention and after final training session (4 weeks later)OM: OSATS	Post-assessment results of the peer-feedback and expert feedback groups were significantly superior to the video-guided learning group in OSATS scores (p<0.05).	Low
Xeroulis et al. 200717Canada	Randomized controlled trial	Medical students (60)	Basic surgical skills (suturing, knot-tying)	1. Control (no additional intervention)2. Self-study with computer-based video instruction (CBVI)3. Expert feedback during practice trials (concurrent feedback) 4. Expert feedback after practice trials (summary feedback)	TP: immediately before and after intervention and 1-month post-intervention OM: OSATS	The CBVI and expert feedback groups were equally effective and superior to the control group immediately post-intervention (p<0.001). However, only the CBVI and summary feedback groups retained superiority over the control at one-month post-intervention (p=0.037).	Low
Shippey et al. 201129USA	Randomized controlled trial	Medical students (58)	Basic surgical skills (subcuticular suturing)	1. Self-directed practice with instructional video2. Practice with expert supervision3. Independent practice	TP: immediately before and after intervention and then 1-week post-intervention OM: Modified OSATS	The video-assisted (1) and expert-supervised group (2) had a significant increase from pre- to post-test measures, with a mean score increase of 3.59 (p=0.005) and 3.06 (p=0.002) respectively.When examining change from pre-test to retention performance, only the video-assisted group (1) showed a significant positive change (p=0.001) with a mean score increase of 3.67.	Low
Pilieci et al. 201825Canada	Randomized controlled trial	Medical students (129)	Basic surgical skills (sterile surgical technique)	1. Control (nurse educator-led skill demonstration)2. Video education	TP: immediately after interventionOM: 30-item multiple-choice test	The video-based education group had significantly superior scores compared to the control group (88% ± 1% versus 72% ± 1%; p<0.0001).	Low

Participants

The 11 studies included a total of 715 participants. After reviewing experimental groups, 603 were included in our qualitative analysis. 32 participants were lost to follow up. Majority of participants were medical students (n=585) and the remainder interns (n=18). The types of basic surgical skills taught include suturing, knot tying, laceration repair, skin flap and sterile surgical technique. If the study design allowed time for independent practice, appropriate materials were provided to the participants.

Surgical education methods

In this review, conventional educational methods for all studies comprised face-to-face live teaching, with one study using text-based education. The interventional educational methods included live web-based video education, live web-based video education with interaction, self-directed video-based education, , , , web-based video education in addition to standard conventional teaching , and access to web-based video education after standard video teaching. , There were four studies with more than two experimental groups. In these cases, the control group was determined as the group receiving expert (concurrent) feedback during face-to-face teaching or practice sessions, as this would be considered conventional teaching. , For the intervention group, interactive virtual classroom training was chosen over non-interactive computer-based learning and practice with instructional video was chosen over independent practice.

Outcome measures

The primary study outcome for all included studies was post-intervention surgical skill proficiency (alone or in comparison to baseline proficiency). Six studies used a standardized objective structured assessment of technical skills (OSATS) scale, three studies used a modified OSATS scale and two studies used a non-standardized skills checklist. Secondary study outcomes included participant confidence, satisfaction and perceptions of video learning.

Quality of studies and risk of bias

The overall risk of bias for the included studies was low. A summary of bias assessment using Cochrane RoB Tool 2 is provided in (Supplementary Table). All studies used adequate randomization processes. Participants and those delivering the intervention were not blinded due to the required study design, however, the outcome assessors (those assessing skill competency) were blinded. Four of the studies that were randomized controlled trials did not include adequate baseline characteristics for the participants but were still considered low risk for bias as per Cochrane guidelines. There was no outcome data missing across all studies and loss to follow-up was less than 10% of participants. All studies had interventions, primary and secondary outcomes initially stated in the study design and all outcomes had results recorded. In conclusion, the overall quality of evidence from the included studies is high.

Efficacy of online video-based education compared to conventional education

Seven of the studies found no significant difference between the control and video-based education groups in the teaching of basic surgical skills when assessed immediately post-intervention. , , , , , , Nathan et al. found interaction between student and teacher to be advantageous, as they showed conventional face-to-face and virtual classroom teaching were equal in efficacy, but both superior to non-interactive computer-based learning. This however is not found in the other studies, as five of the studies showed no significant difference between the control and video-based education groups that utilized self-directed video-based education. , , , , Interestingly, Shippey et al. showed only the group receiving video-based education maintained a significant increase in pre-test to retention performance (measured 1-week post-intervention). However Co et al., Chien et al., Xeroulis et al. and Bochenska et al. all measured skill competency at equivalent or longer timepoints and found no difference between the control and video education groups. Three studies showed video-based education was superior to conventional education in teaching basic surgical skills. , , Autry et al. noted 80% of the control group practiced knot tying less than two times during the study period, whilst 80% of the intervention group practiced three or more times. Bochenska et al. also showed practice correlated to a higher knot tying score, but only in the video education group. de Sena et al. is the only study included in this review to show a superiority of self-directed video-based learning to traditional text-based education. One study favored conventional education over self-directed video-based education in teaching basic surgical skills.

Student and trainee confidence and satisfaction with online video-based education

Participant satisfaction was assessed as a secondary outcome in six studies. , , , , , Nathan et al. showed a significant improvement in confidence (p<0.001) in all groups. Lwin et al. showed 76% wanted to see interactive video-based education in the future, 96% expressed the modules should be expanded to other skills and all the participants agreed interactive video-based education should be included in the curriculum. Autry et al. found 100% of participants receiving video education enjoyed it and found it equivalent to face-to-face. Bochenska et al. report high rates of satisfaction in both video and non-video groups and de Sena et al. report 100% of participants share the opinion that video-based education was the optimal method of teaching when compared to text-based learning. An extensive end of study survey from Pilieci et al. shows video education was preferred for convenience (82%), accessibility (94%), efficiency (75%) and utility to review sterile surgical technique (80%). In contrast, traditional skill demonstration was preferred for ease of completion (40%), retainment of knowledge (64%) and feeling prepared (51%).

Overall effect

Meta-analysis using a random-effects model was completed using pooled data from eight studies totaling 380 participants (see Fig. 2 ). The studies by Co et al. and Autry et al. were not included as they were observational studies. The study by Pilieci et al. was not included as primary outcome measurement varied significantly. There is no overall effect favoring either control (conventional education) or intervention (video-based education), with a standardized mean difference of -0.02 (95% CI: -0.34, 0.30; p = 0.90). The I2 value of 58% suggests moderate heterogeneity between studies. Upon assessment of the funnel plot (see Supplementary Fig. 1) there is no evidence of publication bias.

FIGURE 2

overall effect estimate for conventional versus video-based education with forest plot and heterogeneity assessment.

DISCUSSION

Online video-based education has come to the forefront of medical education due to sudden limitations on conventional face-to-face teaching due to the COVID-19 pandemic. Limited evidence exists for its utility in teaching practical surgical skills to students and trainees. This systematic review with meta-analysis investigates the efficacy of online video-based education in teaching basic surgical skills. 11 studies were included for qualitative analysis and eight for meta-analysis, with no overall effect favouring either online video-based or conventional education. To the best of our knowledge, this is the first systematic review solely assessing the efficacy of online video-based teaching of basic surgical skills and first meta-analysis completed in the broader topic of assessing efficacy of online surgical curriculums.

Seven studies found no difference in skill competency between conventional and video-based learning. , , , , , , This is consistent with a recent study looking at teaching surgical skills to junior surgical trainees, where competency was comparable across groups. Three studies favored video-based education over face-to-face and text-based methods. , , This is consistent with the literature, as Wu et al. previously found video education to be superior to text-based education and Summers et al. suggest it is superior to face-to-face teaching. It is possible the video education utilized in these studies had schematics, imaging and audio of high quality, shown to be beneficial for training doctors. The type of video education is also important, as it can range from pre-recorded lectures, live lectures, interactive workshops and live demonstrations. Kumins et al. suggest video-based learning is effective if structured in small and easily digestible instructions. Utilizing video education as the core or an adjunct of a surgical curriculum remains open to discussion. Evidence exists to support an improvement of knowledge and skill level for both.

One study by Tejos et al. favored conventional education in teaching basic surgical skills. A limitation of this study is that it has potential for discrepancy in the duration of teaching and practice hours between the groups receiving conventional or video education, as there was no methodology in place to confirm the study schedule and practice hours of the self-directed video group. Furthermore, the group receiving conventional face-to-face education had six 120-minute-long teaching sessions. This could show an advantageous association with conventional teaching not due to the mode of delivery but instead due to engagement with a more rigorous level of teaching. This limitation also exists in the study by Autry et al., where the video education group received teaching from a University of California faculty member and the conventional teaching group from a Ugandan faculty member. Outsourcing education from the United States has the potential to act as a confounding variable affecting the quality of education delivered.

Student and trainee satisfaction is high with video-based learning. It holds several unique advantages, most notably repeated accessibility, efficiency and an individualized learning pace. However, some drawbacks may exist, such as retainment of knowledge, preparedness for clinical work and social isolation with worsened mental health. The latter is hypothesized to mostly be due to the sudden shift to online education due to the COVID-19 pandemic, however this point is not to be overlooked. It is also believed more evidence showing online learning is as efficacious as in-person learning could help student anxiety and mental health. These drawbacks could be addressed by optimizing the type of video-based teaching, however, more evidence is required on what is the optimal type. Consideration of student mental health wellbeing and connectedness with peers is important. It would also help if video-based learning was used as an adjunct to conventional teaching in future curricula.

The role of feedback and student interaction in the various teaching methods is not agreed upon in the literature and would benefit from further research. Janda et al. and Nousiainen et al. suggest interactive and non-interactive video material show no difference in the development and retention of skills. Nathan et al. suggest interaction is crucial in achieving equal efficacy to face-to-face learning and Tejos et al. suggest video education without interaction is not sufficient in teaching basic surgical skills. It has also been suggested that structured peer-feedback is equivalent to expert feedback in the teaching of basic surgical skills, enabling students to learn and practice independently from the faculty. For online education where engagement is intrinsically less than face-to-face, keeping the instructor-to-student/trainee ratio small is important and better for participant satisfaction. This however was not a main point of feedback from students who completed the video learning, where working at their own pace and enabling re-winding and re-watching were the greatest benefits.

Practice of the learned skills is required for consolidation of practical skills, where all studies included in this review (if the study design included time to practice) provided students with their own materials. Bochenska et al. showed increased practice was significantly correlated with post-intervention knot tying scores. This may suggest online education is best utilized in a domain and teaching a skill that enables students to engage in self-directed practice and/or practice at home.

This review and meta-analysis has several limitations due to the moderate heterogeneity of the included studies. Potential sources of heterogeneity include variations in primary outcome measurement (different scales or point systems for OSATS), small number of participants and type of video education given. Future research should focus on characterizing the most effective type of video-based education, e.g., randomized controlled trials comparing pre-recorded lectures versus interactive workshops. In addition, research should be done to relate the use of online video-based teaching to long-term surgical skill. This would require larger, blinded prospective trials with long-term follow-up.

CONCLUSION

These results have implications for surgical curricula in a post-COVID-19 era, and also for rural and less socioeconomically developed areas, where students and trainees can attend remote skills workshops via online platforms thus creating greater opportunities. This systematic review with meta-analysis suggests online video-based education and conventional education do not differ in efficacy in teaching basic surgical skills to students and surgical trainees. This is an important conclusion, as comparable outcomes between teaching methods supports the continuing use of video-based teaching tools in future surgical curricula. This will have economic benefits for educational institutions and reduce load on faculty and teaching hospitals.

Declaration of Competing Interest

none

ACKNOWLEDGMENTS

We would like to acknowledge Rajneesh Kaur (University of Sydney, Sydney, Australia) for statistical advice.