Learner Preferences and Perceptions of Virtual Hand Surgery Education During the COVID-19 Pandemic.

PURPOSE: In response to the COVID-19 pandemic, many postgraduate medical education lectures and conferences have been moved to a virtual platform. Questions remain regarding the effectiveness of virtual education, what types of educational offerings can be transitioned to a virtual format, and what types of curricula should still take place in person.
METHODS: This study surveyed trainees from the United States who participated in a single institution's hand surgery virtual flipped classroom curriculum of 6 week-long modules. Demographics, premodule and postmodule achieved levels of learning based on Bloom's taxonomy, technology usage, and preferences were surveyed.
RESULTS: Of the 65 participants, 41 (63.1%) responded to the survey. Trainees included hand surgery fellows (27/41 [65.9%]), orthopedic surgery residents (11/41[26.8%]), and plastic surgery residents (3/41 [7.3%]). On average, most trainees read, viewed, and participated in more than half of the articles (28/41 [68.3%]), electronic videos (31/41 [75.6%]), and conferences (35/41 [85.4%]) per week. The median level of achieved learning increased from "I can apply" to "I can analyze" for all modules. Self-directed learning was preferred for basic facts and knowledge (26/41 [63.4%]) and faculty-directed learning was preferred to review and practice advanced concepts (34/41 [82.9%]). The participants perceived benefits of the virtual curriculum to include increased scheduling flexibility (8/41[19.5%]), expert opinions (7/41 [17.1%]), and diversity of educational formats (3/41 [7.3%]). The perceived drawbacks included decreased interaction (8/41 [19.5%]), technical difficulties (6/41 [14.6%]), excessive detail (3/41 [7.3%]), and single-institution bias (2/41 [4.9%]).
CONCLUSIONS: The flipped virtual classroom model is an effective and preferred method of instruction for trainees. Trainees achieved a higher level of learning following the completion of each week-long module. Considerations for the implementation of a virtual curriculum include content quality, quantity, and reducing single-institution bias. CLINICAL RELEVANCE: The implementation of virtual learning can enhance hand surgery education for the modern learner.

Virtual surgical education has increasingly been used to supplement traditional education methods for both residents and medical students. Virtual surgical education, including lectures, computer-based learning modules, virtual patient encounters, and surgical training modules, have demonstrated objective improvements in resident knowledge acquisition. Specifically in orthopedic surgery, residents who participated in virtual training modules on diagnostic shoulder arthroscopy and percutaneous pinning of pediatric supracondylar humeral fractures have demonstrated improved scores on multiple-choice examinations compared with those who prepared using standard learning methods, including reading textbooks and performing cadaveric dissections. , Additionally, 2 studies have demonstrated improved performance on a standardized clinical examination assessment for medical students following a lecture series and a computer-based module on orthosis fabrication and musculoskeletal examinations. ,

Even with increasing usage of virtual education, the transition has been gradual. No single event has led to as rapid a shift toward the delivery of virtual education and the replacement of in-person lectures and surgical conferences as the global COVID-19 pandemic. To prevent the spread of COVID-19, medical education lectures and conferences, critical components of surgical training, were forced to move to an online format in the spring of 2020.6, 7, 8, 9, 10, 11, 12 Questions remain regarding what types of educational offerings can be moved to a virtual format and what types of curricula should still take place in person, albeit in smaller groups. Furthermore, gaining a better understanding of technology usage can inform educators as to what types of support residencies and fellowships need to provide to their trainees and in what ways schedules can be redesigned to meet this new challenge.

In response to national, local, and institutional guidelines during the COVID-19 pandemic in which gatherings were restricted, our institution’s hand surgery fellowship developed a virtual, 6-week, flipped classroom curriculum. A flipped classroom is one in which learners spend individual time solidifying knowledge, later to be discussed or applied in an interactive format with other learners and educators. Our curriculum consisted of weekly assigned readings, asynchronous video lectures (prerecorded to be viewed any time), and optional synchronous (real-time) or asynchronous virtual conferences. It was distributed to any interested orthopedic residents, plastic surgery residents, and hand surgery fellows in the United States. The purpose of this study was to survey the participants of this curriculum on their usage, experience, perceived effectiveness, and preferences to better inform future decisions regarding curricular redesign and management.

Materials and Methods

This was a survey study of hand surgery fellows, orthopedic surgery residents, and plastic surgery residents who participated in a virtual hand surgery flipped classroom curriculum offered by a single hand surgery fellowship at a single academic medical center. This study was reviewed by our institutional review board and was deemed to be exempt from approval.

A survey was distributed in June 2020 via email to all eligible participants following the completion of the hand surgery virtual education program (Appendix E1, available online on the Journal’s website at www.jhandsurg.org). Participants were solicited through the Hand Fellowship Directors Association of the American Society for Surgery of the Hand (ASSH). As such, the target participants were hand surgery fellows and senior-level residents with an interest in hand surgery. The program consisted of 6 week-long modules with assigned readings, asynchronous webinars as part of the Hand-50 series through the ASSH, and optional synchronous or asynchronous daily virtual conferences, including didactic lectures (day 1), case-based discussions (days 2 and 4), surgical tips and tricks (day 3), and panel debates (day 5) revolving around the topic or module for the week. It was expected that the trainee read and viewed the assigned readings and watched the asynchronous ASSH webinars on their own (approximately 1.5–3 hours of time commitment per week) prior to participating in the daily virtual conferences. All virtual conferences (approximately 30–60 minutes in length) were broadcast live with Zoom (Zoom Video Communications, Inc) for synchronous learning or were recorded and uploaded onto a dedicated YouTube (YouTube, Alphabet Inc) account for asynchronous viewing. The inclusion criteria included trainees (fellows and residents) who were registered for the online hand surgery education modules. No participants were excluded. The study data were collected and managed using REDCap electronic data capture tools hosted at our institution. ,

The survey measured baseline demographics, including the year in training, training type (fellowship or orthopedic/plastic surgery residency), and type of training institution (academic, private practice with academic affiliation, or community-based program). To get a sense of the educational offerings at each institution prior to the COVID-19 pandemic, the use of assigned readings, webinars, didactic lectures, case discussions, and surgical skills laboratories/cadaver dissections were surveyed as faculty-led, trainee-led, or not used.

The average weekly usage of articles, webinars, and virtual conferences was reported on a scale from 1 to 5 (1, none; 2, less than half; 3, one-half; 4, more than half; and 5, all). The predominant technology (phone, tablet, laptop computer, or desktop computer) used by each participant to access these materials was also recorded. To better understand typical preconference preparation, which is integral to the flipped classroom model, the order in which the materials were read or viewed was assessed (ie, readings before conference or conference before readings).

The perceived effectiveness of the different conference formats (case-based discussion, didactic, surgical tips and tricks, and panel debate) was surveyed on a 5-point scale from very ineffective to very effective. Globally, the achieved level of learning was measured on a 6-point scale based on Bloom’s taxonomy of learning (1, I know; 2, I understand; 3, I can apply; 4, I can analyze; 5, I can evaluate; and 6, I can synthesize) for each of the different conference formats listed above (Fig. 1 ). Bloom’s taxonomy is a 6-level hierarchical model of learning organized in order of increasing complexity. In this framework, higher levels of learning can only be achieved following the mastery of the lower levels. Specific to the surgical trainee, this framework represents the steps to surgical mastery, starting with learning anatomy and other surgical knowledge, applying those facts in evaluating others’ work, and ultimately making individual decisions with limited support. To evaluate the level of learning achieved for each individual topic, trainees were asked to rate their perceived level of learning before and after each module.

Figure 1

Bloom’s taxonomy of learning.

Individual preferences (in-person, online synchronous, and online asynchronous) for the various conference formats were recorded. The preferred method (self-directed vs faculty-directed) for learning basic facts and learning advance concepts was also surveyed.

A free-text option for recording the strengths and weaknesses of the educational program was provided. The responses were categorized post hoc into 4 strength and 5 weakness categories and reported as counts and percentages. The strength categories included the following: expert information/opinion, schedule/timing flexibility, diversity of educational formats, and other. The weakness categories included the following: excessive detail/time commitment, technical difficulties, and other.

Categorical variables were summarized using counts and proportions. Because the assumptions for normality were not met, ordinal variables were approximated as nominal and summarized using medians and interquartile ranges.

Results

Demographics

Of the 65 trainees, 41 (63.1%) responded to the survey, the majority of whom were hand surgery fellows associated with academic institutions. Further demographic information is listed in Table 1 .

Table 1

Baseline Characteristics

Characteristics	Overall
	N = 41
Year in training
Fellow	27 (65.9%)
Resident PGY 5	2 (4.87%)
Resident PGY 4	1 (2.44%)
Resident PGY 3	6 (14.6%)
Resident PGY 2	5 (12.2%)
Type of training
Orthopedic residency	11 (26.8%)
Plastic surgery residency	3 (7.32%)
Hand surgery fellowship	27 (65.9%)
Institution
Academic/University	30 (73.2%)
Private practice with academic affiliation	10 (24.4%)
Community-based program	1 (2.44%)
Education method
Asynchronous	24 (58.5%)
Synchronous	12 (29.3%)
Both	5 (12.2%)

PGY, postgraduate year.

Prior educational offerings

Prior to the implementation of the virtual education curriculum, most participants stated that their programs did not use structured readings or webinars for education (Table 2 ). When readings and webinars were used, they were primarily directed or selected by faculty. Didactic lectures, case discussions, and surgical skills/cadaver dissections were primarily faculty-led (Table 2).

Table 2

Pre–COVID-19 Pandemic Educational Offerings at Home Institutions

	Overall	Program Type
		Academic	Private Practice	Community-Based
	N = 41	n = 30	n = 1	n = 10
Readings
Faculty-led	13 (31.7%)	10 (33.3%)	1 (100%)	2 (20.0 %)
Fellow-led/self-directed	6 (14.6%)	3 (10.0%)	0 (0%)	3 (30.0%)
Not used	22 (53.7%)	17 (56.7%)	0 (0%)	5 (50.0%)
Webinars∗
Faculty-led	12 (29.3%)	9 (30.0%)	0 (0%)	3 (30.0%)
Fellow-led/self-directed	1 (2.44%)	1 (3.33%)	0 (0%)	0 (0.0%)
Not used	28 (68.3%)	20 (66.7%)	1 (100%)	7 (70.0%)
Didactic lectures
Faculty-led	34 (82.9%)	26 (86.7%)	0 (0%)	8 (80.0%)
Fellow-led/self-directed	5 (12.2%)	2 (6.67%)	1 (100%)	2 (20.0%)
Not used	2 (4.87%)	2 (6.67%)	0 (0%)	0 (0.0%)
Case discussion
Faculty-led	25 (61.0%)	18 (60.0%)	0 (0%)	7 (70.0%)
Fellow-led/self-directed	15 (36.6%)	11 (36.7%)	1 (100%)	3 (30.0%)
Not used	1 (2.44%)	1 (3.33%)	0 (0%)	0 (0.0%)
Surgical skills/cadaver dissections
Faculty-led	24 (58.5%)	17 (56.7%)	0 (0%)	7 (70.0%)
Fellow-led/self-directed	9 (22.0%)	5 (16.7%)	1 (100%)	3 (30.0%)
Not used	8 (19.5%)	8 (26.7%)	0 (0%)	0 (0%)

ASSH, 50 videos.

Virtual educational usage

The majority of participants viewed the content primarily via asynchronous means (Table 1). The most used technology was a laptop computer (75.6%, 70.7%, and 78.0% for articles, webinars, and conferences, respectively). On average, per weekly module, 28 of 41 (68.3%) trainees read more than half of the articles, 31 of 41 (75.6%) trainees viewed more than half of the webinars, and 35 of 41 (85.4%) trainees participated in more than half of the virtual conferences. The majority of participants (27/41 [65.9%]) read or viewed the articles and webinars prior to attending conferences.

Perceived effectiveness

The perceived effectiveness of learning, based on Bloom’s taxonomy, was identified between conference formats (Table 3 ). The median response for the achieved level of learning was “I can apply” or “I can analyze” (Table 3). The median level of achieved learning increased from “I can apply” to “I can analyze” for all modules (Table 4 ).

Table 3

Perceived Virtual Curriculum Effectiveness

Format	N	Postlearning Bloom Taxonomy Level Achieved, Median (IQR)∗
Case-based discussion	40	4.0 (3.0–5.0)
Topic-based lecture	40	4.0 (2.0–5.0)
Surgical tips and tricks	40	3.5 (2.0–5.0)
Panel debate	40	4.5 (2.3–5.0)

IQR, interquartile range.

Levels: 1 (“I know”); 2 (“I understand”); 3 (“I can apply”); 4 (“I can analyze”); 5 (“I can evaluate”); and 6 (“I can synthesize”).

Table 4

Before and After Self-Reported Achieved Learning Per Week-Long Module

Module	N	Premodule Bloom Taxonomy Level,Median (IQR)∗	Postmodule Bloom Taxonomy Level,Median (IQR)∗	Median Difference [95% CI]†
Radial nerve palsy	31	3.0 (2.0–5.0)	4.0 (3.0–5.0)	1.0 [0.0–1.5]
Fellow	19	4.0 (2.0–4.0)	5.0 (3.0–6.0)	1.0 [0.5–2.0]
Resident PGY 5	2	2.5 (2.0–3.0)	3.0 (2.0–4.0)	0.5
Resident PGY 3	5	2.0 (1.5–3.5)	3.0 (3.0–3.0)	1.0 [0.5–1.5]
Resident PGY 2	5	5.0 (2.0–5.5)	3.0 (2.5–3.5)	−1.0 [−3.0 to 2.0]
Ulnar-sided wrist pain	38	3.0 (2.0–4.3)	4.0 (3.0–5.0)	0.5 [0.0–1.5]
Fellow	26	4.0 (2.0–4.0)	5.0 (2.0–6.0)	1.0 [0.0–2.0]
Resident PGY 5	1	3.0 (3.0–3.0)	4.0 (4.0–4.0)	1.0
Resident PGY 4	1	1.0 (1.0–1.0)	3.0 (3.0–3.0)	2.0
Resident PGY 3	5	2.0 (1.5–4.0)	3.0 (3.0–4.5)	1.0 [−1.0 to 3.0]
Resident PGY 2	5	5.0 (2.5–5.5)	3.0 (2.5–4.5)	−1.0 [−3.0 to 3.0]
Brachial plexus injury	36	3 (2.0–4.0)	4.0 (3.0–5.0)	1.0 [0.0–1.5]
Fellow	24	3.5 (2.0–4.0)	4.0 (3.0–5.0)	1.0 [0.0–2.0]
Resident PGY 5	1	4.0 (4.0–4.0)	5.0 (5.0–5.0)	1.0
Resident PGY 4	1	1.0 (1.0–1.0)	5.0 (5.0–5.0)	4.0
Resident PGY 3	5	1.0 (1.0–1.5)	3.0 (2.0–3.5)	1.0 [1.0–2.0]
Resident PGY 2	5	5.0 (2.5–5.5)	3.0 (2.5–4.0)	−1.0 [−3.0 to 2.0]
Finger arthritis	36	3 (3.0–4.0)	4 (3.0–5.0)	1.0 [0.0–1.5]
Fellow	25	3 (2.5–4.0)	5 (2.0–5.5)	1.0 [0.0–2.0]
Resident PGY 5	1	3 (3.0–3.0)	4 (4.0–4.0)	1.0
Resident PGY 4	1	1 (1.0–1.0)	4 (4.0–4.0)	3.0
Resident PGY 3	4	3 (2.3–4.5)	4 (2.5–4.8)	1.0 [−1.0 to 2.0]
Resident PGY 2	5	5 (3.5–5.5)	3 (3.0–4.5)	−1.0 [−3.0 to 2.0]
Failed carpal tunnel	36	3.0 (2.0–5.0)	4 (3.0–5.8)	1.0 [0.0–1.5]
Fellow	23	4.0 (2.0–5.0)	5.0 (2.0–6.0)	1.0 [0.0–1.5]
Resident PGY 5	1	2.0 (2.0–2.0)	3.0 (3.0–3.0)	1.0
Resident PGY 4	1	1.0 (1.0–1.0)	4.0 (4.0–4.0)	3.0
Resident PGY 3	6	2.0 (1.0–3.8)	3.0 (2.8–4.3)	1.0 [−1.5 to 2.5]
Resident PGY 2	5	5.0 (3.0–5.5)	3.0 (3.0–5.0)	−1.0 [−3.0 to 3.0]
PIP fracture dislocation	36	3.0 (2.25–4.0)	4.0 (2.25–5.0)	1.0 [0.0–1.5]
Fellow	23	3.0 (3.0–4.0)	5.0 (2.0–5.0)	1.0 [0.0–1.5]
Resident PGY 5	2	2.5 (2.0–3.0)	2.5 (2.0–3.0)	0.0
Resident PGY 4	1	1.0 (1.0–1.0)	4.0 (4.0–4.0)	3.0
Resident PGY 3	5	2.0 (1.5–4.0)	4.0 (2.5–4.5)	2.0 [−3.0 to 3.0]
Resident PGY 2	5	5.0 (3.0–5.5)	3.0 (2.5–4.5)	−1.0 [−3.0 to 2.0]

CI, confidence interval; IQR, interquartile range; PIP, proximal interphalangeal; PGY, postgraduate year.

Levels: 1 (“I know”); 2 (“I understand”); 3 (“I can apply”); 4 (“I can analyze”); 5 (“I can evaluate”); and 6 (“I can synthesize”).

CI is omitted where not defined.

Preferences

The preferred method for learning basic facts and knowledge was self-directed learning (26/41 [63.4%]) versus faculty-directed learning (15/41 [36.6%]). To learn and practice advanced concepts, 34 of 41 (82.9%) participants preferred to have faculty-directed teaching. Stratified by virtual conference type, most participants preferred the online educational offerings to in-person learning for case-based discussions, didactic lecture, surgical tips/tricks, and panel debates (Table 5 ). Participants also favored synchronous over asynchronous online delivery for case-based discussions, surgical tips/tricks, and panel debates (Table 5). Trainees indicated that there was no preference between synchronous versus asynchronous delivery of didactic lectures (Table 5).

Table 5

Conference Delivery Preferences

Delivery Method	Case-Based Discussions	Didactic Lecture	Surgical Tips/Tricks	Panel Debate
	N = 41	N = 41	N = 41	N = 41
In person	12 (29.3%)	7 (17.1%)	13 (31.7%)	10 (24.4%)
Online, synchronous	21 (51.2%)	17 (41.5%)	17 (41.5%)	20 (48.7%)
Online, asynchronous	8 (19.5%)	17 (41.5%)	11 (26.8%)	11 (26.8%)

Strengths and weaknesses

Free-text responses for strengths of the virtual education program included increased flexibility with scheduling (8 respondents), expert opinions (7 respondents), diversity of educational formats (3 respondents), and “other” (4 respondents). Free-text responses for weaknesses included decreased interaction (8 respondents), technical difficulties (6 respondents), excessive detail/time commitment (3 respondents), and single-institution bias (2 respondents).

Discussion

This study provides an evaluation of the perceived effectiveness and preferences of residents and hand surgery fellows on their virtual learning experience with a hand surgery flipped classroom curriculum. Most participants in the current study were hand surgery fellows at academic institutions (Table 1). This was unsurprising, given that the curriculum was distributed via the Hand Fellowship Directors Association of the ASSH, and the majority of hand surgery fellowships are associated with academic institutions.

The scholastic structure of the flipped classroom educational model made it an ideal method to be implemented during the COVID-19 pandemic. The flipped classroom is a real-world extension of Bloom’s taxonomy, a 6-level hierarchy of cognitive learning ranging from knowledge to synthesis, wherein the mastery of a subject requires a nonlinear progression through the hierarchy. In the flipped classroom model, learners spend time individually solidifying content, facts, and figures, and later, as a group, share their understanding with experts in the field through a variety of means. This is equivalent to initially achieving Bloom’s first 2 levels of knowledge and comprehension and later developing skills in the higher levels of application, analysis, synthesis, and evaluation. The trainees in the current study’s cohort favored this learning structure, with 63.4% preferring self-directed learning for acquiring basic facts and knowledge and 82.9% preferring faculty-directed teaching to learn and practice advanced topics. Further support of this preference is that 65.9% of trainees reported that they read or viewed the articles and webinars prior to attending the conferences.

The participants achieved a high level of learning after participation in each week-long module. The median self-reported achieved level of learning for each module increased from 3 (“I can apply”) to 4 (“I can analyze”). Intuitively, when stratified by the year in training, fellows achieved a higher level of self-reported achieved level of learning for the radial nerve palsy, finger arthritis, and failed carpal tunnel modules (Table 4) that was not evident in residents. It is likely that this increase in the achieved level of learning was demonstrated in fellows because the modules were geared toward fellows and senior residents’ level of knowledge and skill. In particular, the level of trainee has been shown to influence the preferred type and format of online surgical procedures videos, where the senior learner may prefer a detailed explanation on surgical decision-making while junior learners may favor more background information and clear, step-by-step surgical instructions. Given that a large component of surgical education is in-person, skills-based training, the highest levels of learning can only reasonably be achieved working with faculty in the clinic and operating room in conjunction with educational curricula. “Tele-mentoring” and virtual surgery, wherein trainees are mentored in a structured, virtual environment, might be used in the future to harness the differing expertise from surgeons around the world. ,

The flipped classroom has beneficial scheduling implications for trainees and programs. Some programs have adjusted their schedules by staggering access to common areas and creating on-service and off-service rotations, all with the goal of minimizing the in-person interaction of residents to decrease the risk of COVID-19 exposure and transmission. These varying learner schedules present a unique challenge for coordinating protected education time. Using a flipped classroom model, a large proportion of the learning is independent, and thus does not require any schedule coordination on the part of programs. Over 35% of respondents reported scheduling flexibility as a strength of this virtual curriculum. Given that this was a free-text response, this suggests that the increased flexibility of the flipped classroom model may be a solution to solving future educational scheduling conflicts.

There are 3 key considerations for the implementation of an effective flipped classroom curriculum. First, given the increased focus on the independent learning of facts and figures, there is a need for high-quality readings and asynchronous lectures. In the study by Luu et al, 4 otolaryngology residents evaluated 37 YouTube neck dissection videos using the LAP-VEGaS (LAParoscopic surgery Video Educational GuidelineS) grading scheme, a validated video assessment scale. They demonstrated inconsistent quality among the most popular videos and found no correlation between total view count and the number of likes/dislikes on the quality of the videos. The videos produced by otolaryngology-trained physicians had a significantly higher LAP-VEGaS score. Trainees also prefer expert involvement; 31.8% of our cohort reported this as a strength of our virtual curriculum. Second, given the plethora of available learning materials and the limited time of trainees with clinical and research responsibilities, curricula should be designed to meet clear objectives without overburdening trainees. In the current study, 15% of trainees reported a weakness of the curriculum as being too detailed or too much of a time commitment, and approximately 25% to 30% of our cohort participated in less than half of the course offerings. Additionally, participants favored online delivery of all conference formats, which could be due to increased convenience and decreased travel time between hospital sites for in-person education (Table 5). Lastly, 10% of our cohort reported a weakness of the program as being single-institution bias. Given institutional and regional practice differences, materials should be created across institutions to incorporate differing expert viewpoints. National specialty societies, such as the ASSH, are in a unique position to assist with the development of content. One possible solution is to source material from expert members of these organizations at differing institutions, such as the Hand-50 series webinars. The content could then be tagged as foundational (required) or optional. Trainees could choose to read or view optional content based on their interests, goals, and current clinical or research responsibilities. This would ensure that high-quality, multi-institutional, trainee-focused material is available to all residents and fellows.

Several limitations of this study warrant consideration. First, our cohort only represents a small percentage of trainees who participated in a single institution’s educational curriculum for hand surgery education. Although the invitation to participate in the online curriculum was sent to all hand surgery fellowship directors, who and how the invitation was distributed to the trainees within each respective program is unknown and may have resulted in a selection bias. Furthermore, we did not study surgical skills education, a major component of surgical training that is likely disproportionally affected by the COVID-19 pandemic, compared with in-person didactics and conferences. Another limitation is the possibility of inflated or deflated responses to questions regarding professional behaviors, such as reviewing the preconference resources prior to the virtual education event. Lastly, this survey was distributed following the completion of the 6-week course, not after each module; thus, our perceived data are likely influenced by recall bias.

The study provides guidance to program directors of hand surgery fellowships and orthopedic or plastic surgery residencies as to the preferred learning styles of the contemporary learner. With the potential for a prolonged need for alternative methods of education, programs must adapt to provide trainees with a robust virtual curriculum to meet the void created by decreased surgical case volumes while keeping the trainee safe and healthy.