COVID-19 as a disruptor: innovation and value in a national virtual fracture conference.

OBJECTIVES: The aim of this study was to determine the educational value of a national virtual fracture conference implemented during the COVID-19 disruption of resident education.
DESIGN: Survey study.
SETTING: National virtual conference administered by the Orthopaedic Trauma Association. PARTICIPANTS: Attendees of virtual fracture conference. INTERVENTION: Participation at a national virtual fracture conference. MAIN OUTCOME MEASURE: Surveys of perception of quality and value of virtual conferences relative to in-person conferences.
RESULTS: Ninety-six percent of participants rated the virtual fracture conference as similar or improved educational quality relative to conventional in-person fracture conference. Participants also felt they learned as much (35%) or more (57%) at each virtual fracture conference compared to the amount learned in-person. The quality of interpersonal interactions at both the resident-faculty level and faculty-faculty level was also perceived to be overall superior to those at participants' own institutions. Learners felt they were more likely to engage the primary literature as well. Overall, 100% of participants were likely to recommend virtual conference to their colleagues and 100% recommended continuing this conference even after COVID-19 issues resolve.
CONCLUSIONS: We found that learners find significant educational value in a national virtual fracture conference compared to in-person fracture conferences at their own institution. COVID-19 has proven to be a disruptor not only in health care but in medical education as well, accelerating our adoption of innovative and novel resident didactics. LEVEL OF EVIDENCE: Therapeutic Level III.

Introduction

As a result of the COVID-19 pandemic, many residency and fellowship programs implemented unique and creative solutions to continue required graduate medical education[ while adhering to social distancing guidelines. These strategies include utilization of Google Hangouts for daily surgical lessons in anatomy and clinical practice,[ “platooning” residents into “active duty” and “working remotely” factions that allow focus on service and education respectively,[ attending-led discussions of high-quality surgical videos,[ and weekly morning, afternoon, and evening educational teleconferences.[ It is still unclear, however, how these virtual educational initiatives compare to traditional educational activities, how effective they are in accomplishing their educational goals, and how satisfied residents and faculty have been with these efforts.

At our institution daily fracture conference and weekly subspecialty educational conferences that traditionally met in person were disrupted by social distancing restrictions in mid-March 2020. To continue educating our residents, we implemented both morning and evening teaching via a screen-sharing software platform on personal computers and mobile phones (BlueJeans or Zoom). The evening fracture conference at our institution was met with active participation and the resident response was enthusiastic. For this reason, we expanded the conference format with the support of the national subspecialty association (Orthopaedic Trauma Association). The conference was organized as a case-based imaging presentation similar to traditional fracture conferences, supplemented by “chalkboard” discussions led by a rotating group of orthopaedic trauma faculty across the country. Virtual real-time polling questions were incorporated to assess learner knowledge during the case and to create discussion. Though we felt that the national conference was being well received we were unsure if this was simply a case of confirmation bias due to high attendance and enthusiasm of certain faculty and residents.

For this reason, we conducted a national survey to gauge resident and faculty perceptions of virtual fracture conferences. As restrictions around COVID-19 ease, the data are important as we consider whether we should continue these virtual conferences indefinitely. It is possible that the disruption this pandemic provided has accelerated our adoption of valuable innovations in medical education.

Methods

Orthopaedic traumatologists led case-based discussions on 3 rotating topics: fractures of the upper extremity, lower extremity, and pelvis/acetabulum. The OTA chose Zoom web-based software to host video conferences. Zoom has the benefit of a screen-share function, virtual whiteboard, polling, and chatroom. Traumatologists from across the country volunteered to present their cases, and each week a new traumatologist presented a case. This involved presenting radiographs and/or advanced imaging, leading the discussion, and polling the audience with radio-button multiple choice questions that were prepared in advance. Polling questions were often designed to be controversial or to have more than one “right” answer and were meant to foster discussion. Responses were recorded anonymously and then presented on screen. Other faculty volunteered to manage the chatroom in which participants could type questions about the case and faculty responded in real time. Additional faculty were present to engage in discussion with the case presenter. Questions or topics in the chatroom that garnered significant discussion were often brought to the presenter for further discussion. Approximately 45 minutes were allotted for each conference time.

After IRB approval was obtained at our institution, a survey was created and distributed electronically via email using a commercially available online survey platform (SurveyMonkey). Surveys were voluntary and anonymous. Our main goals of the survey were to determine conference participants’ perceptions in 4 broad categories: technical aspects of orthopaedic trauma care, virtual versus in-person educational value, educational value of faculty–resident and faculty–faculty interactions and discussions, and likelihood of recommending continuation of these conferences even as COVID-19 limitations resolve.

Results

There were 245 responses among the 925 participants who registered for nightly national fracture conference. The response rate was 26%. Survey participants included medical students (6%), fellows (5%), and attending surgeons in practice (12%). The majority of participants were residents (77%). The most common orthopaedic subspecialty interest among participants was trauma (35%), but this was not the majority. Residents with an interest or fellowship placement in hand surgery (10%), arthroplasty (10%), sports medicine (8%), and spine surgery (5%) also participated. Many of the participating residents were yet undecided on subspecialty (17%).

Survey results indicated that the conferences were educational regarding technical aspects of orthopaedic surgery. After 3 of these weekly conferences, 37% of respondents reported feeling more comfortable evaluating radiographs, 54% felt more comfortable with preoperative planning for fracture surgery, and 75% of respondents felt more knowledgeable about complications in fracture management.

The majority of participants noted an improvement in quality and value of virtual conferences relative to in-person conferences (Fig. 1). On a Likert scale, 42% of participants rated the virtual conference as similar quality to in-person conferences at their own institution while the majority (54%) of participants reported that the virtual conference was better than in-person conference. In regard to value, 35% of participants felt that they learned as much at each virtual fracture conference as they did at each pre-COVID educational conference. The majority of participants (57%) felt that each virtual conference was more valuable than their pre-COVID conference.

Figure 1

Overall 96% of participants felt that virtual conference compared similarly or increased the quality of their education compared with their institutions in-person conference. Ninety-two percent felt that they were learning as much or more per conference relative to their pre-COVID in-person conferences.

To our surprise, the quality of interpersonal interactions did not seem to be diminished by virtual conference, and the majority of participants actually noted an improvement (Fig. 2). On a Likert scale 44% of participants noted that educational resident–faculty interactions were of similar quality to their regular in-person conference while 46% noted moderate or significant improvement in the quality of these interactions. In regard to faculty–faculty interactions, 23% replied that the educational quality of these virtual interactions was similar to their regular conference while 74% noted the educational quality of these interactions was moderately or significantly improved.

Figure 2

Overall 90% of participants felt that the faculty–resident interactions they experienced were equal to or superior in value to that which they experienced at their in-person conferences. Ninety-seven percent felt that the faculty–faculty interactions were equal to or superior in value to that which they experienced at their in-person conferences.

We were encouraged to find that relative to in-person lectures, 59% of respondents felt more compelled to engage the primary literature after these web-based virtual conferences. The overwhelming majority (88%) responded that participation in the virtual fracture conference improved their overall educational experience; 47% noted moderate improvement and 41% noted significant improvement (Fig. 3). Additionally, 100% of participants were likely to recommend this virtual conference to their colleagues, with 100% of participants also recommending continuing this conference even after the COVID-19 issues resolve.

Figure 3

Overall 98% of participants rated that virtual conference was equal to or an improvement on their educational experience with 88% feeling it was an improvement. One hundred percent of participants were likely to recommend this virtual fracture conference to residents at their program.

Discussion

We found that the response to the national virtual fracture conference was overwhelmingly positive. Although other surgical subspecialties have also instituted national didactic curricula,[ to our knowledge this is the first study to report resident and faculty perceptions of a national virtual conference. As a result of the positive response, the OTA intends to continue the national conference on a weekly basis. This pandemic disruption has substituted virtual didactics for conventional conferences, and these positive survey results encourage us to continue this valuable innovation in medical education.

The idea of “going virtual” is becoming increasingly commonplace in medicine as well as business enterprises.[ Web-based learning, testing, and group discussions were an emerging entity even prior to COVID-19.[ It is well documented that the educational resources utilized by orthopaedic residents are becoming increasingly web-based as well. In a national survey it was found that 99.5% of orthopaedic residents stated that some form of online or web-based educational resource was their most commonly used study tool.[ Clesham et al[ demonstrated that a novel electronic journal club utilizing the WhatsApp group chat system improved residents’ critical appraisal skills of the literature while providing adequate time and flexibility to reflect on the questions they encountered from attendings in the chatroom. These reports are consistent with our findings of increased resident satisfaction with the national virtual conference as well as the encouraging results of increased utilization of the primary literature when referenced in the chatroom.

Observations on adult education have shown that case-based discussions are integral to problem-based learning,[ and this national virtual fracture conference is modeled as such. This style of pedagogy, wherein learners use “triggers” from a case presentation to identify their own knowledge deficits, derives educational value via subsequent individual self-study and returning of information to their peers.[ In the pre-COVID educational setting, many training programs used regular in-person fracture conferences in which attending or senior residents helped junior residents identify knowledge deficits and follow-up until knowledge was consolidated. With the use of anonymous self-assessment questions during virtual fracture conference as well as the subsequent dissemination of hyperlinks to primary literature in the chatroom function, attending moderators are able to recreate and even improve upon this model of learning. Some participants have anecdotally reported they are more likely to read this primary literature given that it is readily accessible on their desktop. Although in-person fracture conferences commonly reference these “landmark studies,” virtual fracture conferences allow the participant to access the literature in real time. Our data suggest that participants perceive significant educational value from this model.

Stambough et al[ astutely noted that sharing lectures and conferences among various fellowship and residency programs may advance us out of educational “silos” as trainees gain exposure to variations in practice and education across the country rather than just from their home institution. The lack of a well-defined national orthopaedic curriculum can often result in many residents obtaining a relative lack of basic orthopaedic knowledge in favor of experience with highly specialized services depending on which program they match with.[ A residency directors’ peer forum previously recommended the publishing of a reading list on the internet that residents would be required to read to help ensure residents across the country obtain an equal access to a core of orthopaedic knowledge.[ It has been noted that this general evolution in electronic learning may cause an unprecedented increase in the quality and consistency of the educational potential of training programs as many more trainees have access to international experts.[ Our data support this as 74% of participants noted that the educational quality of faculty–faculty interactions in the virtual national fracture conference was moderately or significantly improved relative to their home institution fracture conference.

With the implementation of any novel system there are always challenges to overcome and limitations which are identified as the system evolves. Orthopaedic trauma is both visual and spatial and involves the presentation and interpretation of radiographs as well as advanced imaging. We are fortunate in the ubiquity of the technology to present multiple images in real time as this is not only necessary for the discussion of orthopaedic surgery cases, but it also creates a more engaging presentation. It is possible that our success with this may not apply to other medical and surgical subspecialties that do not rely as heavily on imaging studies. Additionally, our response rate was only 26%. The results may be biased toward individuals who found the conferences useful and were thus more likely to respond to our survey request. We cannot rule out the potential positive effect this may have had on our overall results.

We also cannot rule out unknown negative effects that a large audience might have on learners or the possibility of some Hawthorne effect. The experience of large-format online didactics in emergency medicine may suggest that some residents feel reserved in a larger audience—even online—due to the challenges of establishing “psychological safety” which some residents require in order to feel empowered to ask questions and engage in discussion.[ However, during curriculum development it was felt that the virtual chatroom would address the educational goal of minimizing “embarrassment and threat” which have been shown to diminish learning”.[ The chatroom can be anonymous if the learner desires, and the participants often do not know the faculty personally which may also decrease anxiety. This is useful as anonymity has purported benefits for learning, participation and student comfort. It has been shown that learners are more likely to participate and engage with asked questions when they are anonymous.[ A similar method utilizing “clickers” to answer a question was found to increase participants cognitive engagement and in systems where participants can conceal their identity, learners were more willing to engage in discussions, and ultimately learn more as a result.[

It remains to be seen if interest in our national conference will remain as COVID-19 recedes, elective surgery resumes, and demanding clinical duties resurface. Unfortunately, we cannot predict either the future of COVID-19 or how learners will respond in a post-COVID learning environment. A future direction may involve active trainee involvement. For example, dermatologic surgeons facing similar educational challenges have requested fellows-in-training to create PowerPoint presentations with an audio overlay for incorporation into an online database.[ It may be that even as real-time attendance potentially decreases, we may begin recording our case presentations’ and incorporating them into a trauma database, available for learners to access at their own convenience.

The COVID-19 disruption has accelerated our adoption of clinical innovations that have been years in the making. For example, rapid telehealth expansion has met a positive reception.[ Evolution does not always occur in a steady or gradual manner but is more aptly characterized by eras of dramatic leaps and advances in times of unprecedented stressors. These innovations are reshaping the realm of education as well. In the domain of orthopaedics this has led to the successful implementation of a weekly, national fracture conference which residents feel is more educational than the conventional in-person conferences that had dominated the educational landscape for decades prior. As we move forward into a post-COVID educational era, virtual conference may remain a mainstay in the education of surgeons in-training.