Online Liver Imaging Course; Pivoting to Transform Radiology Education During the SARS-CoV-2 Pandemic.

PURPOSE: The SARS-CoV-2 pandemic has drastically disrupted radiology in-person education. The purpose of this study was to assess the implementation of a virtual teaching method using available technology and its role in the continuity of education of practicing radiologists and trainees during the pandemic.
METHODS: The authors created the Online Liver Imaging Course (OLIC) that comprised 28 online comprehensive lectures delivered in real-time and on-demand over six weeks. Radiologists and radiology trainees were asked to register to attend the live sessions. At the end of the course, we conducted a 46-question survey among registrants addressing their training level, perception of virtual conferencing, and evaluation of the course content.
RESULTS: One thousand four hundred and thirty four radiologists and trainees completed interest sign up forms before the start of the course with the first webinar having the highest number of live attendees (343 people). On average, there were 89 live participants per session and 750 YouTube views per recording (as of July 9, 2020). After the end of the course, 487 attendees from 37 countries responded to the postcourse survey for an overall response rate of (33%). Approximately (63%) of participants were practicing radiologists while (37%) were either fellows or residents and rarely medical students. The overwhelming majority (97%) found the OLIC webinar series to be beneficial. Essentially all attendees felt that the webinar sessions met (43%) or exceeded (57%) their expectations. When asked about their perception of virtual conferences after attending OLIC lectures, almost all attendees (99%) enjoyed the virtual conference with a majority (61%) of the respondents who enjoyed the virtual format more than in-person conferences, while (38%) enjoyed the webinar format but preferred in-person conferences. When asked about the willingness to attend virtual webinars in the future, (84%) said that they would attend future virtual conferences even if in-person conferences resume while (15%) were unsure.
CONCLUSION: The success of the OLIC, attributed to many factors, indicates that videoconferencing technology provides an inexpensive alternative to in-person radiology conferences. The positive responses to our postcourse survey suggest that virtual education will remain to stay. Educational institutions and scientific societies should foster such models.

Introduction

The SARS-CoV-2 pandemic has caused unprecedented disruptions to the health, social, and economic status of individuals at the local and global levels (1). The World Health Organization has reported more than 14 million cases and 593,000 deaths as of July 18, 2020 (2). In response, governments and public health authorities have enforced measures to slow transmission of the virus such as social distancing, air travel bans, limiting public gatherings, mask-wearing, and self-isolation (3). Medical schools have canceled many on-campus educational activities and shifted to remote learning models. To maintain the capacity of their workforce to care for hospitalized patients with COVID-19, hospitals have applied similar risk mitigation strategies, including temporarily halting elective procedures, a major source of revenue for hospitals, as well as enforcing social distancing within the workplace (4). For radiology, in many institutions, this has translated to limiting the number of trainees in reading rooms and lecture halls and potentially converting the readout to a virtual experience (4). Hundreds of scientific conferences, including many major radiology conferences, have been either canceled or switched to virtual models to abide by local public health authority recommendations. Additionally, financially strained institutions may experience a shortage of allocated funds to pay for these conference experiences (5). All projections of how and when this pandemic will end are speculative and dependent on the interplay of many factors (6). Therefore, the future of in-person educational activities remains uncertain.

Historically, adaptation to virtual conferences has been hampered by technological barriers and preference of in-person conferences (7). Teleconferencing technology has rapidly evolved over the past decade, and many reliable, advanced, and affordable platforms are now available (8). Given the technological nature of radiology, adaptation to a virtual environment may be easier than with other disciplines in medicine. In fact, there is precedence for virtual education in radiology. Remote conferencing has been implemented for radiology faculty and residents at the University of Arkansas over a two-year period and has shown that many participants valued the quality and convenience of this approach (7).

As a form of remote conferencing, webinars may reach a broader audience on both national and global levels with the participants attending sessions and interacting with the presenters in real-time. In response to the SARS-CoV-2COVID-19 pandemic, we launched the Online Liver Imaging Course (OLIC) with a series of webinars to address potential barriers in liver imaging education amongst those involved in radiology. In this paper, we share the structure and outcomes of OLIC as well as discuss the potential of this model to offer a reproducible and convenient educational solution for radiologists worldwide.

METHODS

Course design

The OLIC was created with the mission of providing free, high-quality education, accessible worldwide to anyone interested in the topic of liver imaging. Our target audience included radiology trainees (residents and fellows) and practicing radiologists. The course director designed a comprehensive course of 28 online webinar lectures delivered in real-time via the internet over six weeks.

The webinars' content was carefully designed to cover a comprehensive list of topics crucial for practicing radiologists interested in liver imaging. Key themes included Liver Imaging Reporting and Data System (LI-RADS) (9), liver imaging techniques, the correlation between radiological and pathological findings in the liver, and image-guided therapies for hepatocellular carcinoma. Each webinar included one topic, but some topics were discussed over several webinars. The course director invited 28 liver imaging experts to speak as part of the series. Speakers were chosen based on their expertise in liver imaging; this expertise was indicated by track records of scholarly peer reviewed publications and previous involvement in teaching these topics at national and international conferences. Each webinar lasted for 60–80 minutes, was delivered as a didactic presentation by the speaker, and was followed by a live question-and-answer session. Questions were either written in the chatbox or asked by the attendees directly using the audio tools with permission from the moderator. At the end of the course, participants received a survey by email to evaluate the effectiveness of the sessions and solicit feedback to make improvements. To facilitate easy and open access to our OLIC presentations, and in response to high numbers of registrants, all lectures were recorded and uploaded to an online video stream (YouTube powered by Google; San Bruno, California).

Registration

A webinar series interest form was created using an online survey tool (SurveyMonkey; San Mateo, California). The interest form asked potential participants for their contact information and degree of interest in an online liver imaging course. The OLIC was advertised by the course director and speakers through direct emails to personal contacts involved in radiology education nationally and internationally, and on social media platforms such as Facebook and Twitter.

Technology

The webinars were delivered in real-time to attendees via the GoToMeeting platform (LogMeIn, Inc.; Boston, Massachusetts). The course director purchased a business plan account to manage the webinar series. This platform was selected as it met the requirements of being secure and affordable, allowed speakers to share their screen, audio, and video; and allowed attendees to interact with the speaker via voice or a chat window. The business plan was selected as it allowed up to 250 attendees to join the webinar and provided advanced features such as annotation tools and cloud-based recording for later upload to YouTube.

An online marketing automation platform (MailChimp; Atlanta, Georgia) was used to send mass emails containing the webinar link to interested participants. The first webinar was held on April 20, 2020, and the course concluded on May 29, 2020.

Postcourse survey

A 46-question survey was designed and distributed using an online survey tool (SurveyMonkey; San Mateo, California). The questions were reviewed by the OLIC course director to verify the integrity and objectivity of the questions. Attention was made to ensure the survey was free of bias and contained language that was nonleading and easily interpretable by participants. The survey included questions regarding participants’ level of training in radiology, type of medical practice, number of attended live or recorded sessions, perception of the usefulness of virtual conferencing, evaluation of each session, and several more items (see Appendix 1 for a detailed list of questions). The survey was distributed via email to all webinar participants. All questions, except one asking about the country of practice, were multiple-choice with single choice options. The online survey engine's built-in analytics were used for data analysis and chart generation. The origin countries of practice provided in responses were analyzed using Microsoft Excel to generate a map graph of attendees by location.

RESULTS

Attendance

We received 1434 responses to the interest form before the beginning of the course, with 98% (1434/1460) of respondents expressing their interest in the opportunity to attend. The first webinar had the highest number of live attendees (343 people), but it was noted that a number of interested participants were not able to attend these live sessions (see below: Challenges).

In order to ensure that each webinar was accessible to attendees who could not log in or who otherwise could not attend live, each session was recorded and made available on YouTube. On average, there were 89 live participants per session and 750 YouTube views per recording (as of July 9, 2020) (Tables 1 and 2 ).

Table 1

Course Topics, Speakers, and Participants/Webinar

Session #	Date	Topic(s)	Max Participants
1	20-Apr	What's LI-RADS? History & Evolution	343
2	21-Apr	LI-RADS: Overview of CT/MR algorithm and major features	279
3	22-Apr	Malignant lesions that are not definitely HCC: LR-M	274
4	23-Apr	LI-RADS: Tumor in Vein	167
5	24-Apr	LI-RADS Ancillary features	142
6	27-Apr	Pitfalls in Cirrhosis Imaging and User Errors in applying LI-RADS	151
7	28-Apr	LI-RADS: HCC Treatment Response: from Concepts to Algorithms	89
8	30-Apr	Present and Future of LI-RADS	50
9	1-May	LI-RADS Case-based Review	60
10	4-May	LI-RADS: Is There a Clinical Need?	57
11	5-May	Management Implications of LI-RADS Categories	57
12	6-May	LI-RADS Technical Recommendation	45
13	7-May	Ultrasound Screening of Hepatocellular Carcinoma	46
14	8-May	Contrast Enhanced Ultrasound of the Liver	42
15	11-May	Imaging of Liver Iron Deposition and Steatosis	71
16	12-May	Abbreviated MRI for HCC surveillance	59
17	13-May	Liver MR Technique Related Pitfalls	50
18	14-May	MR Imaging of Hepatic Adenomas: Subtypes and their Management	48
19	15-May	Hepatocellular Carcinoma: LI-RADS Treatment Response after locoregional therapy	43
20	18-May	Liver Directed Therapy for HCC	39
21	19-May	Y90 Microspheres: State of the Science	37
22	20-May	Cholangiocarcinoma: Current concepts	42
23	21-May	Assessment of Pediatric Liver Tumors	85
24	22-May	Liver malignancy masquerading as benignancy and how LI-RADS fits in	38
25	26-May	Imaging Evaluation of Liver Transplantation	51
26	27-May	Liver Transplantation: What Surgeons Expect from Radiologists	38
27	28-May	The globalization of LI-RADS: roadmap and barriers to overcome	31
28	29-May	Hepatocarcinogenesis: Rad./Path Correlation	49

Table 2

YouTube Views Per Video (As of July 9, 2020)

Video Title	Video Publish Time	Views
1. What's LI-RADS? History & Evolution - Part 1.	4/21/2020	2367
2. What's LI-RADS? History & Evolution - Part 2	4/21/2020	1260
3. LI-RADS: Overview of CT/MR algorithm and major features - Part 1	4/22/2020	1734
4. LI-RADS: Overview of CT/MR algorithm and major features - Part 2	4/22/2020	828
5. LI-RADS: Malignant lesions that are not definitely HCC: LR-M - Part 1	4/23/2020	1110
6. LI-RADS: Malignant lesions that are not definitely HCC: LR-M - Part 2	4/23/2020	558
7. LI-RADS; LR- Tumor In Vein (TIV) - Part 1	4/24/2020	709
8. LI-RADS; LR- Tumor In Vein (TIV) - Part 2 (Q&A).	4/24/2020	461
9. LI-RADS Ancillary Features - Part 1.	4/25/2020	695
10. LI-RADS Ancillary Features - Part 2	4/26/2020	369
11. Pitfalls in Cirrhosis Imaging and User Errors in applying LI-RADS - Part 1.	4/28/2020	1631
12. Pitfalls in Cirrhosis Imaging and User Errors in applying LI-RADS - Part 2.	4/28/2020	464
13. LI-RADS: HCC Treatment Response: from Concepts to Algorithms - Part 1.	4/29/2020	537
14. LI-RADS: HCC Treatment Response: from Concepts to Algorithms - Part 2	4/29/2020	311
15. Present and Future of LI-RADS - Part 1.	5/1/2020	326
16. Present and Future of LI-RADS - Part 2 (Q&A).	5/1/2020	161
17. LI-RADS Case-based Review - Part 1.	5/2/2020	625
18. LI-RADS Case-based Review - Part 2 (Q&A).	5/2/2020	293
19. LI-RADS: Is There a Clinical Need? - Part 1.	5/5/2020	273
20. LI-RADS: Is There a Clinical Need? - Part 2 (Q&A).	5/5/2020	128
21. Management Implications of LI-RADS Categories - Part 1.	5/6/2020	285
22. Management Implications of LI-RADS Categories - Part 2	5/6/2020	189
23. Ultrasound Screening of Hepatocellular Carcinoma - Part 1.	5/7/2020	305
24. Ultrasound Screening of Hepatocellular Carcinoma - Part 2 (Q&A).	5/7/2020	173
25. LI-RADS Technical Recommendation - Part 1.	5/8/2020	288
26. LI-RADS Technical Recommendation - Part 2 (Q&A).	5/8/2020	123
27. Contrast Enhanced Ultrasound of the Liver - Part 1.	5/9/2020	266
28. Contrast Enhanced Ultrasound of the Liver - Part 2	5/9/2020	78
29. Imaging of Liver Iron Deposition and Steatosis - Part 1.	5/12/2020	348
30. Imaging of Liver Iron Deposition and Steatosis - Part 2	5/12/2020	124
31. Abbreviated MRI for HCC surveillance - Part 1.	5/13/2020	171
32. Abbreviated MRI for HCC surveillance - Part 2	5/13/2020	75
33. Liver MR Technique Related Pitfalls.	5/14/2020	235
34. MR Imaging of Hepatic Adenomas: Subtypes and Their Management - Part 1.	5/15/2020	365
35. MR Imaging of Hepatic Adenomas: Subtypes and Their Management - Part 2	5/15/2020	137
3. Virology and Immunology of COVID-19.	5/17/2020	163
36. Hepatocellular Carcinoma: LI-RADS Treatment Response after locoregional therapy - Part 1	5/18/2020	302
37. Hepatocellular Carcinoma: LI-RADS Treatment Response after locoregional therapy - Part 2 (Q&A)	5/18/2020	123
38. Liver Directed Therapy for HCC - Part 1	5/19/2020	175
39. Liver Directed Therapy for HCC - Part 2	5/19/2020	70
40. Y90 Microspheres: State of the Science - Part 1.	5/19/2020	160
41. Y90 Microspheres: State of the Science - Part 2.	5/19/2020	54
42. Cholangiocarcinoma: Current concepts - Part 1.	5/20/2020	368
43. Cholangiocarcinoma: Current concepts - Part 2(Q&A).	5/20/2020	143
44. Assessment of Pediatric Liver Tumors - Part 1 .	5/21/2020	281
45. Assessment of Pediatric Liver Tumors - Part 2(Q&A) .	5/21/2020	72
46. Liver malignancy masquerading as benignancy and how LI-RADS fits in-Part 1.	5/22/2020	270
47.Liver malignancy masquerading as benignancy... - Part 2(Q&A) .	5/22/2020	120
48. Imaging Evaluation of Liver Transplantation Part 1.	5/26/2020	272
49. Imaging Evaluation of Liver Transplantation Part 2(Q&A).	5/26/2020	136
50. Liver Transplantation; (What Surgeons Expect from Radiologists). Part 1.	5/29/2020	192
51.Liver Transplantation; What Surgeons Expect from Radiologists. part2(Q&A).	5/29/2020	136
52. The globalization of LI-RADS: roadmap and barriers to overcome.	5/29/2020	107
53.Hepatocarcinogenesis; Rad./Path Correlation.	6/12/2020	221
Total Views		21367

Postcourse survey results

Demographics

The survey was sent to 1,460 registrants of the OLIC webinar series. There were 487 respondents to the survey for an overall response rate of 33%. The survey revealed that attendees joined from 37 countries and six continents with the largest percentage of attendees (28%) living in the United States (Fig 2). Approximately 63% (306/487) of participants were practicing radiologists while nearly all of the remainder 37% (181/487) of participants were either fellows or residents, if not medical students (Fig 1 ). Of the practicing radiologists, approximately 72% (219/306) had completed their residency training more than 5 years ago and 28% (87/306) finished their training less than 5 years ago.

Figure 2

Post-course survey respondents by country of practice. (Color version of figure is available online.)

Figure 1

Post-course survey respondents’ level of training. (Color version of figure is available online.)

Perception of OLIC

Responding to questions in this section was made optional for participants. As a result, a few respondents skipped some of the questions. The overwhelming majority, 97% (473/487), found the OLIC webinar series to be either somewhat or very beneficial; 82% (398/487) indicated that the information gained from the webinar series was very beneficial while 15% (75/487) stated that the information was somewhat beneficial (Fig 3 ).

Figure 3

Post-course survey respondents’ evaluation of the lecture series benefit in their future practice. (Color version of figure is available online.)

There were 484 responses to the questions evaluating the course expectations. Essentially all attendees felt that the webinar sessions met 43% (207/484) or exceeded 57% (276/484) expectations. Only one respondent indicated that the series was not beneficial or did not meet their expectations (Fig 4 ).

Figure 4

Attendees expectations of the lectures. (Color version of figure is available online.)

We asked respondents to rate their perception of virtual conferences before and after February 2020, when the effects of the SARS-CoV-2 pandemic started to emerge. Of the 475 people who responded to this question, 11% (50/475) would not have considered virtual conferences before February 2020, 35% (168/475) would have considered but preferred in-person conferences, 35% (170/475) would have watched recorded lectures, and 18% (87/475) would have preferred virtual conferences occurring in real-time (Fig 5 ). When asked about their perception of virtual conferences after attending OLIC lectures, 61% (295/481) of the respondents enjoyed the virtual format more than in-person conferences, while 38% (184/481) enjoyed the webinar format but preferred in-person conferences. Only 0.5% (2/418) respondents disliked the format (Fig 6 ).

Figure 5

Post-course survey respondents’ perception of virtual conferences before February 2020. (Color version of figure is available online.)

Figure 6

Post-course survey respondents’ perception of virtual conferences after attending the online liver imaging series (OLIC). (Color version of figure is available online.)

Finally, participants rated their willingness to attend virtual conferences in the future, assuming that there can be a widespread resumption of in-person conferences. Of the 480 respondents, 84% (405/480) said that they would attend future virtual conferences even if in-person conferences resume while 15% (72/480) were unsure. Only 0.5% (3/480) indicated that they would not attend virtual conferences in the future (Fig 7 ).

Figure 7

Post-course survey respondents’ answers to the question “Assuming widespread resumption of in-person conferences, will you continue to attend virtual conferences in the future?”. (Color version of figure is available online.)

Discussion

The SARS-CoV-2 pandemic has severely disrupted traditional educational models throughout healthcare, particularly in-person learning both at one's institution, and attendance at regional, national, and global continuing medical education conferences. The new social distancing guidelines, restrictions on travel, and restrictions on academic mission funding pose challenges for radiologists as they try to educate themselves and also educate the next generation of radiologists. According to the Accreditation Council for Graduate Medical Education, the effect of the pandemic on imaging volumes and procedure mix could necessitate the need to lengthen the training of some radiology residents and fellows (10). This possible increased length in training combined with the persistent need for social distancing highlights the need to develop alternative educational approaches for teaching residents, fellows, as well as practicing radiologists.

As new approaches are developed, clinician-educators can leverage modern technology to expand the reach of their training programs at a low cost to the program and attendees. The costs of the course were limited mainly to subscriptions of the necessary online services and platforms; GoToMeeting business plan annual subscription costs $192, SurveyMonkey annual plan is $384, and the MailChimp plan costs $50 per contacts per month. The total expenses were under $1000 without considering the costs of other equipment such as personal computers, webcams, audio devices, and internet access subscriptions which are widely available nowadays and instructors used their own. Another significant cost reduction factor was that instructors did not get any financial compensation. Guest faculty at in-person conferences or meetings usually costs an average of $2000–$4000 to cover travel, food, and other expenses such as honoraria. In our course, speakers were generous to volunteer (11, 12). As a result, it was possible to offer the course for free both in real-time and on-demand through YouTube.

We found that the OLIC had a large international audience with nearly three-quarters of survey-responding attendees living outside of the United States. We are hopeful that courses like the OLIC can help to standardize educational opportunities across the world and mitigate existing global educational inequalities (13). We believe that the high number of registrants, real-time attendees, on-demand viewers, and positive response to the survey reflects the strength of our faculty as well as the need for such a format. A further step would be seeking Continued Medical Education accreditation of the course material as enduring activities. While this would require extensive work and resources, it will maximize the benefit to radiologists who could use this credit to maintain their certification (14). We included one open-ended question asking if the respondents would change anything about the course. While the majority agreed that nothing should be changed, the requests mainly involved providing PDF handouts in future lectures. Some participants mentioned the timing was a challenge for them but making the course available on YouTube solved this problem. Regarding the course content, some participants suggested the inclusion of less LI-RADS lectures and more case discussions. Others commended the question and answer sessions and being able to directly interact with the speakers.

Challenges

Despite sharing the link privately via direct emails with interested applicants, we had an unfortunate episode of a non-registrant interrupting and disturbing one of the lectures. Consequently, security precautions were put in place to avoid future disturbances. Thereafter, the webinars were password-protected and locked shortly after the presentation began. This makes it difficult for any potential hackers to be allowed in the webinar. Additional security measures included the manual assignment of host status, which allows screen-sharing and more control over the webinar, to the presenter(s) before the start of each webinar, and restricting the ability of other participants to share their screen. Following the implementation of these security measures, no further disturbances to the presentations took place after the initial incident. Participants were still able to interact with the course instructors as usual since questions and discussions go through the audio and chat channels, not through the screen-sharing option that was disabled for attendees.

Similar disruptive interruptions have been reported by users across multiple video conferencing platforms. As the SARS-CoV-2 pandemic progressed, the video conferencing platforms released multiple security updates helping users to avoid malicious interruptions (15). Furthermore, these platforms developed more tailored solutions for large webinars that make it easier to manage large numbers of attendees by disabling camera and screen sharing and offer more features such as a dedicated question and answer section and live polling that could enhance teaching.

Another challenge was the unexpectedly high number of registrants. The plan purchased for video conferencing allowed for a maximum of 250 attendees. Thus, nearly 100 attendees were not able to log-in and view the first session live. The number of attendees surpassed the account limits on two other occasions (webinars 2 and 3). However, there are more advanced webinar solutions tailored to such big numbers at an added cost. With early planning and resources, this problem could be practically addressed. Despite the high attendance rate at the beginning, numbers of live attendees started to drop towards the end of the course (Table 1). High dropout rates are a known phenomenon in online learning with the most common reason being the unexpected time of commitment by registrants (16). Since learners signed up for this course voluntarily, we can assume they might have reached their learning goals and hence shouldn't be considered as dropout (17) Another reason for the decline in attendance could be the availability of the recorded sessions on YouTube which could be accessed at the learner's convenience (18). The inclusion of incentives such as certificates of completion might help future projects if maintaining a high attendance rate is desired.

The lack of a control group limits the evaluation of the effectiveness of this model compared to standard instruction. Evaluation using a pre-test-post-test design was impractical because of limited preparation time, the statistical weakness of this design, and the fact that it would have complicated the registration process given the large number of participants (19). However, further comparison studies of the videoconferencing and standard didactic instruction models could provide valuable insights to optimize future virtual radiology education.

Conclusion

The success of the OLIC indicates that videoconferencing technology can be used as an inexpensive, yet agile solution to address the educational needs of radiologists internationally. We believe that our success was due in part to having a committed course director, a well-defined curriculum, a cohort of experienced faculty, the effective use of social media as a marketing tool, and the utilization of multiple online tools for content consumption.

The positive responses to our post-course survey suggest that virtual education is here to stay, even in a postpandemic world. Educational institutions and scientific societies should consider directing resources to adapt and develop such models.