A virtual sub-internship for otolaryngology-head and neck surgery.

OBJECTIVE: During the COVID-19 pandemic, away rotations were suspended for safety purposes. This led to the development of online interactive learning modules for students, now known as virtual sub-internships (VSIs). To date, VSIs within otolaryngology-head and neck surgery (OHNS) have been limited in their description and design. STUDY
DESIGN: Cross-sectional survey.
SETTING: ACGME-accredited OHNS residency program.
METHODS: Our curriculum for an OHNS VSI is presented. Based on the model used by our OHNS residency program, the VSI consisted of a 2-week block of activities and interactive small-group discussions. A post-VSI feedback survey was distributed to participants; results are reported.
RESULTS: Six 2-week VSI sessions were administered. Twenty-one individuals participated from all US regions, median 4 individuals per session. Fifteen participants (71.4%) completed the feedback survey. Of survey respondents, 20% lacked a home OHNS residency program, and no respondents' home-programs featured a VSI. All respondents were satisfied with the schedule and organization of the VSI, and 73.3% (11 of 15) felt it was of appropriate duration. All respondents reported a high degree of familiarity with the program, with a greater (86.7%) or equal (13.3%) level of interest in applying to the residency program as a result of participating in the VSI.
CONCLUSION: This VSI curriculum offers a well-received virtual learning experience for medical students applying to OHNS residency. It provides an opportunity for programs to expand their appeal to potential applicants who may otherwise be restricted in their ability to travel.

INTRODUCTION

Visiting sub‐internships, also known as externships or away rotations, are an important aspect of the fourth‐year curriculum for medical students, particularly those applying into a surgical residency. Within competitive surgical fields such as otolaryngology‐head and neck surgery (OHNS), the away rotation serves as an opportunity for education and audition, allowing programs and applicants to demonstrate their strengths and assess compatibility. For students from institutions without an OHNS residency program, away rotations are a crucial time to gain experience, demonstrate competence, and establish connections in the field. Prior to COVID‐19, participation in away rotations was nearly universal for successful applicants to OHNS, with a significant majority completing 2 months of rotations away from their home program. , , In surgical specialties, completion of an in‐person sub‐internship has been shown to increase the odds of matching at the host program. , ,

In response to COVID‐19, away rotations were suspended during the 2020 to 2021 academic year. Recommendations for “novel teleconferencing experiences” consisting of “virtual didactic programs and grand rounds” were suggested in lieu of in‐person sub‐internship rotations. , , As program websites and social media accounts have become a key means of connection with prospective applicants, programs across multiple specialties developed various forms of virtual sub‐internships (VSIs). , , , , , Dean et al (2020) first described the implementation of a plastic surgery VSI built around the established Accreditation Council for Graduate Medical Education (ACGME) core competencies for in‐person rotations. In other surgical subspecialties, similar curricula have been well‐received and highly valued by the majority of participating students. , , In OHNS, only single‐day, townhall, or webinar format VSIs have been described in the literature to date. , For the 2021 to 2022 application year, the Coalition for Physician Accountability and the Association of American Medical Colleges (AAMC) announced that in‐person away rotations are to be limited to one per student, with specific exceptions for students without home‐programs. In this new paradigm, OHNS programs will continue to face the problems of accurately assessing candidate appropriateness‐of‐fit and developing program interest through inherently limited virtual interactions. Therefore, implementing a comprehensive and interactive VSI in OHNS is critical, and may soon become a practice used by residency programs throughout the United States.

METHODS

Our VSI consisted of a 2‐week curriculum comprised of scheduled meetings and activities with our institution's OHNS faculty and residents. Core objectives addressed by the curriculum were based upon ACGME objectives and a surgical committee statement, outlined in Issa et al (2015). , (curriculum objectives are reported in Table 1).

TABLE 1

Virtual sub‐internship curriculum objectives and educational modalities

Objective	Educational modality
Didactic learning	1. Weekly journal club
	2. Case reviews with faculty and residents
	3. Virtual grand rounds
	4. Subspecialty‐specific lectures
	5. Virtual conferences held by professors
	6. Exposure to broad spectrum of subspecialties, including head and neck surgical oncology, facial plastic and reconstructive surgery, rhinology, laryngology, sleep surgery, salivary surgery, and otology
Opportunity for mentorship	Weekly sessions with faculty and residents
Professional development	1. Interview preparation session
	2. Student final presentation
Assessing mutual program‐applicant “personality fit”	Virtual happy hour with residents

Curriculum

The 2‐week curriculum consisted of a rotating schedule of events arranged throughout the day. Sessions included journal club, faculty‐led discussions of various OHNS topics, resident‐led study sessions, operating room sessions, Zoom interview preparation, grand rounds, student presentations, and a social hour. Topics for faculty‐led discussion included otology, rhinology, facial nerve reanimation, facial plastic and reconstructive surgery, head and neck surgery, and laryngology. Sessions were held via Zoom from faculty offices, the resident workroom, and various operating rooms. The resident study session is an informal discussion of surgical cases with an emphasis on board review and clinical pearls. The operating room sessions consisted of tours of the operating rooms along with faculty‐ and resident‐led discussions of surgical planning, relevant anatomy, and diagram‐based teaching in a question‐and‐answer format. No identifying personal health information or images of patients in surgery were shared during the VSI. The Zoom interview preparation session was led by faculty and consisted of a common interview question review, a detailed account of the structure of the Zoom interview day, tips and tricks to maximize presentability, and a workshop on how to objectively decide and prepare a rank list. At the start of the VSI, participants were assigned to a resident mentor and were encouraged to schedule additional time with individual faculty throughout the course (a complete course schedule is outlined in Table 2).

TABLE 2

Virtual sub‐internship course schedule

	Activity	Topic	Duration (min)	Frequency
Monday	Journal club	Head and neck oncology	30	Weekly
	Faculty‐led discussions	Facial trauma and reconstruction	30 to 60	Weekly
Tuesday	Faculty‐led discussion	Facial plastics and reconstructive surgery	30 to 60	Weekly
	Operating room session	Various subspecialties	60 to 90	Weekly
Wednesday	Faculty‐led discussion	Rhinology; otology	30 to 60	Weekly
	Resident office hours	Miscellaneous topics	60	Weekly
Thursday	Faculty‐led discussion	Laryngology; salivary surgery; facial nerve injury case review; head and neck reconstruction	90 to 150	Weekly
	Operating room session	Various subspecialties	60 to 90	Weekly
Friday	Resident study session	Miscellaneous topics	60	Week 1
	Zoom interview preparation	Professional growth/education	30	Week 1
	Grand rounds	Various subspecialties	180	Week 2
	Student presentations	Various subspecialties	10 min/presentation	Week 2
	Resident social hour	Program atmosphere	60	Week 2

Feedback survey

A brief feedback survey was generated using Qualtrics software. (Qualtrics, Provo, UT). The survey consisted of 14 multiple choice and free‐response questions. Upon conclusion of the 2‐week VSI, all participants were emailed a survey link and informed that completing the survey was optional, and that results from the survey could be used in a study analysis. Completing the survey indicated their consent. Responses were kept anonymous and it was emphasized several times throughout the survey that participation and content of responses would not be shared with residency selection committee members to influence the chance of interviewing or matching. Totals and frequencies were calculated. Inclusion criteria were defined as a fourth‐year US medical student who participated in the entire 2 weeks of the VSI. This study was deemed exempt from review by our institution's Internal Review Board.

RESULTS

Across the study period, 21 individuals participated in the OHNS VSI, of whom 33.3% (7) identified as female. The VSI involved a total of six 2‐week courses beginning in September 2020, with a median of four participants per 2‐week course. Upon conclusion of the course, 16 (76.2%) participants completed the exit survey. One respondent was unable to complete the 2‐week curriculum due to personal reasons and was excluded from the final analysis. The 15 participants were distributed across United States census‐designated regions evenly: South (33.3%), Midwest (33.3%), Northeast (20.0%), and West (13.3%). All applicants from the West region were from California‐based programs. Among participants, 53.3% first learned of the VSI through the affiliated residency website, whereas 33.3% found it via the Otomatch.com message board or through social media posts by our institution's OHNS‐affiliated accounts (Instagram, Facebook, and Twitter). (Table 3).

TABLE 3

VSI survey demographics and program characteristics

Category	No. (%)
Total participants	21
Female	7 (33.3)
Total VSI sessions	6
Median class size [range]	4 [2‐4]
Duration (weeks)	2
VSI feedback survey (n = 15)
US region
South	5 (33.3)
Midwest	5 (33.3)
Northeast	3 (20.0)
West	2 (13.3)
OHNS residency offered at home institution
Yes	12 (80.0)
No	3 (20.0)
OHNS VSI offered at home institution
Yes	0
No	15
Previously completed OHNS sub‐internship
Yes	13 (86.7)
No	2 (13.3)
Number of additional intended VSIs
0	5 (33.3)
1 to 2	10 (66.6)
≥3	0

Abbreviations: OHNS, otolaryngology‐head and neck surgery; VSI, virtual sub‐internship.

Twelve of 15 (80%) survey respondents were from programs that featured an OHNS residency, and 73.3% (13 of 15) had already completed at least one in‐person sub‐internship in OHNS. None of the participants had home‐programs which featured a VSI. Of the participants who completed the OHNS VSI, 66.6% (10 of 15) reported their intent to complete 1 to 2 additional VSI's at other institutions, whereas the remaining 33.3% did not intend to do other VSI's.

Regarding the length of the OHNS VSI curriculum, 11 of 15 participants (73.3%) felt the 2‐week timeline was of appropriate duration, whereas 2 (13.3%) wanted a longer, and 2 (13.3%) wanted a more condensed curriculum. All participants reported being at least “satisfied” with the schedule and organization of the VSI, with 11 of 15 (73.3%) reportedly being “highly‐satisfied.” (Table 4).

TABLE 4

Responses from the 2020 to 2021 OHNS VSI feedback survey

Survey	No. responses (%) (n = 15)
1. Where did you first hear about this OHNS VSI?
OtoMatch.com message board	3 (20.0)
Otolaryngology residency website	8 (53.3)
Social media (Instagram, Facebook, etc.)	2 (13.3)
Word of mouth	1 (6.7)
Other	1 (6.7)
2. Regarding the 2‐week length of the VSI
It could be shorter	2 (13.3)
It was “just right”	11 (73.3)
I wish it was longer	2 13.3)
Other	0
3. Satisfaction with VSI schedule and organization
Highly satisfied	11 (73.3)
Satisfied	4 (26.7)
Unsatisfied	0
4. New level of interest/excitement about the affiliated OHNS residency program
Greater than before VSI	13 (86.7)
Same (baseline)	2 (13.3)
Less	0

Abbreviations: OHNS, otolaryngology‐head and neck surgery; VSI, virtual sub‐internship.

Having completed the OHNS VSI, all participants reported continued interest in applying and matching to the affiliated otolaryngology residency program, with 86.7% (13 of 15) reporting a greater level of interest in the program than before participating in the VSI. Following completion of the VSI, participants reported a median rating of “Extremely Comfortable” (5 out of 5) with topics related to the residency program, including living in the region, interacting with residents and faculty, and exploring research options within the field (Figure 1).

FIGURE 1

Average self‐reported familiarity with each topic is represented by bar height (0 = not at all familiar or comfortable, 5 = extremely familiar or comfortable). VSI, virtual sub‐internship; OHNS, otolaryngology‐head and neck surgery; ENT, ear/nose/throat. , median familiarity rating

DISCUSSION

The OHNS VSI format described can be easily implemented by programs to offer a representative virtual learning experience. This curriculum served to capture the attention of potential applicants from around the country, with the effect of maintaining or increasing their interest in the affiliated residency program. After completing the VSI, despite having limited to no in‐person experience, participants reported a high level of comfort with topics related to residency at the affiliated institution. Though there can be no substitute for an in‐person rotation, a VSI provided a high degree of program familiarity, suggesting that this format can be a surrogate experience in lieu of an in‐person rotation.

The format used for the VSI was well‐received based on the survey feedback responses. For programs interested in implementing their own VSI, the development of a 2‐week schedule is a reasonable length of time based upon our model. Programs should encourage participation by faculty from as many subspecialties as possible, set side adequate time for each session, and feature a mix of group activities as well as one‐on‐one sessions with faculty and residents. Of note, because the VSI was classified as an informal course by the affiliated medical school, grades, and course credit were not assigned to participants.

In the era of COVID‐19, many opportunities for travel and in‐person education were suspended for safety purposes. , In light of these changes, the VSI offers an opportunity for programs to expand their appeal to potential applicants who may otherwise be restricted in their travel. As travel restrictions are lifted, the success of the VSI should not be discounted in the post‐COVID‐19 era. Based on this data, the VSI format can stimulate applicant interest in the field and in hosting institutions. In addition, it has been suggested that VSIs may enhance the diversity of a program's applicant pool. Importantly, these benefits are possible without the added cost burden to applicants for travel and housing, and without the legal liability typically assumed by institutions for visiting students. , , Much like working from home and the associated increased utilization of tele‐medicine, the VSI may become a permanent experience alongside in‐person away‐rotations for residency programs well into the future.

There are myriad ways to improve upon this OHNS VSI model. Feedback received from VSI participants voiced a desire for greater opportunities to demonstrate their unique value as future residents. Dean et al outline a series of hands‐on workshops in addition to virtual meetings, which aim to challenge students' motor skills and demonstrate their competency for surgical mechanics. Programs looking to develop the VSI model may consider adding hands‐on experiences and graded assignments.

Limitations of the VSI survey include a low number of participants, which may affect the generalizability of the findings. Furthermore, it was not possible to remove self‐selection bias where only the most interested and impressed candidates may have replied to the survey. The timing of the survey shortly after completion of the VSI may also introduce a recency bias, allowing for the conflation of participants' excitement over the prospect of matching to residency. Despite numerous reassurances of anonymity, given the highly competitive residency match environment, participants may have felt compelled to rate aspects of the VSI more highly. In addition, reporting familiarity with certain topics is a relative metric, and may only reflect one's perception of familiarity more than his or her true knowledge. Lastly, lacking a pre‐VSI questionnaire limits our ability to determine the true effect of the VSI on building familiarity and positive feelings about the program. These limitations highlight the need for future efforts and a further collaborative, multi‐site study on this evolving topic in medical education.

CONCLUSION

A successful and easy‐to‐implement VSI curriculum for OHNS programs is described. As during the COVID‐19 pandemic, VSI opportunities can foster interest in programs for those applicants who are otherwise unable to travel. VSIs are convenient, effective, and may become more prevalent in years to come.

CONFLICT OF INTEREST

The authors declare no conflicts of interest.

AUTHOR CONTRIBUTIONS

Jeffrey D. Bernstein: study design, data collection, data analysis, manuscript writing, manuscript editing, and preparation of figures. Laurel L. Ball: manuscript writing, manuscript editing, preparation of figures. Zachary B. Nardone: manuscript writing, manuscript editing, preparation of figures. Deborah Watson: study design, manuscript editing.