A scoping review on adaptations of clinical education for medical students during COVID-19.

Rapid advances in clinical education in response to the COVID-19 pandemic are taking place globally. This scoping review updated the educational strategies which could be applied by clinical educators in their practice to effectively maintain clinical attachment programs for medical students amidst public health crises. Almost all elements of clinical teaching were deliverable, whether it was online, onsite, virtual or blended, their educational effectiveness should be further examined. Increase in the number of telemedicine related publications were remarkable, and they could serve as a scalable model for future educational programs to be incorporated into the medical student curricula.

Background

Despite the profound effects from the COVID-19 pandemic, the medical community has fought to maintain high-quality education by flinging open the doors of technology into teaching and patient care [1]. Recent publications include reviews encompassing the rapid advances in medical education in response to the current pandemic and provided easily accessible summaries to grateful readers [1,2].

Previous systematic or scoping reviews up to September 2020 on medical educational responses to COVID-19 have highlighted areas of relative strength, but also parts in need of further development and research such as telemedicine. A rapid systematic review by Gordon et al. [1] suggested that the range of options deployed offered good guidance for the medical education community despite outcome data mostly being Kirkpatrick’s level 1. Daniel et al.’s [2] scoping review identified publications in telehealth, student admission, and faculty development to teach in remote settings to be areas in urgent need of further research.

Previous reviews dealt with medical education across the continuum, therefore detailed programs focusing on clinical teaching in undergraduate medical students, when direct patient contact limited due to the COVID-19, have been somewhat lacking. Especially, as clinical teaching is known to be effectively taught and learned through workplace based experiences, concerns exist amongst teachers [3] and students [4] on the educational effectiveness of online clinical teaching replacing hands-on experiences in clinical settings. Clinical teachers have been overloaded with the sudden advent of new technologies during COVID-19, having to deliver online or telemedicine clinics at short notice. Clinical educators in both primary/community care and hospitals would benefit from information on practical and efficient ways of adapting their educational programs in such disruptions with easily referenced resources summarizing the available literature. The quality and details of additional developments flooding the literature since the previous reviews are also in need of further analyses.

The purpose of this scoping review was to summarize the extent and details of clinical teaching adaptation for medical students in response to COVID-19 up to March 2021, which could be readily and easily used by clinicians in both primary care settings and hospitals.

Methods

Our scoping review applied Arksey and O’Malley’s [5] protocol methodology.

Identifying the research question

The research questions of this study were: (1) what clinical education development has been performed for undergraduate medical students to cope with the limitations on direct patient contact due to COVID-19? (2) What further advancements in educational developments or research have been suggested as areas of improvement since the previous reviews?

Identifying relevant studies

Our scoping review included peer-reviewed publications including primary literature, review editorial, commentary/opinion pieces, but did not include grey literature, and conference proceedings. PubMed, Embase, Scopus, Web of Science (including ESCI) and ERIC were searched with keywords in the title and/or abstract and subject headings (e.g., MeSH, EMTREE) as appropriate.

Selecting the studies to be included

To select studies for inclusion, we used an iterative approach. Two authors (HP and SHS) independently reviewed the titles and/or abstracts of all the articles against an initial set of inclusion and exclusion criteria. Articles without abstracts underwent full-text screening to assess suitability. The two authors met 5 times. On the first meeting the authors worked to create a shared understanding of the criteria, and on the following meetings the selected citations were compared and discussed for any discrepancies. Any coding disagreements were facilitated by author YML. Inter-rater reliability was calculated using Cohen’s Kappa.

Inclusion criteria

Published articles on: (1) research or description on clinical teaching programs developed by medical schools, affiliated hospitals or primary care practices; (2) for undergraduate medical students; (3) brought by COVID-19 (4) from 1st March 2020 to 1st March 2021; (5) published in the English language; (6) listed in the MEDLINE and EMBASE databases including case reports, case studies, case control studies, cohort studies and randomised control trials, letters to the editor, commentaries, editorials, perspectives, and any potential reviews or meta-analyses. (7) Any type of study were included such as quantitative, qualitative, and mixed method studies.

Exclusion criteria

Articles focusing on (1) postgraduate medicine (e.g. resident, fellow), or (2) other health care professionals bar medical students (e.g. nursing, physio, dentist); (3) on educational theories/concepts or general opinions without actual implementation or adaptations in clinical teaching and; (4) non-clinical education (e.g. anatomy, basic sciences) were excluded.

Charting the data

Our protocol was drafted using the Preferred Reporting Items for Systematic Reviews and Meta-analysis Protocols [6,7]. Both the PRISMA-ScR checklist and the protocol for scoping review can be found in the appendix section. All three authors independently reviewed and extracted data from all the included articles and any discrepancy was resolved by iterative discussions. YML served to discuss any coding disagreement and as a tiebreaker.

Collating, summarizing, and reporting results

Authors developed a coding sheet in the protocol (Appendix C). The abstracted data was analysed in quantitative and thematic content analysis. Frequency was calculated for: (1) overall characteristics including publication year, curricular phase, institutional setting, educational platform, type of publication, country of papers, medical specialities; (2) papers that reported educational outcomes were classified by Kirkpatrick level. Content analysis was performed in the following categories: (1) telemedicine; (2) online clinical education by curricular phase, patient type, types of clinical skills/procedures, clinical case scenarios discussion; (3) onsite clinical education. Microsoft Excel and SPSS v25 were used for data management and analysis.

Results

Database searching yielded 1369 records and 11 extra from manual searching. Removing duplicates resulted in 759 articles. After title and abstract screening, 615 records were excluded. 144 underwent full text screening and 87 were excluded. Inter-rater reliability at the screening phase was 0.899 suggesting excellent consistency. Fig. 1 shows the PRISMA flow diagram for article identification.

Fig. 1

The PRISMA flow diagram for article identification.

UGME: Under Graduate Medical Education.

PGME: Post Graduate Medical Education.

Overall characteristics of the reviewed articles

Table 1 shows the general features of the reviewed papers. Literature on educational developments for clerkship accounted for 84.2% of the papers, 8.8% for preclinical years and four papers (7.0%) included both preclinical and clinical phases. Most articles were university based clinical education programs (84.2%), and online education accounted for the majority (91.2%) of the reported developments, which included 17 papers (29.8%) in telemedicine. Publication types were diverse: which included original research (35.1%), short report (28.1%), letters, perspective opinion, commentaries and reviews. Articles originated from 18 countries and USA accounted for half of the 57 analysed publications (Fig. 2 ). 16 programs were conducted by medical schools and an array of specialties delivered alternative ways of clinical teaching (Fig. 3 ). 27 papers reported the educational outcomes of their applied programs, mostly at Kirkpatrick level 1 (77.8%) (Fig. 4 ).

Table 1

Characteristics of the analysed papers (N = 57).

		Freq	%
Year	2020	35	61.4
	2021	22	38.6
Curricular phase	Both	4	7.0
	Clinical	48	84.2
	Preclinical	5	8.8
Institutional setting	Both	1	1.8
	Community	8	14.0
	University	48	84.2
Educational platform	Telemedicine	17	29.8
	Online	32	56.1
	Onsite	5	8.8
	Online/onsite both	3	5.3
Type of publication	Original article	20	35.1
	Short report	16	28.1
	Letter	7	12.3
	Perspective	6	10.5
	Opinion	4	7.0
	Commentary	2	3.5
	Review	2	3.5
Total Analysed Articles		57	100.0

Fig. 2

Country of origin of analysed papers.

Fig. 3

Medical specialties that reported clinical education developments.

# Programs developed by medical schools.

Fig. 4

Kirkpatrick Level of programs reporting outcome data n = 27.

Clinical education via telemedicine

Adding to previous reviews, we found reports using telemedicine to overcome the challenges in clinical education amidst the pandemic disruption (Table 2 ). 17 papers [8,9,[18], [19], [20], [21], [22], [23], [24],[10], [11], [12], [13], [14], [15], [16], [17]] described the use of telemedicine as an educational tool mostly for clerkship rotations but three also included pre-clinical year students. Whilst telemedicine was mainly implemented in university hospitals, six programs were also executed in the community. Through the telehealth platform, diverse clinical competencies including history taking, communication skills, information management and diagnostic skills were taught, whilst education on physical examination and clinical procedures were scarce. In addition, professionalism was also learned through patient encounters over telemedicine platforms (Fig. 5 ). Practice guidelines [11,12,20,22] and training for faculty and students were provided [10,14,16,18,20,22,24] prior to encountering patients over the telehealth platform. In-advance informed consent from the patients were obtained and screening took place to ensure patients had the appropriate virtual interfaces [18,19,21].

Table 2

Clinical education development via telemedicine n = 17.

Author	Key findings	Clinical competencies	Site	Specialty	Country
Dawoud et al. [8]	Students guided through curated surgical video content of select neurosurgical procedures by faculty. Students joined telemedicine clinics observing and participating after prior completion of online telehealth training modules	2,3,4,9,11,12	Univ	Surgery	US
Pellegrini et al. [9]	1-on-1 attending and 4th yr. med student telemedicine clinic from available exam room. Two computers to access electronic health records.	1,2,4,8,10	Univ	ENT	US
DeVaro et al. [10]	3-way telehealth clinic with 3rd/4th yr. student, consented patient and ophthalmology attending. History taking and focused ophthalmic exam by student, presentation and feedback.	4,5,8	Univ	Ophthalmology	US
Rupley et al. [11]	Public health, telehealth visit & post-partum outreach teams ran by students. Rapid protocol and practice guideline updates by hospital leadership, EMR education and academic credit reward for student participation.	1,2,4,8,9,12	Univ	OBGYN	US
Coffey et al. [12]	Provision of practice guidelines for telehealth visits. Students joined as observers or participation.	2,4,5,7	Univ	MS	US
Iancu et al. [13]	Based on experience in applying telemedicine consultation as part of core and elective clerkships, 12 tips were suggested in setting stage, needs assessment, creating tools for student integration into telemedicine	1,2,4,5	Univ	MS	US
Ho et al. [14]	5 h student shifts on remote tele-ICU patient care, performing real interventions (medication, ventilator adjustment, lab recheck) as part of interdisciplinary care.	1,2,3,4,6,8,9	Univ	Med	US
Lal et al. [15]	Flipped classroom over zoom, role-play with faculty assessing symptoms and disclosing prognosis. Outpatient tele-palliative care clinic participation.	3,8	Univ	Med	US
Bhatia et al. [16]	Students guided patients with varying levels of technological literacy leveraging their communication skills. Switching to a telephone visit if the patient was still unable to connect.	1,2,3,4,6,8,9	Univ	FM	US
Hayes et al. [17]	20+ telemedicine family medicine patient encounters over 4 weeks. Illness script assignments reviewed by attending & different disease templates shared with students via online drive.	1,2,3,4,6,8,9	Univ	FM	US
Weber et al. [18]b	4-week virtual elective. Prior EMR training and introduction to workflow allowed students to document clinical information after encountering consented patients.	2,4,7,8	Univ	FM	US
Cain et al. [19]	Patients screened in advance, consented and ensured appropriate virtual interfaces in place. Students able to document the visit in patients' EMR and attested by attending.	2.4,5,7	Comm	FM	US
Carson et al. [20]b	Medical students staffed the hotline and assisted the underserved rural populations of northern Nevada; results showed significant improvements in students’ clinical skills including screening patients for COVID19, and triaging patients.	1,2,3,4,8,9	Comm	FM	US
Peterseim and Watson [21]	Telehealth clinic shadowing family medicine attending or student led e-consultation taking history and describing physical exam manoeuvres to patient to perform on themselves.	4,6,8,9	Comm	FM	US
Ruiz Colon et al. [22]	Students performed history taking, delivery of patient education and counselling after receiving basic telehealth training.	2,4,5	Comm	FM	US
Darnton et al. [23]a	Pre-clinical students already educated in basic clinical communication and exam skills consulted from own home whilst supervised by primary care physician in three-way telemedicine clinic.	1,2,4	Comm	FM	GB
Paul et al. [24]	Community care medicine involved final year students in telemedicine clinics and provided online small group teachings.	1,2,4,5,8,9,12	Comm	FM	GB

Clinical Competencies: 1. Professionalism; 2. Communication; 3. Scientific knowledge application; 4. History-taking; 5. Physical and mental examination; 6. Clinical Testing; 7. Clinical Procedures; 8. Information Management; 9. Diagnosis Skill; 10. Treatment, Prevention & Palliation; 11. Prognosis Skill; 12. Care provision in clinical context.

EMR: Electronic Medical Record.

Univ: University; Comm: Community.

ENT: Ear Nose Throat, otorhinolaryngology; FM: Family medicine; Med: medicine; MS: Medical school; OBGYN: Obstetrics and gynaecology.

US: United States of America; GB: United Kingdom (Great Britain).

Pre-clinical phase.

Both clinical and pre-clinical.

Fig. 5

Clinical competencies taught via telemedicine.

The categorization of clinical competency was adapted from the AAMC (American Association of Medical Colleges) [66].

At an ophthalmology telemedicine clinic, third and fourth-year students took histories, performed focused ophthalmic examinations, presented the findings to the faculty, and received 5-minute feedback sessions [10]. Pellegrini et al. [9] described teaching of fourth-year medical students during their otolaryngology telemedicine outpatient clinics. Rupley et al. [11] implemented three obstetric projects where students provided: (1) public health and access information to vulnerable patients; (2) telehealth outreach to antepartum and post-partum patients scheduled for telehealth visits; and (3) post-partum outreach interdisciplinary efforts. Family medicine departments used telemedicine for communication skills education [16]; scripts and disease templates used in telemedicine were shared amongst students to widen their clinical knowledge [17]; and prior training in EMR (Electronic Medical Records) allowed students to contribute to telemedicine documentation of consented patients visits [18]. Ho et al. [14] developed a pilot project integrating medical students into tele-ICU patient care using a 2-way video conferencing platform.

Telemedicine was also used for clinical teaching in the community health care setting. Carson et al. [20] staffed medical students in multicounty telehealth COVID-19 hotlines letting them assist the underserved rural population conducting audio-only exams, screening and triaging patients. Cain et al. [19] reported family medicine clerkship students reviewing and documenting on EMR charts later reviewed by the attending. Similar pilot programs were reported by family medicine telehealth clinics at the Medical University of South Carolina [21]. A unique program led by Standford University students, called Cardinal Free Clinics, opened immediately after the onset of the pandemic, and changed to a telehealth clinic model when in-person patient clinics were forced to close due to university COVID-19 policies [22]. In the UK, Darnton et al. [23] used telemedicine for teaching pre-clinical medical student in the community. Thirty five second-year students were spread across nine primary care organisations and participated in real-life telemedicine consultations. Final year medical students from Kings College London [24] reported their insights on participating in virtual general practice clinics amidst the pandemic and how their practices as incoming junior doctors would continue.

Online clinical education

Pre-clinical & across all years

Some studies reported online delivery of clinical education for preclinical students [23,[25], [26], [27], [28]]. Knie et al. [27] implemented communication skills by interviewing simulated patients over Zoom, resulting in higher satisfaction levels than role-playing amongst themselves. Elengickal et al. [26] reported online critical community-based projects through an interdisciplinary approach and Shah et al. [28] described maximizing active and collaborative learning over online platforms.

Four medical schools described clinical educational programs across all curricular stages, including pre-clinical and clinical phase students [20,[29], [30], [31]]. A half century old course in Germany on ethics and professionalism involving real patients was successfully digitalised [30] and diagnostic reasoning and diagnosis training were delivered to medical students via an artificial intelligence–based virtual patient simulator developed by a medical school in Italy [29].

Patient seen in hospital

Most papers with real patient involvement were in the realms of outpatient telemedicine, but Pennell et al. [32] described live-streaming inpatient ward rounds as a clerkship educational tool. Medical students were engaged securely via mobile phone to participate in the ward round, including discussion before and after a patient visit. Sukumar et al. shared internal medicine clerkship experiences at a university hospital [33] in three steps; students pre-rounded an assigned hospitalised patient by remotely accessing their electronic health records, then calling into hospital rounds virtually, followed by oral presentation to their small group of instructors and fellow students. Kopp et al. [34] provided third-year medical students an opportunity to join inpatient medicine e-Consult teams which included COVID-19 patients. The orthopaedic department at the Chinese University of Hong Kong [35] set up a live-feed camera that allowed students to interact with consented patients in the online clinic and an attending physician demonstrated physical examinations techniques.

Teaching clinical skills or procedures

To compensate restricted real patient physical examination, tutors played the role of standardised patients and taught via video conferencing [13]. Schleicher et al. [36] taught musculoskeletal and neurological system examinations using models as well as demonstrating physical examinations on a second tutor via live video link. Student demonstration also took place in reverse with live feedback. In Jiang et al.’s [37] program, instructors played children’s parents’ roles and the students were able to take histories and propose necessary physical examinations via video link. Torres et al. [38] reported online clinical skills education by ‘borrowing’ the hand of a technician during medical simulation over zoom, where students guided the technician through a clinical examination on a fully articulating patient simulator. In a surgical skills study in Hong Kong, a synthetic skin and suture set was sent to each final year student, and 30 students per group joined an interactive tutorial. The students’ hands-on practice was monitored and assessed by the surgical tutor online [39].

Final year Italian students [40] underwent virtual reality online platform training with simulated clinical scenarios. Body Interact™ is one of the increasing virtual patient platforms that allows students to interact with virtual patients, to collect histories, perform physical examinations, request and view vital signs and laboratory images, administer medications and decide on interventions.

Clinical case scenarios discussion

Sixteen papers from an array of countries described the use of online clinical case scenarios to remedy the disruption in clinical clerkships for clinical year students [41,42,[51], [52], [53], [54], [55], [56],[43], [44], [45], [46], [47], [48], [49], [50]]. The University of Bergen’s radiology department [41] allowed fifth-year students to download the multiplatform image viewer on their personal computers and taught online practical skills on how to read and interpret CT examinations.

Online case discussions at a university hospital [51] was performed by students and residents together: third-year students produced a preliminary list of differential diagnoses; fourth-year students helped arrive their junior to the correct answer; and surgical residents provided examples of real-life experiences and evidence-based best practices to contextualize and enrich the patient presentations. An otolaryngology department [53] provided students online cases in response to the markedly reduced outpatient and surgical volume due to COVID-19. Not only video media, but auscultation of cardiac murmurs were incorporated into an interactive case-based online course [50]. To prevent disengagement during online clinical case discussions, Zottmann et al. [55] formally trained selective students in didactics of higher education encouraging fellow students during virtual teachings. At a general practice program, rural family physician preceptors led online education sessions on rural hospice and end-of-life care, obstetrical services and maternity care, approaches to opioid use disorder and medication-assisted treatment, and COVID-19 rural disaster preparedness and response [42].

Onsite clinical education

Whilst onsite modified clinical clerkships continued in parts of the world [[57], [58], [59], [60], [61]], in others it was combined with online teaching [28,62,63]. Thomas von Lengerke et al. [64] described communication skills teaching for preclinical students using simulated patients in a classroom setting under social distancing and infection control guidelines. Lee et al. [60] reported a modified paediatric clerkship in response to reduced patient volume and parents’ reluctancy to be seen by medical students. The University College London infectious diseases team [61] trialled an 8-week ‘COVID-19 apprenticeship’ for the graduating students to work as Doctors’ Assistants or Healthcare support worker assistants. The Doctors’ Assistant program was also positively reported by Lavender et al. [59] and Cowley and White [57] who reported the assistantships to form part of the mandatory medical school as successfully piloted in the UK.

Discussion

Authors of this scoping review aimed to provide practical information to help teachers effectively maintain clinical attachment programs for medical students when patient contact or clinical attachment become restricted due to public health crises. Our research has amalgamated knowledge on assorted online and onsite methods of clinical teaching over various delivery platforms adding to the body of evidence from previous reviews [1,2].

COVID-19 has been the spark that accelerated the implementation of already existing technology in medicine. Students being excluded from the clinical space led to many novel ideas from clinical teachers who used the available technology to facilitate real patient contact in the ward or outpatient setting [29,38,58,[60], [61], [62],65]. Even without real patients, the passion of clinical teachers has led them to become simulated patient themselves [36,37] and taught using clinical case scenarios allowing access to real patient imaging and EMR [18,19]. Albeit fewer in number, there were attempts at teaching clinical skills such as suturing, demonstrating that with advance preparations, basic technology and willingness, such education was possible over online platforms.

We identified telemedicine as an area of additional publication from previous reviews. Despite the exponential growth in telemedicine in the last decade, its enactment has been lukewarm until social distancing rules created the opportunity for its expansion in medical student education [13]. Iancu et al. [13] highlighted telemedicine education as an important aspect of future medical student training, especially in augmenting traditional in-person clinical experiences. Telemedicine engaged students and patients in diverse clinical settings providing a platform to teach wide range of clinical skills including clinical reasoning, professionalism, and communication skills during a pandemic [14,19] with high satisfaction levels [14,33]. The publications reporting clinical teaching via telemedicine during the COVID-19 could serve as a scalable model for future educational programs to be incorporated into the medical student curricula [21] and expanded into outreach programs, especially in underserved rural communities [19].

Although the papers showed that most clinical competencies could be learned online or in the virtual setting, concerns as per their effectiveness on clinical skills learning are abound. There is also a need for further outcome data collection in regards to the benefits or adverse effects of the new approaches in clinical teaching as our review has shown that reported educational outcomes stagnated at Kirkpatrick level 1 assessing only participants’ reactions or satisfaction to the educational programs.

Limitations

Our scoping review confined peer reviewed published papers in English, but grey literatures or conference proceedings were not included. These restrictions may affect the whole picture of mapping clinical teaching adaptations for medical students due to COVID-19, despite the widespread educational developments attempted to counteract the pandemic disruption across the globe. Evidence for substituting or effectively teaching physical examination online or virtual platforms could not be identified.

Strengths

Our analysis included papers up to March 2021 filling the gap in the literature and updating previous published reviews with the focus on clinical teaching for medical students. We have especially updated clinical educational developments in telemedicine, an area highlighted in a previous scoping review to be in urgent need for further research and development.

Conclusion

The new educational developments brought by the pandemic are still fraught with challenges and there is still lack of evidence in their educational effectiveness. However, this scoping review updated the educational strategies which could be applied by clinical educators in their practice to effectively maintain clinical attachment programs for medical students amidst public health crises. In particular, the increase in telemedicine related publications were remarkable, which may serve as a scalable model for future educational programs to be incorporated into the medical student curricula.

Conflict of interest

The authors declare that they have no conflict of interest.

Funding

There are no funding sources to declare.

CRediT authorship contribution statement

Hyunmi Park: Methodology, Validation, Formal analysis, Data curation, Visualization, Writing - original draft, Writing - review & editing. Sunhee Shim: Validation, Data curation, Formal analysis, Visualization, Writing - review & editing. Young-Mee Lee: Conceptualization, Methodology, Validation, Investigation, Data curation, Writing - original draft, Writing - review & editing, Supervision, Project administration.

Section	Item	PRISMA-ScR checklist item	Reported on page #
Title
Title	1	Identify the report as a scoping review.	1
Abstract
Structured summary	2	Provide a structured summary that includes (as applicable): background, objectives, eligibility criteria, sources of evidence, charting methods, results, and conclusions that relate to the review questions and objectives.	1
Introduction
Rationale	3	Describe the rationale for the review in the context of what is already known. Explain why the review questions/objectives lend themselves to a scoping review approach.	2
Objectives	4	Provide an explicit statement of the questions and objectives being addressed with reference to their key elements (e.g., population or participants, concepts, and context) or other relevant key elements used to conceptualize the review questions and/or objectives.	2
Methods
Protocol and registration	5	Indicate whether a review protocol exists; state if and where it can be accessed (e.g., a Web address); and if available, provide registration information, including the registration number.	3
Eligibility criteria	6	Specify characteristics of the sources of evidence used as eligibility criteria (e.g., years considered, language, and publication status), and provide a rationale.	4
Information sources	7	Describe all information sources in the search (e.g., databases with dates of coverage and contact with authors to identify additional sources), as well as the date the most recent search was executed.	4
Search	8	Present the full electronic search strategy for at least 1 database, including any limits used, such that it could be repeated.	24
Selection of sources of evidence	9	State the process for selecting sources of evidence (i.e., screening and eligibility) included in the scoping review.	4
Data charting process	10	Describe the methods of charting data from the included sources of evidence (e.g., calibrated forms or forms that have been tested by the team before their use, and whether data charting was done independently or in duplicate) and any processes for obtaining and confirming data from investigators.	4
Data items	11	List and define all variables for which data were sought and any assumptions and simplifications made.	5
Critical appraisal of individual sources of evidence	12	If done, provide a rationale for conducting a critical appraisal of included sources of evidence; describe the methods used and how this information was used in any data synthesis (if appropriate).	5
Synthesis of results	13	Describe the methods of handling and summarizing the data that were charted.	5
Results
Selection of sources of evidence	14	Give numbers of sources of evidence screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally using a flow diagram.	5
Characteristics of sources of evidence	15	For each source of evidence, present characteristics for which data were charted and provide the citations.	5
Critical appraisal within sources of evidence	16	If done, present data on critical appraisal of included sources of evidence (see item 12).	5−10
Results of individual sources of evidence	17	For each included source of evidence, present the relevant data that were charted that relate to the review questions and objectives.	5−10
Synthesis of results	18	Summarize and/or present the charting results as they relate to the review questions and objectives.	5−10
Discussion
Summary of evidence	19	Summarize the main results (including an overview of concepts, themes, and types of evidence available), link to the review questions and objectives, and consider the relevance to key groups.	10
Limitations	20	Discuss the limitations of the scoping review process.	12
Conclusions	21	Provide a general interpretation of the results with respect to the review questions and objectives, as well as potential implications and/or next steps.	12
Funding
Funding	22	Describe sources of funding for the included sources of evidence, as well as sources of funding for the scoping review. Describe the role of the funders of the scoping review.	1

Author	Title	Journal	Country	Year	Clinical/pre-clinical	Univ/Comm	Specialty	Online/onsite	Patient type	Teaching methods	KP	Type of publication	Study design & data collection
Adeleke et al. [62]	Opportunity for change: Undergraduate training in family medicine	S Afr Fam Pract	ZA	2020	Clinical	Univ	MS	Both	Real	Online self-directed learning, ward rounds	NA	Opinion	NA
Afonso et al. [25]	"I Have a Cough": An Interactive Virtual Respiratory Case-Based Module	MedEdPORTAL	US	2020	Pre-clinical	Univ	Med	Online	Stand /sim	Demonstration of percussion and auscultation, SP patient encounter SP for communication skills and “web-side” manner, faculty feedback, clinical reasoning and case discussion session	NA	Original article	NA
Bhatia et al. [16]	Transforming a clerkship with telemedicine	J Am Osteopath Assoc	US	2021	Clinical	Univ	FM	Online™	Real	Clinical case discussion, clinical skill teaching, lecture, patient interview	NA	Review	NA
Biermann et al. [41]	An Open Source Solution for “Hands-on” teaching of PET/CT	Nuklearmedizin	NO	2020	Clinical	Univ	Med	Online	Clinical scen	Online software for PET/CT teaching, e-lectures, diagnostic nuclear medicine viewing system, face to face seminar via Zoom	1	Original article	Cross-sectional; survey
Boulger and Onello [42]	Transforming Rural Family Medicine Curriculum From Experiential to Virtual: A Response to COVID-19 Limitations	PRiMER	US	2020	Clinical	Com	FM	Online	Clinical scen	Case discussion, journal presentation, panel discussion, lecture	1	Original article	Cross-sectional; survey
Cain et al. [19]	Telemedicine implementation in family medicine: Undergraduate clerkship during COVID-19 pandemic	Med Educ	US	2020	Clinical	Com	FM	Online™	Real	Students able to document the telemedicine clinic visit in patients' EMR and attested by attending	1	Letter	Cross-sectional; survey
Carson et al. [20]	Student Hotline Improves Remote Clinical Skills and Access to Rural Care	PRiMER	US	2020	Both	Com	FM	Online™	Real	Online education sessions on rural hospice, end-of-life care, obstetrical services and maternity care. Education on COVID-19 rural disaster preparedness and response.	1	Original article	Cross-sectional; survey
Co and Chu [39]	Distant surgical teaching during COVID-19 - A pilot study on final year medical students	Surg Pract	HK	2020	Clinical	Univ	Surg	Online	Stand /sim	Suture skills	1	Original article	Cross-sectional; survey
Coffey et al. [12]	Student Perspectives on Remote Medical Education in Clinical Core Clerkships During the COVID-19 Pandemic	Med Sci Educ	US	2020	Clinical	Univ	MS	Online™	Real	Synchronous case conference, dictating lecture, evaluation & progress review, telehealth experience	1	Original article	Cross-sectional; survey
Cowley and White [57]	Healthcare support worker assistantships should form a mandatory part of medical school curricula: A perspective from UK medical students	Med Teach	GB	2020	Clinical	Univ	Med	Onsite	Real	Working as Healthcare support worker assistants	NA	Opinion	NA
Darnton et al. [23]	Medical students consulting from home: A qualitative evaluation of a tool for maintaining student exposure to patients during lockdown	Med Teach	GB	2020	Pre-clinical	Com	FM	Online™	Real	Clinical communication and exam skills consulted from own home whilst supervised by primary care physician in three-way telemedicine clinic	1	Original article	Qualitative
Dawoud et al. [8]	Letter to the Editor “Virtual Neurosurgery Clerkship for Medical Students”	World Neurosurg	US	2020	Clinical	Univ	Surg	Online™	Real	Curated surgical video content on select neurosurgical procedures, Students joined telemedicine clinics observing and participating	NA	Letter	NA
De Ponti et al. [40]	Pre-graduation medical training including virtual reality during COVID-19 pandemic: a report on students' perception	BMC Med Educ	IT	2020	Clinical	Univ	Med	Online	Virtual patient	Case based scenarios	1	Original article	Cross-sectional; survey
DeVaro et al. [10]	Ophthalmology Education in COVID-19: A Remote Elective for Medical Students	J Acad Ophthalmol	US	2020	Clinical	Univ	Med	Online™	Real	Online self-directed learning, virtual patients, telemedicine clinics, case-based discussions	2a	Original article	Mixed method; survey; knowledge test
Dong et al. [67]	Clinical medical education: Cardiothoracic surgery in the era of COVID-19	J Card Surg	US	2021	Clinical	Univ	Surg	Online	Clinical scen	Small group teaching discussions after lectures	NA	Letter	NA
Elengickal et al. [26]	Adapting Education at the Medical College of Georgia at Augusta University in Response to the COVID-19 Pandemic: the Pandemic Medicine Elective	Med Sci Educ	US	2021	Pre-clinical	Univ	MS	Online	Clinical scen	Lecture, Service learning, online community-based projects	1	Short report	Cross-sectional; survey
Furlan et al. [29]	A Natural Language Processing–Based Virtual Patient Simulator and Intelligent Tutoring System for the Clinical Diagnostic Process	JMIR Med Inform	IT	2021	Both	Univ	MS	Online	Virtual patient	Computer programsimulating real-life clinical scenarios using Natural language processing	2b	Original article	Pre/post comparision; perf test(MCQ)
Geha and Dhaliwal [45]	Pilot virtual clerkship curriculum during the COVID-19 pandemic: Podcasts, peers and problem-solving	Med Educ	US	2020	Clinical	Univ	Med	Online	Clinical scen	Interactive videoconferences, resident-level case conferences, conversation-style medical podcasts	1	Short report	Cross-sectional; survey
Halbert et al. [58]	Clinical placements for medical students in the time of COVID-19	Med J Aust	AU	2020	Clinical	Univ	MS	Onsite	Real	Clinical clerkship was continued for the final 2 years	NA	Opinion	NA
Hayes et al. [17]	Building a Successful, Socially-Distanced Family Medicine Clerkship in the COVID Crisis	PRiMER	US	2020	Clinical	Univ	FM	Online™	Real	Telemedicine family medicine clerkship clinical encounters. Illness script assignments reviewed by attending & different disease templates shared with students via online drive	1	Short report	Cross-sectional; survey
Ho et al. [14]	Developing the eMedical Student (eMS)-A Pilot Project Integrating Medical Students into the Tele-ICU during the COVID-19 Pandemic and beyond	Healthcare	US	2021	Clinical	Univ	Med	Online™	Real	Student shifts on remote tele-ICU patient care, performing real interventions (medication, ventilator adjustment, lab recheck) as part of interdisciplinary care.	1, 2b	Original article	Cross-sectional; survey
Iancu et al. [13]	Twelve tips for the integration of medical students into telemedicine visits	Med Teach	US	2020	Clinical	Univ	MS	Online™	Real	12 tips: set stage, lay out expectations, assess & provide software, hardware & workflow to all. Pilot system, share information and educate faculty & students, collect feedback.	NA	Perspective	NA
Jiang et al. [37]	Twelve tips for teaching medical students online under COVID-19	Med Educ	CN	2021	Clinical	Univ	MS	Online	Stand /sim	Clinical case discussion, clinical skill teaching. Patient interview	NA	Review	NA
Kaliyadan et al. [46]	An Online Dermatology Teaching Module for Undergraduate Medical Students amidst the COVID-19 Pandemic	Indian Dermatol Online J.	IN	2020	Clinical	Univ	Med	Online	Clinical scen	Small group live interactive sessions, tutor acted as patient in history taking, relevant lab test discussed, student describe skin lesions in image & present tx plan	1	Short report	Cross-sectional; survey
Kasai et al. [63]	Alternative approaches for clinical clerkship during the COVID-19 pandemic	BMC Med Educ	JP	2021	Clinical	Univ	Med	Both	Stand /sim	Assigment of one surgical case for each studendent, LMS review, EHR upload, e-PBL discussion.	1	Original article	Mixed;Pre-& post survey; FGIs
Khalil et al. [47]	The sudden transition to synchronized online learning during the COVID-19 pandemic in Saudi Arabia: a qualitative study exploring medical students' perspectives	BMC Med Educ	SA	2020	Clinical	Univ	MS	Online	Clinical scen	Lectures, case discussions, 4-box case analysis, clinical case discussions, online seminars, and dry labs (online laboratory demonstrations).	1	Original article	Qualitative; FGIs
Knie et al. [27]	To zoom or not to zoom - the training of communicative competencies in times of Covid 19 at Witten/Herdecke University illustrated by the example of "sharing information"	GMS J Med Educ	DE	2020	Pre-clinical	Univ	MS	Online	Stand /sim	Clinical skill teaching, lecture, patient interview, interactive Q&A, role play	NA	Short report	NA
Kopp et al. [34]	Medical student remote eConsult participation during the COVID-19 pandemic	BMC Med Educ	US	2021	Clinical	Univ	Med	Online	Real	Students joined internal medicine eConsult teams seeing hospitallised COVID-19 patients	1	Original article	Cross-sectional; survey
Kunisch et al. [30]	Learning in peer teaching of patient relations and communication skills at the "Anamnesegruppen" Munich - proof-of-concept and lessons learned	GMS J Med Educ	DE	2021	Both	Univ	MS	Online	Real	Half century old medical ethics and professionalism course digitalised involving real patients	NA	Short report	NA
Kuo et al. [48]	Efficacy of vascular virtual medical student education during the coronavirus disease 2019 pandemic	J Vasc Surg	US	2021	Clinical	Univ	Surg	Online	Clinical scen	Lectures, video instruction of physical examinations or relevant surgical procedures. Virtual skills laboratory for suturing technique	NA	Letter	NA
Lal et al. [15]	Teaching Telepalliative Care: An Elective Rotation for Medical Students during the COVID-19 Pandemic	J Palliat Med	US	2021	Clinical	Univ	Med	Online™	Real	Flipped classroom over zoom, role-play with faculty assessing symptoms and disclosing prognosis. Outpatient tele-palliative care clinic participation	NA	Letter	NA
Lavender et al. [59]	Rising to the challenge: medical students as Doctors' Assistants; an evaluation of a new clinical role	J Adv Med Educ Prof	GB	2020	Clinical	Univ	MS	Onsite	Real	Lecture about DA's job via E-mail and directly practice as DA at the hospital	1	Original article	Cross-sectional; survey
Lee et al. [60]	New Paradigm of Pediatric Clinical Clerkship during the Epidemic of COVID-19	J Korean Med Sci	KR	2020	Clinical	Univ	Med	Onsite	Real + stand/sim	Outpatient clinic, case discussion	NA	Opinion	NA
Ludwig et al. [31]	Telephone-based communication training in the era of COVID-19	GMS J Med Educ	DE	2021	Both	Univ	MS	Online	Stand /sim	Conversation with relatives course converted into telephone converstaion with simulation subjects→remote participation to test students' doctor-patient conversation skills	NA	Short report	NA
Nackers et al. [68]	Patient care, public health, and a pandemic: adapting educational experiences in the clinical years	FASEB Bioadv	US	2021	Clinical	Univ	Med	Online	Real	Best practice guides for students and faculty & provision of telehealth platform access training	NA	Perspective	NA
Nic Dhonncha and Murphy [43]	Learning new ways of teaching and assessment: the impact of COVID-19 on undergraduate dermatology education	Clin Exp Dermatol	IE	2021	Clinical	Univ	Derm	Online	Clinical scen	Student-led case and topic presentations. Image based questions. Open book.	NA	Short report	NA
Ochsendorf et al. [49]	Corona pandemic: Teachings for dermatological teaching	J Dtsch Dermatol Ges	DE	2020	Clinical	Univ	Derm	Online	Clinical scen	Video vignettes of standard procedures, online dermatology courses, self-study slides, lecture videos, e-learning cases	NA	Letter	NA
Ong et al. [69]	Impact of COVID-19 on orthopaedic clinical service, education and research in a university hospital	J Orthop Translat	US	2020	Clinical	Univ	OS	Online	Real	Students' history taking and real-time management discussion via live-feed camera with patient and tutor. Physical examinations techniques & clinical signs demonstrations by tutors.	NA	Perspective	NA
Paul et al. [24]	Integration of Technology in Medical Education on Primary Care During the COVID-19 Pandemic: Students' Viewpoint	JMIR Med Educ	GB	2020	Clinical	Com	FM	Online™	Real	Community care medicine involved final year students in telemedicine clinics and provided online small group teachings	NA	Perspective	NA
Pellegrini et al. [9]	Medical Student Participation in Otolaryngology Telemedicine Clinic During COVID-19: A Hidden Opportunity	Otolaryngol Head Neck Surg	US	2020	Clinical	Univ	Surg	Online™	Real	1-on-1 attending and 4th yr med student telemedicine clinic from available exam room. Two computers to access electronic health records.	NA	Commentary	NA
Pennell et al. [32]	Live-streamed ward rounds: a tool for clinical teaching during the COVID-19 pandemic	Med J Aust	GB	2020	Clinical	Univ	O&G	Online	Real	Ward round, Clinial case discussions, Interactive Q&A	1	Original article	Cross-sectional; survey
Peterseim and Watson [21]	Family Medicine Telehealth Clinic With Medical Students	PRiMER	US	2020	Clinical	Com	FM	Online™	Real	Telehealth clinic shadowing family medicine attending or student led econsultation taking hisotry and describing physical exam manoeuvres to patient to perform on themselves.	2b	Original article	Case control comparsion; performance test(MCQ)
Roskvist et al. [56]	Provision of e-learning programmes to replace undergraduate medical students’ clinical general practice attachments during COVID-19 stand-down	Educ Prim Care	NZ	2020	Clinical	Com	FM	Online	Clinical scen	E-learning interactive, podcasts, videos, academic papers	NA	Short report	NA
Ruiz Colón et al. [22]	The COVID-19 Pandemic as an Opportunity for Operational Innovation at 2 Student-Run Free Clinics	J. Prim. Care Comm Heal	US	2021	Clinical	Com	FM	Online™	Real	Students performed history taking, delivery of patient education and counselling after receiving basic telehealth training	NA	Short report	NA
Rüllmann et al. [50]	Virtual auscultation course for medical students via video chat in times of COVID-19	GMS J Med Educ	DE	2020	Clinical	Univ	Var.	Online	Clinical scen	Virtual auscultation course via videochat	2a	Short report	Cross-sectional; self-assessed performance
Rupley et al. [11]	Mobilization of health professions students during the COVID-19 pandemic	Semin Perinatol	US	2020	Clinical	Univ	O&G	Online™	Real	Public health, telehealth visit & post-partum outreach teams ran by students	NA	Short report	NA
Schleicher et al. [36]	Training of physical examination techniques in video conferences	GMS J Med Educ	DE	2021	Clinical	Univ	MS	Online	Stand /sim	Clinical skill teaching, lecture, interactive Q&A	NA	Short report	NA
Shah et al. [28]	Beginning a new medical school curriculum amidst a global pandemic	FASEB Bioadv	US	2021	Pre-clinical	both	MS	Both	Real	Daily assignments and quizz, narrative reflection, feedback and review, virtual clinical rounds	NA	Perspective	NA
Nnamani Silva et al. [51]	Surgery Clerkship Curriculum Changes at an Academic Institution during the COVID-19 Pandemic	J Surg Educ	US	2021	Clinical	Univ	Surg	Online	Clinical scen	Extened mastery learning rotation(EMLR): case-based learning, combined with the virtual operating room, clinical science case-review, virtual patient contact	NA	Perspective	NA
Stout et al. [61]	Necessity is the mother of invention: how the COVID-19 pandemic could change medical student placements for the better	Postgrad Med J	GB	2021	Clinical	Univ	Med	Onsite	Real	LIC, Ward rounds, TBL	2a	Original article	Pre & post self-assessed conficence
Sukumar et al. [33]	Designing and Implementing a Novel Virtual Rounds Curriculum for Medical Students' Internal Medicine Clerkship During the COVID-19 Pandemic	MedEdPORTAL	US	2021	Clinical	Univ	Med	Online	Real	Remote EMR access and joining hospital rounds of assigned patients, oral presentations shared in small groups	1	Original article	Cross-sectional; survey
Torres et al. [38]	Transition to online is possible: Solution for simulation-based teaching during the COVID-19 pandemic	Med Educ	PL	2020	Clinical	Univ	MS	Online	Stand /sim	Clinical case discussion, clinical skill teaching, patient interview, interactive Q&A	NA	Short report	NA
Wagner-Menghin et al. [52]	Designing virtual patient based self-study quizzes covering learning goals in clinical diagnostic sciences for undergraduate medical students - the radiology example	GMS J Med Educ	AT	2020	Clinical	Univ	Rad	Online	Clinical scen	Clinical case discussion, quizz, webinars	1	Short report	Cross-sectional; survey
Weber et al. [18]	An outpatient telehealth elective for displaced clinical learners during the COVID-19 pandemic	BMC Med Educ	US	2021	Clinical	Univ	FM	Online™	Real	Virtual elective. Prior EMR training and introduction to workflow, document clinical information after encountering consented patients.	1	Original article	Cross-sectional; survey
Wickemeyer and Yu [53]	A Model for Undergraduate Medical Student Education in Otolaryngology During the Post-COVID-19 Era	Otolaryngol Head Neck Surg	US	2021	Clinical	Univ	Surg	Online	Clinical scen	Clinial case discussions, Interactive Q&A, Lectures	NA	Commentary	NA
Zeinali et al. [54]	Facing COVID-19, jumping from in- person training to virtual learning: A reviewoneducational and clinical activities in a neurology department	Basic Clin Neurosci	IR	2020	Clinical	Univ	Med	Online	Clinical scen	Virtual morning reports and joint virtual educational classes	NA	Letter	NA
Zottmann et al. [55]	Isn't here just there without a "t" - to what extent can digital Clinical Case Discussions compensate for the absence of face-to-face teaching?	GMS J Med Educ	DE	2020	Clinical	Univ	MS	Online	Clinical scen	Clinial case discussions	NA	Short report	NA