Application of digital education in undergraduate nursing and medical interns during the COVID-19 pandemic: A systematic review.

BACKGROUND: Due to the rapid spread of coronavirus disease 2019 (COVID-19) around the world, the World Health Organization (WHO) declared it a global pandemic on March 11, 2020. This declaration had an unprecedented impact on health profession education, especially the clinical clerkship of nursing and medical students. The teaching hospitals had to suspend traditional bedside clinical teaching and switch to digital education.
OBJECTIVE: To systematically synthesize the available literature on the application of digital education in undergraduate nursing and medical interns during the COVID-19 pandemic.
DESIGN: A systematic review informed by PRISMA guidelines. DATA SOURCES: Five electronic databases were systematically searched: PubMed, Embase, MEDLINE (OVID), CINAHL and the Cochrane Library. REVIEW
METHODS: The retrieved articles were screened at the title, abstract, and full text stages. The Mixed-Methods Appraisal Tool (MMAT) was used to assess the quality of quantitative and mixed-method studies. Then, two reviewers extracted the quantitative data of the included studies.
RESULTS: A total of 4596 studies were identified following a comprehensive search, and 16 studies were included after removing duplicates and screening, which focused on undergraduate nursing students (3 studies) and medical students (13 studies). We found that the standalone digital education modalities were as effective as conventional learning for knowledge and practice. Different educational technologies have different effects on the knowledge and practice of interns.
CONCLUSION: Digital education plays a significant role in distance training for nursing and medical interns both now and in the future. The overall risk of bias was high, and the quality of evidence was found to be variable. There is a need for further research designing more quasi-experimental studies to assess the effectiveness of standalone digital education interventions for the remote training of nursing or medical interns to be fully prepared for emergencies.

Introduction

Due to the rapid spread of coronavirus disease 2019 (COVID-19) around the world, the World Health Organization (WHO) declared it a global pandemic on March 11, 2020 (Dao et al., 2021). As of June 2021, there have been more than 3.8 million deaths and 170 million confirmed cases in 222 countries and territories. The pandemic has caused enormous economic and social upheaval internationally; furthermore, it has had an immeasurable impact on the delivery of health profession education worldwide (Hodgson et al., 2021; Seymour-Walsh et al., 2020a). Therefore, to protect students' education and wellbeing, the Ministry of Education of the People's Republic of China issued a notice postponing the start of the school year, and the Association of American Medical Colleges (AAMC) recommended that health professions schools pause all clinical practicums (Harries et al., 2021).

Clinical instructional experiences and exposure to real patients are essential elements of nursing education and an indispensable part of personal medical education and development (AlThiga et al., 2017; Mandan et al., 2016). However, with the rapid spread of COVID-19, nationwide lockdown restrictions have had a strong impact on medical and nursing students' access to patients as well as medical staff (Dost et al., 2020). The resulting passive form of the clinical practicum has led to issues in the quality of medical education. Thus, the faculty preceptors had to urgently fill the gaps in the clinical practicum and complete the mission of cultivating qualified future health care workers. Given that the pandemic is a moment to promote the culture of professionalism and altruism and can present a once-in-a-lifetime disaster response learning experience for medical and nursing students, some medical schools encouraged their students to work on the front line during the COVID-19 pandemic (Lee et al., 2020; Leigh et al., 2020). However, shortages of personal protective equipment (PPE), limited availability of COVID-19 testing and imperfect infection control schemes have all been major safety concerns (Wang et al., 2020). Thus, digital education without security risks may be an alternative to clinical education that can be implemented even during the lockdown (Park et al., 2020).

Digital education is the act of teaching and learning via digital technologies (Car et al., 2019), including diversified teaching methods such as offline learning, online learning, serious games, mobile learning or virtual reality (VR) (George et al., 2014; Kononowicz et al., 2019). Different digital education methods have different characteristics, mainly embodied in technical preparation, measurement tools, delivery settings and learning approaches (Kyaw et al., 2019). Over the past 50 years, digital education has been widely used in health professional education (Triola et al., 2012), and may provide a more flexible, accessible and affordable alternative to traditional learning (Tudor Car et al., 2019b). Numerous published studies have evaluated the potential benefits of digital education for health professional education (Cook et al., 2008), for instance, without the limitations of time, resources and geography (Brusamento et al., 2019). The World Health Organization (WHO) has also affirmed its significant contribution to filling the gap between low- and high-income regions in medical education (Semwal et al., 2019).

Health professional education has been severely influenced by the novel coronavirus, but this emergency has fortunately provided an impetus for the innovation of teaching formats (Dedeilia et al., 2020; Seymour-Walsh et al., 2020b). Distance training has undoubtedly been one of the feasible teaching reform measures. A previous review evaluated the effectiveness and potential benefits of digital education in health professional education (Curran et al., 2017), but it was conducted before the COVID-19 pandemic. Therefore, the authenticity of the results may inevitably be interfered with by traditional offline teaching. In addition, some reviews synthesized data on medical or surgical education during the period of COVID-19, but they never took nursing education into consideration (Dedeilia et al., 2020; Wilcha, 2020). To the best of our knowledge, no systematic review has examined digital education as an alternative to training medical and nursing interns during the novel coronavirus pandemic. Therefore, this review filled a gap in the literature. We not only assessed the effectiveness of digital education but also evaluated its impact on the knowledge, attitudes, skills, and satisfaction of nursing and medical students.

Methods

Design

This systematic review is a critical appraisal of all available evidence to evaluate the substitutability of digital education to bedside clinical practicum during the pandemic. In this study, the Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) was adopted to inform reporting of the search and results, owing to its clarity and transparency (Moher et al., 2009).

Data sources

A systematic search of the following five electronic databases was performed in April 2021: PubMed, Embase, MEDLINE (OVID), CINAHL, and the Cochrane Library. The databases were searched for studies published between 2019 and April 2021. A combination of medical subject headings (MeSH) and free text terms were utilized to generate a systematic search strategy, which was developed and adjusted for use across databases (Kerr et al., 2020). Boolean connectors AND and OR were used to maximize saturation of terms searched (Cleary et al., 2018). The keywords used to search relevant literature included “nurs* undergraduate” OR “medical student*” AND “clinical clerkship” AND “SARS-CoV-2” and “digital*” OR “e-learn*”. An example of the search strategy for the PubMed database is shown in Supplementary File. Additionally, we adopted additional methods to improve the comprehensiveness of the retrieval, including checking the reference list of included studies and citation tracking (Wu et al., 2018).

Inclusion and exclusion criteria

We adopted specific screening criteria to ensure the relevance of the included articles. The inclusion criteria were as follows: (1) quantitative and mixed-method primary studies published from 2019 to April 2021; (2) the study samples were restricted to undergraduate nursing students or medical students participating in the clinical practicum; (3) studies describing digital education as a principal teaching alternative; (4) studies conducted during the COVID-19 pandemic; and (5) English language publications. The exclusion criteria were as follows: (1) the majority of the samples were not undergraduate nursing students or medical students; (2) papers in which the background was not relevant to the COVID-19 pandemic; and (3) qualitative articles, papers not written in English, and those in which only the abstract was available.

Critical appraisal

The Mixed Methods Quality Appraisal Tool (MMAT) revised by Pace et al. (2012) in 2011 was used to assess the risk of bias of studies with quantitative and mixed methods designs. MMAT is an effective and practical quality assessment tool developed for systematic mixed studies reviews and has been utilized in previous systematic reviews (Humphries et al., 2014; Ivanova et al., 2018; Pieper et al., 2013). This tool includes two screening questions for all research types and three to four questions for each type. The quality of the included studies was scored as “Yes”, “No”, or “Can't tell” according to whether the research met the criteria. The overall quality score is equal to the number of criteria met (Yes) divided by the total number of related questions and multiplied by 100 (Mącznik et al., 2015). Two researchers independently assessed the quality of the full texts, and any bifurcations were rigorously discussed until a consensus was reached. Given the limited number of studies, we decided to take an inclusive approach to include all eligible studies in this review.

Data extraction and synthesis

We extracted the quantitative data from the included studies into a self-developed data extraction form to gather details about the author, location, year of publication, study design, characteristics of the study population, specific teaching methods, assessment methods or instruments, main evaluation content and main findings. Meta-analysis was not appropriate given the heterogeneity of the digital teaching methods, study designs and outcome measurements identified. Therefore, systematic review findings are presented in narrative format.

Results

Included studies

A search of the selected databases yielded 4596 articles, with 16 remaining after the four-stage article screening according to the guidelines of Moher et al. (2009) (Fig. 1 ). Table 1 summarizes the main characteristics of the 16 included studies, which were conducted between 2020 and 2021. Studies were conducted in the United States of America (n = 7), China (n = 3), the UK (n = 1), Japan (n = 1), Korea (n = 1), Italy (n = 1), Arabia (n = 1), and Israel (n = 1). The sample sizes in the included papers ranged from 10 to 211 students. A total of 1174 participants were enrolled in this review, of whom 457 were undergraduate nursing students (3 studies) and 717 were medical students (13 studies). Only one of these studies contained a mixture of educational levels (He et al., 2021), such as undergraduate and PhD students in medicine. However, we only extracted data related to undergraduate students. The 16 papers used a variety of methodologies, including one cross-sectional study, two quantitative descriptive studies, two quasi-experimental studies, one prospective cohort study and the rest of the ten mixed-method studies. The publications were devoted to exploring the different modalities of digital education, including virtual reality-based simulation training (3 studies) (De Ponti et al., 2020; Kang et al., 2020; Weston and Zauche, 2021), teleconsultation and virtual rounds (3 studies) (Bala et al., 2021; Sukumar et al., 2021; Weber et al., 2021), web-based specialized skills learning (2 studies) (Alpert et al., 2021; Shahrjerdi et al., 2020), and multimodal online curriculums (8 studies) (Coffey et al., 2020; He et al., 2021; Kaliyadan et al., 2020; Kasai et al., 2021; Michener et al., 2020; Samueli et al., 2020; Williams et al., 2021; Zhou et al., 2020). Study outcomes were measured utilizing a series of homemade tools. For instance, standardized questionnaires, scales, anonymous surveys and Likert scales were used. In addition, academic performance was assessed in 3 studies in terms of examinations, and one study used specialized instruments with high reliability and validity to evaluate learning outcomes.

Fig. 1

Summary of the search results and selection process.

Table 1

Basic characteristics and results of included studies.

Author, location (year)	Study design	Participants (undergraduate interns)	Other participants	Digital teaching methods	Quantitative methods to assess teaching effect	Evaluation contents	Main findings
Weston and Zauche, US (2020)	A cross-sectional study	N = 186，traditional and second-degree nursing students (second semester)	–	virtual simulation (VS)	ATI Nursing Care of Children examination	The ATI score	There were no significant differences in ATI scores between students who completed their pediatric clinical practicum in the clinical setting compared with virtually
Shahrjerdi et al., Hong Kong (China) (2020)	A quantitative descriptive study	N = 30, medical students, males: 14, famales:16 median age:23 (22–24) right-handed: 28 (93.4%)	–	A new web-based surgical skills learning (WSSL)	A standardized questionnaire	Ease of acquiring basic surgical skills; The degree of recommendation of the new web-based surgical teaching format	96.7% of the students believed that online demonstration was clear; Most students felt that WSSL was as difficult/easy as conventional face-to-face teaching; 90% of the students recommended the WSSL format
Samueli et al., Israel (2020)	A mixed-method study	N = 59, medical students (M3 and M4)	–	Whole slide images (WSI)	An anonymous questionnaire	Learning experience; Understanding of the content; Technical challenges	The course was rated very favorably by the students; The online format seems to be preferable to students; The single worst disadvantage of the class were technical challenges in accessing the slides
Kaliyadan et al., Saudi Arabia (2020)	A mixed-method study	N = 45, medical students (M4 and M5)	N = 4, faculty	An online module (Powerpoint presentations, videos, quizzes, live interactive sessions in small groups)	A structured questionnaire	The general experience and satisfaction levels with the online teaching module	Both students and faculty gave a lower score for practical skills training and assessment, and gave high scores in overall content coverage and technical aspects
He et al., China (2021)	A quantitative descriptive study	N = 55, medical students (clinical medicine, n = 47 and psychiatry, n = 8)	N = 37, doctor of medicine	Online neurology training course based on Small Private Online Course (SPOC) and blending learning mode	An online questionnaire and the final score	The interns' satisfaction; The recommendation of the incorporation of the online course into the future intern training mode after the crisis	No significant difference was detected between the final scores of different groups of interns; All students were satisfied with the online course; The interns might have a better learning experience with a smaller learning group Size;
Roberto et al., Italy (2020)	A mixed-method study	N = 122, medical students (M6)	–	A virtual reality platform (21 patient-based clinical scenarios)	A 12-item questionnaire	The perceived quality of this training modality such as clarity, coherence, and relevance of items	The perceived quality of this training modality was considered satisfactory; 85% of the students considered this training modality useful also in the absence of potential obstacles to traditional medical training
Zhou et al., China (2020)	A quasi-experimental study	N = 60, nursing studentsIG: male: 1 females: 29 years: 22.06 ± 0.56CG: male: 1 females: 28 years: 22.10 ± 0.63	–	IG: combined mode of massive open online course (MOOC) micro-video	Examination of theoretical knowledge; Simulation examination of practice dialogue; An anonymous questionnaire	The total score (theory and practice); Teaching satisfaction	There was no significant difference between the two groups in terms of theory and practice; Compared with the CG, the interns' teaching satisfaction of IG was higher in multiple dimensions
Kang et al., Korea (2020)	A quasi-experimental study	N = 211, nursing studentsResponse rate: 91.0% (192)males: 38 females: 154years: 22.34 ± 1.88 (20−30)	–	Group 1 (G1): lecture and VsimGroup 2 (G2): lecture and high fidelityGroup 3 (G3): lecture and Vism and high fidelity	The knowledge scale of nursing care for children with asthma; Modified confidence tool for clinical performance; Modified clinical performance tool	Knowledge level; Confidence in practice (CP); The ability to care for children with asthma	Knowledge (*): G1 > G2, G3 > G2CP (**): G1 > G2, G3 > G2Total performance (**): G2 > G1, G3 > G1 Assessment (**) and education (**): G2 > G1, G3 > G1
Alpert et al., US (2020)	A prospective cohort study	N = 83, medical studentsResponse rate: 81.9% (68)Cohort 1: n = 27 Cohort 2: n = 41	–	Cohort 1: Conventional in-person general diagnostic radiology courseCohort 2: Remote course with Virtual Read-Out sessions (VRO)	A five-point Likert scale of perceived frequency	Students' sense of involvement in reviewing radiology exams; Technical limitations they encountered; Educational value of the learning experience	Remote students reported a higher frequency of interaction than Conventional students; There were no technical limitations to remote learning; Conventional students reported a sense of boredom at a higher frequency than remote students;
Williams et al., US (2021)	A mixed-method study	N = 10, medical students	N = 20, faculty	A combination of asynchronous and synchronous courses	A pre- and post-course questionnaire	Assessing participants perceptions of urology	By the end of the curriculum, median scores significantly improved in every topic area; At the completion of the course, 2 students reported a shift of specialty commitment to Urology
Weber et al., US (2021)	A mixed-method study	N = 64, medical studentsResponse rate: 45.3% (29)	–	Teleconsultation	A questionnaire	Experience gained in telehealth technologies; Trends in the number of patient encounters; The barriers in telehealth	Students gained substantial experience with telehealth technologies; The percentage of patients successfully transitioned to a Doxy.me virtual encounter trending higher over time
Sukumar et al., US (2021)	A mixed-method study	N = 29, medical studentsResponse rate: 48% (14)	N = 34, volunteer tele-instructorsResponse rate:74% (25)	Virtual Rounds	A five-point Likert scale	Student perception of changes to clinical abilities and confidence and evaluation of curricular elements; Tele-instructor perceptions of ability to teach, student learning and the personal impact	The majority of students perceived improvement in their clinical abilities; Most students believed that virtual rounds curricular was useful; Over two-thirds of tele-instructors felt that their overall ability to teach on VR was better than during in-person rounds
Michener et al., US (2020)	A mixed-method study	N = 34, medical studentsResponse rate: 67.6% (23)	–	A multimodal virtual geriatrics elective	An optional and anonymous survey	The quality of virtual geriatrics elective; The influence of different course modalities on learning effects	95.6% of respondents agreed that the course was well organized and objectives were clear; The majority of respondents believed that the different course modalities could enhance learning
Kasai et al., Japan (2021)	A mixed-method study	N = 43, medical students (respiratory unit: n = 22; general medicine: n = 21)Respond rate: 100%	–	Online education for clinical practice (online-sCP) (sECR, e-PBL, Online-VMIs)	A questionnaire	Clinical skills competence; Self-evaluation of participants' medical performance; The difference of self-study time in different stages;	All students indicated improvement for all aspects of clinical performance; Students' satisfaction level with the online-sCP conducted by each department was acceptable; Self-study time was longer during online-sCP than during standard CCs or the period when CCs was postponed
Coffey et al., US (2020)	A mixed-method study	N = 132, medical studentsResponse rate: 73% (96)	–	Remote curriculum	An anonymous, optional survey	Student utilization of remote learning resources; Students' assessment of the quantity and structure of remote learning curriculum; Preparedness for subsequent stages of training;Living Environments and Connectedness	73% of the respondents felt that the number of remote learning resources was “about right.”The majority of students felt that the quantity of required content and the degree of structure was appropriate; 27% of the students felt prepared for the shelf examination
Bala et al., UK (2021)	A mixed-method study	N = 11, medical studentsMales: 2, females: 9	N = 2, patientsMale:1 female: 1N = 2, faculty	Remote access teaching ward round	An anonymous questionnaire	The feasibility, acceptability and effectiveness of remote access teaching ward rounds	All students agreed that MR technology enabled access to clinical teaching that was otherwise not feasible secondary to the COVID- 19 pandemic; The majority agreed that the quality of the holographic content was adequate; The majority of students agreed that they could interact with the clinician and have their questions answered

ATI: Assessment Technologies Institute; M3: Students in their third year; M4: Students in their fourth year; M5: Students in their fifth year; M6: Students in their sixth year; IG: Intervention group; CG: Control group; Vsim: Virtual reality simulation; *: P < 0.05; **: P < 0.001; sECR: Simulated electronic health records; e-PBL: Electronic problem-based learning; Online-VMIs: Online virtual medical interviews; CCs: clinical clerkships; MR: Mixed reality.

Methodological quality of included studies

Methodological quality, as appraised by the MMAT exercise for the included 16 articles, ranged from 25% to 100%, with one study rated at 100% (Kang et al., 2020); two studies rated at 75% (Alpert et al., 2021; Weston and Zauche, 2021); five studied rated at 50% (Coffey et al., 2020; He et al., 2021; Kasai et al., 2021; Shahrjerdi et al., 2020; Zhou et al., 2020), and the remaining eight studies rated at 25% (Bala et al., 2021; De Ponti et al., 2020; Kaliyadan et al., 2020; Michener et al., 2020; Samueli et al., 2020; Sukumar et al., 2021; Weber et al., 2021; Williams et al., 2021) (Table 2 ).

Table 2

Results of quality assessment based on the Mixed Methods Appraisal Tool (MMAT-2011) for qualitative, quantitative and mixed methods studies.

Author, year	Study design	S1	S2	1.1.	1.2.	1.3.	1.4.	3.1.	3.2.	3.3.	3.4.	4.1.	4.2.	4.3.	4.4.	5.1.	5.2.	5.3.	Score
Weston et al., 2020	A cross-sectional study	Y	Y					Y	Y	N	Y								75%
Shahrjerdi et al., 2020	A quantitative descriptive study	Y	Y									N	N	Y	Y				50%
Samueli et al., 2020	A mixed-method study	Y	Y	Y	N	N	N					N	N	Y	N	N	N	N	25%
Kaliyadan et al., 2020	A mixed-method study	Y	Y	Y	N	N	N					N	N	Y	UC	N	N	N	25%
He et al., 2021	A quantitative descriptive study	Y	Y									N	N	Y	Y				50%
Roberto et al., 2020	A mixed-method study	Y	Y	Y	N	N	N					N	N	Y	Y	N	N	N	25%
Zhou et al., 2020	A quasi-experimental study	Y	Y					N	Y	N	Y								50%
Kang et al., 2020	A quasi-experimental study	Y	Y					Y	Y	Y	Y								100%
Alpert et al., 2021	A prospective cohort study	Y	Y					Y	Y	N	Y								75%
Williams et al., 2021	A mixed-method study	Y	Y	Y	N	N	N					N	N	Y	Y	N	N	N	25%
Weber et al., 2021	A mixed-method study	Y	Y	Y	N	N	N					N	N	Y	N	N	N	N	25%
Sukumar et al., 2021	A mixed-method study	Y	Y	Y	Y	N	N					N	N	Y	Y	N	N	N	25%
Michener et al., 2020	A mixed-method study	Y	Y	Y	N	N	N					N	N	Y	Y	N	N	N	25%
Kasai et al., 2021	A mixed-method study	Y	Y	Y	Y	N	N					N	N	Y	Y	N	Y	N	50%
Coffey et al., 2020	A mixed-method study	Y	Y	Y	N	N	N					N	N	Y	Y	N	Y	N	50%
Bala et al., 2021	A proof-of-concept studyA mixed-method research	Y	Y	Y	N	N	N					N	N	Y	Y	N	N	N	25%

Y: Yes; N: No; UC: Unclear; S1: Are there clear qualitative and quantitative research questions (or objectives*), or a clear mixed methods question (or objective*)? S2: Do the collected data allow address the research question (objective)? For score, the higher score indicates the better of the research quality. 1.1.: Are the sources of qualitative data (archives, documents, informants, observations) relevant to address the research question (objective)? 1.2.: Is the process for analyzing qualitative data relevant to address the research question (objective)? 1.3.: Is appropriate consideration given to how findings relate to the context, e.g., the setting, in which the data were collected? 1.4.: Is appropriate consideration given to how findings relate to researchers' influence, e.g., through their interactions with participants? 3.1.: Are participants (organizations) recruited in a way that minimizes selection bias? 3.2.: Are measurements appropriate (clear origin, or validity known, or standard instrument; and absence of contamination between groups when appropriate) regarding the exposure/intervention and outcomes? 3.3.: In the groups being compared (exposed vs. non-exposed; with intervention vs. without; cases vs. controls), are the participants comparable, or do researchers take into account (control for) the difference between these groups? 3.4.: Are there complete outcome data (80% or above), and, when applicable, an acceptable response rate (60% or above), or an acceptable follow-up rate for cohort studies (depending on the duration of follow-up)? 4.1.: Is the sampling strategy relevant to address the quantitative research question (quantitative aspect of the mixed methods question)? 4.2.: Is the sample representative of the population understudy? 4.3.: Are measurements appropriate (clear origin, or validity known, or standard instrument)? 4.4.: Is there an acceptable response rate (60% or above)? 5.1.: Is the mixed methods research design relevant to address the qualitative and quantitative research questions (or objectives), or the qualitative and quantitative aspects of the mixed methods question (or objective)? 5.2.: Is the integration of qualitative and quantitative data (or results*) relevant to address the research question (objective)? 5.3.: Is appropriate consideration given to the limitations associated with this integration, e.g., the divergence of qualitative and quantitative data (or results*) in a triangulation design?

The modality of digital education

Virtual reality-based simulation training

Three studies reported the use of virtual reality-based simulation training as an alternative to bedside clinical practice. Two of them focused on nursing students (Kang et al., 2020; Weston and Zauche, 2021), and one paid attention to medical students (De Ponti et al., 2020). Virtual simulation platforms such as i-Human® and Body Interact™ were sufficiently utilized for case-based learning to present students with an interactive medical patient experience. In addition, one quasi-experimental study included two additional intervention groups in which high-fidelity simulation (HFS) training and collaborative teaching combining HFS and virtual simulation training were conducted for nursing students (Kang et al., 2020). The three studies assessed the interns' Assessment Technologies Institute (ATI) scores, the perceived quality of this training modality, the level of knowledge, confidence in practice pre- and post-training and the aggregate performance. One study compared the students who were in clinical practice with those who participated in virtual simulation and reported that there were no significant differences in ATI scores (Weston and Zauche, 2021). The study focused on medical students indicated that more than half of them preferred simulation training with virtual platforms rather than online formal teaching, although technical troubles may inevitably emerge when accessing the online platform (De Ponti et al., 2020). The study included additional educational modalities, such as HFS and lectures, and showed that virtual reality-based simulation training could further enhance the knowledge level and confidence in clinical practice for the interns compared with high-fidelity simulation. However, the promoting effect of simulation training on overall clinical performance was slightly inferior (Kang et al., 2020).

Teleconsultation and virtual rounds

For the purpose of maintaining the continuity of the clinical practicum for medical students, three studies reconnected the interns with clinicians and patients through telemedicine, including teleconsultation and virtual rounds. The interns performed real-time interaction with patients scheduling a telemedicine appointment via Doxy.me video calls during the teleconsultation to recover the internship experience furthest (Weber et al., 2021). This study revealed that students obtained substantial experience with telemedicine technologies; for example, they could make a personal connection with patients and determine patients' perspective of their illness independently. The remaining two studies used Zoom video conference and Microsoft Dynamic 365 Remote Assist Application to interact with patients during virtual rounds. One study demonstrated that the clinical ability and professional confidence improved in the majority of the students and that the overall ability to teach tele-instructors in virtual rounds was better than that in in-person rounds (Sukumar et al., 2021). Beyond that, this study found that smaller groups were better for teaching and further affirmed the value of virtual rounds after the release of isolation. However, while another study using mixed reality (MR) technology suggested that virtual rounds were acceptable and effective, it denied its significance secondary to the COVID-19 pandemic (Bala et al., 2021).

Web-based specialized skills learning

Due to social distancing mandates, it is an urgent requirement for medical colleges to explore new approaches to learning clinical technical skills. Two studies were conducted in this general environment and mainly developed online web-based surgical skills learning (WSSL) and virtual read-out (VRO), which included learning groups composed of a faculty member and several students reviewing selective lists of imaging exams. One study used Zoom video conferencing for online presentation, focusing on surgical instruments and hand knots (Shahrjerdi et al., 2020). In this study, most of the students believed the new web-based surgical instruction was as difficult/easy as conventional face-to-face teaching, and they considered that this new teaching format was worthy of recommendation. Another study that performed remote radiology review by means of WebEx, a teleconferencing and screen sharing application, gauged the learning experience of two groups of interns (in-person group and VRO group) in reviewing radiology exams (Alpert et al., 2021). The results showed that remote training was more entertaining and that students tended to play an active role in reviewing radiology exams.

Multimodal online curriculums

A large number of medical colleges and their affiliated teaching hospitals developed multimodal online curricula to supersede face-to-face clinical internships during the novel coronavirus pandemic. In this review, 8 studies conducted online curriculums consisting of synchronous and asynchronous content and compared them with no controls. Asynchronous content included online videos, massive open online courses (MOOCs), discussion posts and so on. Synchronous content contained virtual conferences, impromptu role playing and lectures. Canvas, an online learning management system, was responsible for hosting asynchronous content. There were a variety of asynchronous platforms, such as Zoom, Tencent Class, and Blackboard®.

Three of the studies assessed the preference of the students for the different curriculum formats, and the results suggested that students had a greater interest in interactive discussion learning patterns and were less interested in passive teaching resources (Coffey et al., 2020; He et al., 2021; Michener et al., 2020). In addition, two of the three studies indicated that the majority of students were satisfied with multimodal online courses (He et al., 2021; Michener et al., 2020). One study found the same result as Sukumar et al. (2021), that a smaller learning group size might bring a better learning experience for interns (He et al., 2021). Additionally, this study also showed that an overwhelming majority of students recommended continuing the online curriculum after the pandemic. Another study found that interns gradually reconnected with school and their peers by taking online curricula (Coffey et al., 2020).

Two studies involved impromptu role-playing instruction, in which a faculty member played the role of a patient and a student acted as a doctor to simulate real clinical scenarios (Kaliyadan et al., 2020; Kasai et al., 2021). In addition, these two studies also separately included short presentations (PPT courseware), case-based teaching and other curriculum formats. One study indicated that both faculty and interns gave good feedback in terms of the overall content coverage and technical aspects of this online module, but they were less satisfied with practical skills training and assessment (Kaliyadan et al., 2020). Another study showed that interns had an acceptable satisfaction level with online simulated clinical practice (online-sCP); however, bedside practice was more useful for learning associated with physical examinations, medical interviews and humanistic qualities such as professionalism than online-sCP.

The remaining three studies varied in curriculum formats and assessment contents (Samueli et al., 2020; Williams et al., 2021; Zhou et al., 2020). One study training the diagnostic capability of medical students by browsing and reviewing slide sets of surgical biopsies found that the online format seemed to be preferable to students (Samueli et al., 2020). In addition, it is worth noting that the single worst disadvantage of the online curriculums was the technical challenges when accessing the slides. Another study with a massive open online course (MOOC) and microvideo as the major nursing teaching approach proved that there was no significant difference in the scores of the examination of theoretical knowledge and practice dialogue between interns who took face-to-face internships and those who participated in remote curriculums (Zhou et al., 2020). More interestingly, those who took the online curriculums reported higher levels of satisfaction with their internship. The last remaining study showed that interns' confidence in their urology practice and knowledge level increased significantly after the online curriculums (Williams et al., 2021).

Discussion

In this systematic review, we synthesized recent evidence on the effectiveness of thoughtful implementation of digital education for nursing and medical interns during the COVID-19 pandemic. Due to the limitations of the research design, only a few studies have compared digital education either with conventional face-to-face internships or with another form of educational technology. We found that for knowledge and practice, the standalone digital education modalities were as effective as conventional learning, which was consistent with previous research (Dunleavy et al., 2019). By comparing two different modalities of digital education, we found that different educational technologies have different effects on the knowledge and practice of interns. In addition, most studies examined in this review demonstrated the generally positive impact of digital education on the included interns in terms of skills, attitudes, satisfaction and other aspects. However, there was a high risk of bias in this part of the studies due to the deficiency of control groups.

There were some meaningful findings in terms of learning group size and teaching resources. First, a smaller learning group size might exert a better clinical teaching effect compared with a larger group size. A published study indicated that group size was a significant predictor of satisfaction with clinical experience for interns (Ofei-Dodoo et al., 2018), which was similar to our findings. Smaller groups help students dig deeper into what they are exploring and enable greater participation, collaboration and discussion (Bristol and Kyarsgaard, 2012; Burgess et al., 2017). Therefore, even though online teaching modalities allow a large number of users to attend a virtual space concurrently, the faculty should try to avoid centralized teaching where possible. Second, we found that interns had a greater interest in interactive training than passive access to resources. Similar findings have been found in previous reviews, indicating that interactive education was more effective than preaching education, which had little or no impact on professional practice (Bloom, 2005; Forsetlund et al., 2009). One of the possible reasons is that the motivation of learning can be heightened by an interactive style of teaching (Dolmans et al., 1998). In addition, the instantaneous feedback provided in interactive teaching helps students learn more effectively and internalize the material (Shapiro et al., 2014). Thus, it is a feasible and effective teaching reform measure to integrate interactive training into conventional didactic teaching to promote better learning after the COVID-19 epidemic is easing.

However, some deficiencies of digital education that played a significant role during the period of social isolation are worth noting. First, the technical challenges in distance access were the common shortcomings of most digital education modalities, which has also been confirmed in published studies (Hodgson and Hagan, 2020; Sleiwah et al., 2020). In this review, nearly half of the studies were conducted in developing countries, so costs such as computers and IT equipment might be one of the barriers to use. In addition, due to the outbreak of COVID-19, faculty members might have difficulty coping with the online delivery mode, so there was limited direction on how to implement such tools or programs (O'Doherty et al., 2018; Sahu, 2020). Therefore, a clear institutional strategy is recommended when implementing digital learning to reduce technical barriers to access (O'Doherty et al., 2018). Second, while most studies gave high praise for digital education and affirmed its value after the pandemic, they consistently believed that bedside clinical teaching was irreplaceable. Conventional clinical training and interaction with tutors was invaluable (De Ponti et al., 2020). Research has suggested that there might not be a competitive relationship between digital education and face-to-face teaching in some respects (Pei and Wu, 2019). Therefore, it may be worthwhile to combine the advantages of digital and face-to-face teaching methods to maximize the benefits (Pei and Wu, 2019).

The global scientific community has made great progress in combating COVID-19 (Chakravarty, 2021). With the advent of vaccines against COVID-19 and the implementation of social isolation measures worldwide, we have entered a period of relaxation of restrictions, during which the national economy and health education have been temporarily restored. In accordance with the teaching arrangements, we have resumed face-to-face teaching and ward rotation in a limited form (Khoo et al., 2021); additionally, digital education seems to continue to play a significant role in health professional education. However, there are still potential risks, such as gene mutation in SARS-CoV-2, which is one of the challenges that vaccine development has to face (Silveira et al., 2021). In addition, the popularization of vaccines also has insurmountable bottlenecks, including mass production and delivery of billions of doses to the worldwide population (Izda et al., 2021). As a result, the development of SARS-CoV-2 resistance to vaccines and unexpected outbreaks are highly possible in the near future (Dos Santos, 2021), which will lead to a new wave of digital learning in health profession education. Digital education is vital to remote training for interns both now and in the future. Nevertheless, it is worth noting that the ethical issues of patient privacy and data security are often ignored in the implementation of digital education, which may compromise the welfare of patients (Lall et al., 2019). To better apply it to clinical training, teaching hospitals should further standardize the specific implementation guidelines of digital education.

There are some limitations to the evidence evaluated in this review. First, there was a high risk of biases in many studies, such as attrition and selection bias. Second, the results of the included studies were too heterogeneous to be pooled. Third, non-English language articles were excluded due to lack of translation resources, which was likely to influence the findings of the results. Moreover, only four of the included studies (Alpert et al., 2021; Kang et al., 2020; Weston and Zauche, 2021; Zhou et al., 2020) contained controls, so the assessment of the effectiveness of digital education was relatively incomplete. Although randomized controlled trials that provide the highest level of evidence for the effectiveness of interventions are not appropriate for educational research (Tudor Car et al., 2019a), a quasi-experimental design can be used at least in future studies to explore the effectiveness of standalone digital education in health professions. Finally, almost all studies used homemade questionnaires to measure outcomes, which made it challenging to compare the use of digital education in different settings. Therefore, alternative methods such as the total time spent on digital learning or deeply qualitative analyses on the perceptions of digital education should be used to investigate interns' attitudes with digital education in future studies (Huang et al., 2019).

Conclusion

This is the first systematic review to synthesize the evidence relating to the application of digital education for nursing and medical interns during the COVID-19 pandemic. After comprehensive analysis, we found that digital education plays a significant role in distance training for nursing and medical interns both now and in the future. The overall risk of bias was high, and the quality of evidence was found to be variable. There is a need for further research adopting designs that provide a high level of evidence to assess the effectiveness of standalone digital education interventions for the remote training of nursing or medical interns. Additionally, it is significant for teaching hospitals to formulate systematic guidelines for the implementation of digital education to be fully prepared for emergencies.

Funding statements

This study was supported by the Higher Education Reform Research Project of Jilin Province (2019XZD068).

CRediT authorship contribution statement

Xiaonan Hao and Xin Peng: developing the review questions and drafting the protocol, developing a search strategy, conception and writing of the manuscript, updating the review.

Xinxin Ding and Yuan Qin: developing a search strategy, searching for articles, obtaining copies of articles, selecting which articles to include.

Miaohua Lv and Jing Li: extracting data from included articles, carrying out the analysis, assessing the risk of bias.

Kun Li: drafting the protocol, providing methodological expertise and advice, conception, writing and proof-reading of the manuscript, updating the review.

Xiaonan Hao and Xin Peng contributed equally to this work.

Declaration of competing interest

There is no competing financial interest in the research. The purpose of the research did not reflect the official policy or position of the organization and government.