The effectiveness of e-healthcare interventions for mental health of nurses: A PRISMA-compliant systematic review of randomized controlled trials.

BACKGROUND: Mental health problems, including burnout among nurses, are common and important. With the rapid development of information and communication technologies and the rise in use of smartphones, the use of e-mental health strategies is increasing in public and clinical settings, and initial clinical trials using this intervention have been conducted. This systematic review evaluated whether e-healthcare interventions improve burnout and other mental health aspects in nurses.
METHODS: Six electronic databases including MEDLINE (via PubMed), EMBASE (via Elsevier), the Cochrane Library Central Register of Controlled Trials, the Cumulative Index of Nursing and Allied Health Literature, the Allied and Complementary Medicine Database, and PsycARTICLES were searched to collect relevant randomized controlled trials up to January 28, 2021, using e-healthcare interventions for mental health in nurses. The e-healthcare intervention was classified as web-based, smartphone-based, and real-time online interventions. The primary outcome was burnout in this population. Due to the heterogeneity of the interventions used in the included studies, quantitative synthesis was not performed, but included studies were analyzed qualitatively. Also, the details of e-healthcare for the mental health of nurses were analyzed. The methodological quality of included studies was assessed using Cochrane's Risk of Bias tool.
RESULTS: Seven randomized controlled trials were included in this study. The 20-minute session of an online form of the emotional freedom technique was reported to significantly improve burnout severity compared to no intervention (P < .001). Other outcomes, such as career identity, quality of work life, workplace bullying, job stress, turnover intention, distress, anxiety, and resilience in nurses, were also reported to be improved by e-healthcare interventions. The methodological quality of the included studies was generally poor.
CONCLUSIONS: In conclusion, there was some evidence that e-healthcare interventions may improve mental health outcomes, including burnout in nurses, compared with no intervention. However, due to the poor methodological quality and wide heterogeneity of the interventions and outcomes in the included studies, we were not able to reach sufficiently reliable conclusions. E-healthcare intervention for nurses in the new coronavirus disease era was discussed. High-quality clinical trials in this area should be conducted in the future.

Introduction

Due to the nature of their jobs, nurses are exposed not only to high-intensity stressful situations, such as emotional labor, but also to environments that can adversely affect their physical and mental health such as shift work. Thus, mental health problems are very common among nurses. According to a recent systematic review of 113 studies, the estimated prevalence of burnout among nurses is 11% globally.[ Moreover, the new coronavirus disease (COVID-19) has negatively impacted the mental health of nurses, and one-third of this population suffers from psychological symptoms, including depression, anxiety, and sleep disturbance.[ Burnout can exhibit symptoms including emotional exhaustion, depersonalization, and a sense of low personal accomplishment, and it is associated with reduced efficiency at work.[ Thus, burnout in healthcare workers can lead to significant professional consequences, such as lower patient satisfaction, impaired quality of care, and even medical errors, as well as negative consequences at the individual level (e.g., depression, anxiety, perceived stress).[ According to a study on 7076 registered nurses working in 161 hospitals in Pennsylvania, a 30% reduction in nurse burnout in acute care settings could save $68 million per year in infection-related costs.[

With the rapid development of information and communication technologies (ICT) and the rise in use of smartphones, the use of e-mental health strategies is increasing in public and clinical settings, and initial clinical trials using this intervention have been conducted.[ For example, previous studies have highlighted the potential benefits of e-mental health strategies in improving the mental health of caregivers of people with dementia,[ managing anxiety and depressive symptoms in adult population,[ and managing workers’ stress.[ The e-mental health strategy is advantageous in terms of improved accessibility, cost reduction, standardization, and personalization, and it is recognized as an important mental health management strategy in the current Corona era wherein non-face-to-face treatment is required more often.[ For example, in Korea, mind–body modalities have been provided in the form of telemedicine to improve the mental health of confirmed patients and public during the COVID-19 crisis.[ In addition, a retrospective observational study conducted at an academic medical center in Los Angeles reported that primary care services provided by telemedicine after the COVID-19 pandemic were well-accepted by patients with high satisfaction.[ As such, after the COVID-19 pandemic, e-healthcare has been considered as a promising coping strategy, and research on this topic has proliferated during this crisis.[

Since mental illness is generally not acceptable within the context of nursing culture, nurses with mental illness are faced with stigma and sometimes lack relevant treatment options.[ In this situation, ICT-based e-healthcare that promotes non-face-to-face treatment has the potential to provide a breakthrough for improving the mental health of nurses. Moreover, mind-body modalities have a positive effect on improving the mental health of healthcare professionals,[ and public mental health strategies combining mind-body modalities and ICT have recently been utilized.[ However, there has been no systematic review of e-healthcare interventions for mental health in nurses. Therefore, the purpose of this systematic review was to evaluate whether e-healthcare interventions improve burnout and other mental health aspects of nurses.

Methods

The protocol of this systematic review was registered in the Open Science Framework (OSF) registry (doi: 10.17605/OSF.IO/C236B) and complied with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement[ (Supplementary Digital Content file S1). The OSF registry is a free and open source project management tool developed by the Center for Open Science. This registry offers a preregistration service, and all research protocols, regardless of research type, can be registered with this registry, avoiding selective reporting bias. Unlike PROSPERO, the most well-known systematic review protocol registry, the OSF registry does not enforce strict criteria for the protocol registration, but provides robust time-stamped records to avoid selective reporting bias. Some guidance of systematic reviews, including that of the Cochrane Rapid Reviews Methods Group,[ supports the use of this registry. As this systematic review used published patient data, the need for ethical approval was waived.

Data sources and search strategy

A total of 6 international electronic databases, including MEDLINE (via PubMed), EMBASE (via Elsevier), the Cochrane Library Central Register of Controlled Trials, the Cumulative Index of Nursing and Allied Health Literature, the Allied and Complementary Medicine Database, and PsycARTICLES were comprehensively searched to collect relevant articles. In addition, a Google Scholar search was conducted to collect other potentially relevant literature. The search date was January 28, 2021, and all studies published up to the search date were considered, regardless of language and publication type. A single researcher (BL) performed the study search. Search strategies for each database and the results are shown in Supplementary Digital Content file S2.

Study selection and inclusion criteria

Two independent authors (JHP and SEJ) selected the studies using the title/abstract and a full-text review of the article. The inclusion criteria were as follows:

study design: only randomized controlled trials (RCTs) were included;

participants: nurses, regardless of sex, age, nationality, and ethnicity;

interventions: e-healthcare including internet-based intervention, smartphone-based intervention, and telemedicine;

comparators: no treatment, wait-list, sham control, attention control, and active comparators; and

outcomes: the primary outcome was the level of burnout assessed using validated assessment tools such as the Maslach Burnout Inventory[; secondary outcomes were other mental health aspects, including depression, anxiety, and perceived stress.

Disagreements during the study selection process were resolved by the intervention of a third author (CYK).

Data extraction

Two independent researchers (JHP and SEJ) extracted relevant data from the included studies using a predefined format in Microsoft Excel 2016 (Microsoft, Redmond, WA, USA). The extracted data included the following: first author, publication year, country of the first author, risk of bias evaluation, sample size, mean age, ward in which nurses work, pathological condition of nurse, e-health-related interventions used, control interventions, intervention period, outcomes, and results. Pathological condition of nurses was identified, when a cutoff evaluation score used in the included study was presented, the participants’ baseline characteristics exceeded the cutoff score, or a medical diagnosis was made for the participants. If evaluation scale without a cutoff score was used, the meaning was estimated as a clinical characteristic of the participant, considering the meaning of the Likert scale used. Disagreements between the extracted data were resolved through agreement between two researchers (JHP and SEJ).

Quality assessment

Two independent reviewers (JHP and SEJ) assessed the methodological quality of the included RCTs using the Cochrane's Risk of Bias tool. This tool assesses the methodological quality of RCTs in 6 domains of bias including selection bias, performance bias, detection bias, attrition bias, reporting bias, and other bias, as “low,” “high,” and “unclear.” This methodological quality assessment process complied with the content described in the Cochrane Handbook for Systematic Reviews of Interventions version 5.1.0.[ Disagreements during the risk of bias assessment process were resolved under the intervention of a third author (CYK). The risk of bias figure was created using Review Manager software (version 5.4; Cochrane, London, UK).

Data analysis

All included studies were analyzed qualitatively. Due to the heterogeneity of the interventions used in the included studies, quantitative synthesis was not performed. The reported results were instead classified into outcomes related to work functioning, job stress, mental health, biomarkers, and costs according to their characteristics. Moreover, e-healthcare interventions were largely divided into web-based, smartphone-based, and real-time online interventions. The e-healthcare interventions were also characterized. For comparisons between 2 groups, a P value less than 0.05 was considered to be statistically significant.

Publication bias

Quantitative synthesis was not performed in this study. The interventions and outcomes of the included studies were heterogeneous, and the number of included studies was small; therefore, evaluation of publication bias using a funnel plot was not performed.

Results

Study selection

A total of 8986 articles were identified through searching the databases, and no additional records were identified through other sources. First-round screening of titles and abstracts was conducted for 6279 articles, after excluding duplicates. The Cohen's kappa coefficient for the agreement of the 2 reviewers on the screening process was 0.84, indicating almost perfect agreement. As a result, 12 potentially relevant articles were selected. After reviewing the full text of these, four[ were found to be unrelated to mental health and one[ was excluded because the participants were not limited to nurses. Finally, a total of 7 RCTs[ were included in this review (Fig. 1).

Figure 1

A PRISMA flow diagram of the literature screening and selection processes. AMED = allied and complementary medicine database, CENTRAL = cochrane central register of controlled trials, CINAHL = cumulative index to nursing and allied health literature.

Study characteristics

Among the included studies, 6 were parallel RCTs,[ while the other was a crossover RCT.[ All included studies were published between 2008 and 2020, and were conducted in Japan,[ the Netherlands,[ the USA,[ Iran,[ Korea,[ Turkey,[ and Taiwan.[ The analyzed sample sizes ranged from 36 to 617. Regarding the pathological condition of the participants, most studies[ did not use any evaluation scale with a validated cutoff score and/or did not recruit participants with a medical diagnosis. Regarding the baseline clinical characteristics of the participants, Yamagishi et al [ used the Career Identity Scale (CIS; 1–7 Likert), the Brief Job Stress Questionnaire (BJSQ; 1–4 Likert), and mental health domains of BJSQ, resulting in moderate levels of career identity (mean CIS score, 4–5), job stress (mean BJSQ score, 2–3), vigor, anger, fatigue, and anxiety (mean each subscale score in BJSQ, 2–3), and low level of depression (mean subscale score in BJSQ, 1.9). Noven et al [ used some subscales of Nurses Work Functioning Questionnaire (NWFQ; 0–4 or 0–6 Likert) to assess baseline clinical characteristics of the participants. However, since they did not provide the calculation method they used, it was difficult to estimate the participants’ baseline work functioning. Hersch et al [ used the Nursing Stress Scale (NSS; 1–4 Likert), Symptoms of Distress (1–4 Likert), Coping with Stress (1–4 Likert), and the 16-item Work Limitations Questionnaire (WLQ; 1–5 Likert), resulting in moderate levels of nursing stress (mean NSS score, 2–3), distress symptoms (mean Symptoms of Distress score, 2–3), and stress coping (mean Coping with Stress score, 2–2.5) and a high level of work limitations (mean WLQ score, 4–4.5). Motamed-Jahromi et al [ used the Quality of Work Life Questionnaire (QWLQ; 1–5 Likert), resulting in a moderate level of quality of work life (mean QWLQ score, 2.5–3). Kang et al [ used the Negative Acts Questionnaire-Revised (NAQ-R; 1–5 Likert) and turnover intention questionnaire (1–5 Likert), resulting in a low-to-moderate level of workplace bullying (mean NAQ-R score, 1.5–2) and moderate-to-high level of turnover intention (mean score, 3.5–4). Dincer et al [ used the Subjective Units of Distress (SUD; 0–11 Likert), State-Trait Anxiety Inventory (STAI; 1–4 Likert with a cutoff score of 20), and burnout scale (1–7 Likert), resulting in moderate levels of distress (mean SUD score, 7.5–8) and burnout (mean burnout scale score, 3.5–4) and severe state anxiety (mean STAI total score, 60–70). Lastly, Hsieh et al [ used the Centre for Epidemiologic Studies Depression Scale (CES-D; 0–3 Likert with a cutoff score of 15), the Resilience Scale (1–7 Likert), and the Occupational Stress Indicator-2 (OSI-2; 1–5 Likert), resulting in borderline significant depression (mean CES-D total score, 14.96), high occupational stress (mean the Resilience Scale score, 5.18), and a low level of resilience (mean OSI-2 score, 1.55). In summary, participants with severe state anxiety and borderline significant depressive symptoms were included using evaluation scales with a validated cutoff score in 2 studies. Likewise, participants with moderate levels of stress, anger, fatigue, distress, and burnout and those with high levels of work limitation and occupational stress were included in other studies.

Regarding the interventions used in the included studies, 3 used web-based interventions,[ 3 used smartphone-based interventions,[ and 1 used real-time online intervention.[ Five studies were two-arm studies, of which the control group of 4 studies had no intervention,[ and an active control was used in 1 study.[ Two three-arm studies were conducted. Noben et al compared occupational physician conditions, e-mental health conditions, and no intervention,[ while Hsieh et al compared biofeedback training, smartphone-delivered biofeedback training, and no intervention[ (Table 1).

Table 1

Characteristics of included studies.

Study (Country)	Sample size (included→analyzed)	Mean age (range) (years)	Ward in which nurses work	Pathological condition of nurse	(A) Treatment intervention	(B) Control intervention	Intervention period (Assessment)	Outcomes
Yamagishi 2008 (Japan)	51→36 (A) 26→20 (B) 25→16 (only first phase of the cross-over design)	33.0	SU, MU, etc.	NA	Web-based career identity training	No intervention	3 Weeks (Week 0, 3)	1. CIS; 2. JSQ; 3. BJSQ
Noben 2014 (Netherlands)	633→617 (A1) 210→207 (A2) 212→204 (B) 211→206	(A1) 42.56 ± 11.357 (A2) 37.5 ± 12.16 (B) 41.83 ± 11.305	Not limited (SU, ANE, AHP)	NA	(A1) Occupational physician condition(A2) E-mental health condition	No intervention	6 Months (Month 0, 6)	1. NWFQ; 2. Medical costs; 3. Non-medical costs
Hersch 2016 (USA)	104→104 (A) 52→52 (B) 52→52	41 (22-65)	Not limited (ICU, CCU, MU, SU, OR, ED, NICU, etc.)	NA	Web-based stress management program for nurses	No intervention	3 Months (Month 0, 3)	1. NSS; 2. Symptoms of distress; 3. Coping with stress; 4. WLQ; 5. Use of substances for stress relief; 6. Drinking quantity and frequency; 7. Understanding depression and anxiety; 8. NJSS
Motamed-Jahromi 2017 (Iran)	100→100 (A) 50→50 (B) 50→50	30 (22–55)	Not limited (ICU, GU, PU)	NA	Smartphone application (positive thinking)	Positive thinking books	3 Months (Month 0, 3)	QWLQ
Kang 2019 (South Korea)	73→72 (A) 36→36 (B) 37→36	(A) 30.78 ± 8.06 (B) 29.56 ± 7.24	GU, ICU	NA	Smartphone application (cognitive rehearsal intervention)	No intervention	4 Weeks (Week 0, 4, 8)	1. NAQ-R; 2. Turnover intention
Dincer 2020 (Turkey)	72→72 (A) 35→35 (B) 37→37	(A) 33.54 ± 9.83 (B) 33.37 ± 9.58	Caring for COVID-19 patients	Severe state anxiety (baseline STAI score)	Brief online form of EFT	No intervention	1 Session (Min 0, 20)	1. SUD; 2. STAI; 3. Burnout scale
Hsieh 2020 (Taiwan)	159→135 (A1) NR→49 (A2) NR→47 (B) NR→39	(A1) 38.45 ± 9.23 (A2) 32.21 ± 6.36(B) 35.61 ± 7.47	PU	Abused psychiatric nurses, borderline significant depression (baseline CES-D score)	(A1) BT (A2) Smartphone-delivered BT	No intervention	6 Weeks (Week 0, 6)	1. CES-D; 2. RS; 3. OSI-2; 4. HRV (including SDNN, LF, HF); 5. RR

AHP = allied health professional, ANE = anesthesiology, BJSQ = Brief Job Stress Questionnaire, BT = biofeedback training, CCU = cardiac care unit, CES-D = Centre for Epidemiologic Studies Depression Scale, CIS = Career Identity Scale, ED = emergency department, EFT = emotional freedom technique, GU = general unit, HF = high frequency, HRV = heart rate variability, ICU = intensive care unit, JSQ = Job Stress Questionnaire, LF = low frequency, MU = medical unit, NA = not applicable, NAQ-R = Negative Acts Questionnaire-Revised, NICU = neonatal intensive care unit, NJSS = Nurses Job Satisfaction Scale, NR = not reported, NSS = Nursing Stress Scale, NWFQ = Nurses Work Functioning Questionnaire, OR = operation room, OSI-2 = Occupational Stress Indicator-2, PU = psychiatric unit, QWLQ = Quality of Work Life Questionnaire, RR = respiration rate, RS = Resilience Scale, SDNN = standard deviation of normal to normal, STAI = State-Trait Anxiety Inventory, SU = surgical unit, SUD = Subjective Units of Distress, WLQ = 16-item Work Limitations Questionnaire.

Risk of bias within studies

For the random sequence generation domain, 4 studies[ that used proper random sequence generation methods, such as random number tables, were rated as having a low risk of bias. Two studies[ did not report the random sequence generation methods, while one[ that used the selection of balls from a bag, but did not return the drawn balls to the bag, was rated as having a high risk of bias. For allocation concealment, only one study[ that used an opaque bag was rated as having a low risk of bias. The other 5 studies[ that did not address allocation concealment, and another study,[ which reported that proper allocation concealment could not be performed, were rated as having a high risk of bias. In 6 studies,[ the blinding of participants and personnel was not clearly reported and another study[ reported that no blinding procedure was employed. Only 2 studies[ reported blinding of outcome assessors, while the others[ did not report blinding. For the incomplete outcome data domain, 5 studies[ were evaluated as having a low risk of bias because there were no dropouts or missing values were replaced using appropriate statistical techniques. However, in one study[ there were dropouts, but the explanation for the reason was insufficient, and in another study,[ information about the dropout was not reported. For the selective reporting domain, 2 studies[ that reported all outcomes described in preregistered protocols were evaluated as having a low risk of bias, and 2 studies[ that did not report the raw data of their outcomes had a high risk of bias. In the other 3 studies,[ there was insufficient information to evaluate this domain. For the other bias domain, 5 studies[ that reported statistical homogeneity of the included groups at baseline were evaluated as having a low risk of bias. However, one study[ that used a crossover design and another[ that reported statistical heterogeneity at baseline between the treatment and control groups were evaluated as having a high risk of other potential biases (Fig. 2).

Figure 2

Risk of bias summary for all included studies. Low, unclear, and high risk, respectively, are represented with the following symbols: “+”, “?”, and “-”.

Main results

Primary outcome (burnout)

Real-time online intervention: Only one study reported the effects on burnout, the primary outcome of this review. Dincer et al [ reported that the treatment group (n = 35) that received a 20-minute session of an online form of the Emotional Freedom Technique (EFT) showed significantly lower results for burnout severity, evaluated by the burnout scale, compared to the control group (n = 37) with no intervention (P < .001) (Table 2).

Table 2

Main results of included studies.

Outcomes	Comparison	Results	Reference
Outcomes related to work functioning
1. CIS (career identity)	Web-based career identity training vs No intervention	1) Knowledge of career identity: 7.00 ± 2.62 vs 4.63 ± 1.50 (P = .004)2) Understanding: 4.57 ± 0.67 vs 4.48 ± 0.89 (P = .395)3) Identifying: 4.32 ± 1.07 vs 4.19 ± 1.51 (P = .611)4) Meaning: 5.37 ± 0.90 vs 5.04 ± 1.00 (P = .071)5) Self-efficacy: 4.22 ± 0.75 vs 4.00 ± 1.29 (P = .791)6) Self-determination: 4.42 ± 0.86 vs 4.69 ± 1.34 (P = .334)7) Impact on organization: 4.50 ± 0.70 vs 3.94 ± 0.86 (P = .005)8) Impact on patients: 5.35 ± 0.75 vs 5.27 ± 0.85 (P = .493)	Yamagishi 2008
2. NWFQ (work functioning)	Occupational physician condition vs E-mental health condition to improve work functioning vs No intervention	NR (It was used only as data for economic evaluation, and the raw data was not reported.)	Noben 2014
3. WLQ (work limitations)	Web-based stress management program for nurses vs No intervention	1) Time and Scheduling Demands: 4.078 ± 0.92 vs 3.908 ± 0.79 (P = .778)2) Focus and Concentration Demands: 4.305 ± 0.73 vs. 3.964 ± 0.82 (P = .806)3) Interpersonal Demands: 4.340 ± 0.68 vs 4.222 ± 0.74 (P = .754)4) Quality and Quantity Demands: 4.193 ± 0.89 vs 4.001 ± 0.77 (P = .603)	Hersch 2016
4. QWLQ (quality of work life)	Smartphone application (positive thinking) vs Positive thinking books	1) Home life: 4.16 ± 22.84 vs 4.31 ± 19.14 (P < .001)2) Work design: 30.68 ± 5.39 vs 26.12 ± 5.48 (P < .001)3) Work context: 62.16 ± 12.03 vs 51.18 ± 13.52 (P < .001)4) Work world: 16.60 ± 3.42 vs 15.10 ± 3.46 (P = .08)5) Total score: 132.28 ± 19.89 vs 111.54 ± 22.43 (NR)	Motamed-Jahromi 2017
5. NAQ-R (workplace bullying)	Smartphone application (cognitive rehearsal intervention) vs No intervention	1) Person related: 17.78 ± 6.77 vs 20.28 ± 5.29 (P = .022)2) Work related: 9.50 ± 3.36 vs 8.31 ± 3.28 (P = .017)3) Intimidation related: 5.69 ± 1.21 vs 5.67 ± 1.22 (P = .226)	Kang 2019
Outcomes related to job stress
1. JSQ (job stress)	Web-based career identity training vs No intervention	1) Workload: 2.83 ± 0.67 vs 2.97 ± 0.83 (P = .090)2) Mental workload: 2.58 ± 0.60 vs 2.79 ± 0.81 (P = .262)3) Job control: 2.40 ± 0.58 vs 2.48 ± 0.68 (P = .316)4) Personal relationships: 2.18 ± 0.88 vs 2.40 ± 1.06 (P = .165)5) Support from others: 2.75 ± 0.53 vs 2.64 ± 0.66 (P = .618)6) Reward from work: 2.73 ± 0.50 vs 2.69 ± 0.74 (P = .837)	Yamagishi 2008
2. NSS (job stress)	Web-based stress management program for nurses vs No intervention	2.072 ± 0.38 vs 2.350 ± 0.49 (P = .003)	Hersch 2016
3. NJSS (job satisfaction)	Web-based stress management program for nurses vs No intervention	NR (No raw data were reported, only it was explained that no statistically significant differences were found between groups.)	Hersch 2016
4. Turnover intention	Smartphone application (cognitive rehearsal intervention) vs No intervention	3.13 ± 0.92 vs 3.36 ± 0.77 (P = .001)	Kang 2019
5. OSI-2 (occupational stress)	BT vs Smartphone-delivered BT vs No intervention	55.55 ± 30.73 vs 42.97 ± 37.91 vs 62.18 ± 40.27 (P = .015)∗BT vs Smartphone-delivered BT: P = .319∗Smartphone-delivered BT vs No intervention: P = .005	Hsieh 2020
Outcomes related to mental health
1. BJSQ (mental health)	Web-based career identity training vs No intervention	1) Vigor: 3.30 ± 0.53 vs 3.31 ± 0.66 (P = .525)2) Anger: 2.37 ± 0.76 vs 2.44 ± 1.00 (P = .670)3) Fatigue: 2.82 ± 0.82 vs 3.10 ± 0.79 (P = .271)4) Anxiety: 2.26 ± 0.73 vs 2.33 ± 0.78 (P = .444)5) Depression: 1.93 ± 0.67 vs 2.04 ± 0.85 (P = .543)	Yamagishi 2008
2. Symptoms of Distress (distress)	Web-based stress management program for nurses vs No intervention	2.968 ± 0.51 vs 2.840 ± 0.54 (P = .125)	Hersch 2016
3. Coping with Stress (stress)	Web-based stress management program for nurses vs No intervention	1.949 ± 0.39 vs 1.992 ± 0.31 (P = .537)	Hersch 2016
4. Use of substances for stress relief (substance use)	Web-based stress management program for nurses vs No intervention	1) Using Alcohol to Relieve Stress: 1.738 ± 0.86 vs 1.912 ± 0.99 (P = .379)2) Using Prescription Drugs as Prescribed to Relieve Stress: 1.104 ± 0.48 vs 1.112 ± 0.47 (P = .806)3) Using Prescription Drugs Not as prescribed to Relieve Stress: 1.020 ± 0.11 vs 1.038 ± 0.28 (P = .609)	Hersch 2016
5. Drinking quantity and frequency (substance use)	Web-based stress management program for nurses vs No intervention	1) Drank in the past 12 Mo: 0.876 ± 0.32 vs 0.886 ± 0.31 (P = .787)2) Number of Drinking Days in Past 30 Days: 4.452 ± 5.89 vs 4.918 ± 4.65 (P = .107)3) Number of Drinks Per Drinking Day in Past 30 Days: 1.513 ± 1.19 vs 1.455 ± 1.04 (P = .680)4) Number of Days Having 5 or More Drinks on One Occasion in Past 30 Days: 0.282 ± 0.78 vs 0.353 ± 0.76 (P = .397)	Hersch 2016
6. Understanding depression and anxiety (depression and anxiety)	Web-based stress management program for nurses vs No intervention	3.470 ± 0.38 vs 3.357 ± 0.47 (P = .117)	Hersch 2016
7. SUD (distress)	Brief online form of EFT vs No intervention	2.85 ± 1.21 vs 7.40 ± 1.53 (P < .001)	Dincer 2020
8. STAI (anxiety)	Brief online form of EFT vs No intervention	32.25 ± 4.67 vs 64.43 ± 7.68 (P < .001)	Dincer 2020
9. Burnout scale (burnout)	Brief online form of EFT vs No intervention	2.48 ± 1.06 vs 3.43 ± 0.76 (P < .001)	Dincer 2020
10. CES-D (depression)	BT vs Smartphone-delivered BT vs No intervention	8.72 ± 6.10 vs 8.13 ± 6.33 vs 12.29 ± 6.95 (P = .135)	Hsieh 2020
11. RS (resilience)	BT vs Smartphone-delivered BT vs No intervention	164.15 ± 23.16 vs 158.77 ± 19.20 vs 153.67 ± 23.75 (P < .001)∗BT vs Smartphone-delivered BT: P = .355∗Smartphone-delivered BT vs No intervention: P < .001	Hsieh 2020
Outcomes on biomarker
1. HRV	BT vs Smartphone-delivered BT vs No intervention	1) SDNN: 44.48 ± 23.8 vs 54.94 ± 32.6 vs 52.10 ± 81.56 (P = .922)2) LF: 364.00 ± 416.66 vs 737.77 ± 343.44 vs 455.45 ± 442.49 (P = .170)3) HF: 410.58 ± 416.71 vs 625.15 ± 597.35 vs 858.28 ± 716.50 (P = .688)	Hsieh 2020
2. RR	BT vs Smartphone-delivered BT vs No intervention	13.84 ± 2.84 vs 14.73 ± 2.8 vs 15.34 ± 2.5 (P = .084)	Hsieh 2020
Outcomes related to costs
1. Medical cost	Occupational physician condition vs E-mental health condition to improve work functioning vs No intervention	1) Service use (€): 98.5 vs 100.39 vs 97.962) Medication (€): 2.21 vs 2.37 vs 0.99	Noben 2014
2. Non-medical cost	Occupational physician condition vs E-mental health condition to improve work functioning vs No intervention	1) Absenteeism (€): 234.1 vs 230.03 vs 373.952) Presenteeism (€): 916.09 vs. 1016.28 vs 1266.783) Direct non-medical costs (€): 10.83 vs 9.77 vs 9.37	Noben 2014

BJSQ = Brief Job Stress Questionnaire, BT = biofeedback training, CES-D = Centre for Epidemiologic Studies Depression Scale, CIS = Career Identity Scale, EFT = emotional freedom technique, HF = high frequency, HRV = heart rate variability, JSQ = Job Stress Questionnaire, LF = low frequency, NAQ-R = Negative Acts Questionnaire-Revised, NJSS = Nurses Job Satisfaction Scale, Note. If it was associated with a statistically significant benefit compared to the control group, the intervention was highlighted with bolding. NR = not reported, NSS = Nursing Stress Scale, NWFQ = Nurses Work Functioning Questionnaire, OSI-2 = Occupational Stress Indicator-2, QWLQ = Quality of Work Life Questionnaire, RR = respiration rate, RS = Resilience Scale, SDNN = standard deviation of normal to normal, STAI = State-Trait Anxiety Inventory, SUD = Subjective Units of Distress, WLQ = 16-item Work Limitations Questionnaire.

Secondary outcome

Excepting outcomes without raw data, the results were as follows.

Web-based interventions: Regarding work functioning outcomes, compared with no intervention, Yamagishi et al [ found that web-based career identity training significantly improved some outcomes of career identity assessed by CIS, including knowledge of career identity (P = .004) and impact on organization (P = .005), but not understanding (P = .395), identifying (P = .611), meaning (P = .071), self-efficacy (P = .791), self-determination (P = .334), and impact on patients (P = .493). However, Hersch et al [ found that compared with no intervention, web-based stress management programs did not significantly improve any subscales of work limitations assessed by WLQ, including time and scheduling demands (P = .778), focus and concentration demands (P = .806), interpersonal demands (P = .754), and quality and quantity demands (P = .603). Regarding outcomes related to job stress, compared with no intervention, Yamagishi et al[ found that web-based career identity training did not significantly improve any subscales of job stress assessed by the Job Stress Questionnaire, including workload (P = .090), mental workload (P = .262), job control (P = .316), personal relationships (P = .165), support from others (P = .618), and rewards from work (P = .837). However, Hersch et al [ found that compared to no intervention, a web-based stress management program significantly improved job stress, as assessed by NSS (P = .003). Regarding outcomes related to mental health, both studies used web-based interventions and did not find any significant improvement in any mental health index outcomes compared with no intervention, including the mental health-related subscales (i.e., vigor, anger, fatigue, anxiety, and depression) of BJSQ (P = .271–.543),[ distress assessed by Symptoms of Distress (P = .125),[ stress coping assessed by Coping with Stress (P = .537),[ use of substances for stress relief (i.e., using alcohol to relieve stress, prescription drugs as prescribed to relieve stress, and prescription drugs not as prescribed to relieve stress) (P = .379–.806),[ drinking quantity and frequency (P = .107–.787),[ and depression and anxiety assessed by Understanding Depression and Anxiety (P = .117).[ Regarding outcomes related to costs, Noben et al [ found that the incremental cost-effectiveness ratio for the e-mental health condition versus no intervention was estimated at €4,054 (added costs) per treatment responder. In summary, web-based interventions were related to significant benefits in career identity; non-significant effects on work limitations, general mental health, distress, stress coping, substance use, and negative emotions; and mixed results regarding job stress in nurses (Table 2).

Smartphone-based interventions: Regarding outcomes related to work functioning, compared to books about positive thinking, Motamed-Jahromi et al [ found that smartphone application-supported positive thinking significantly improved some outcomes of quality of work life assessed by QWLQ, including home life (P < .001), work design (P < .001), and work context (P < .001), but not work environment (P = .0.08). In addition, Kang et al [ found that compared with no intervention, smartphone application-supported cognitive rehearsal intervention significantly improved some outcomes of workplace bullying assessed by NAQ-R, including personal-related (P = .022) and work-related (P = .017), but not intimidation-related (p = 0.226). Regarding outcomes related to job stress, both studies used smartphone-based interventions and found significant improvement in turnover intention (P = .001)[ and occupational stress assessed by OSI-2 (P = .005),[ compared with no intervention. Regarding outcomes related to mental health, the study by Hsieh et al ,[ which used smartphone-delivered biofeedback training, found that the treatment group showed significantly higher resilience as assessed by RS (P < .001), but not depression, as assessed by CES-D (P = .135), compared with the no intervention group. Regarding outcomes related to biomarkers, Hsieh et al found that compared with no intervention, smartphone-delivered biofeedback training did not significantly impact the outcomes of heart rate variability (HRV), including the standard deviation of normal to normal (P = .922), low frequency (P = .170), and high frequency (P = .688), as well as those of respiration rate (P = .084). In summary, smartphone-based interventions were related to significant benefits in the quality of work life, workplace bullying, job stress, turnover intention, and resilience and non-significant effects on depression, some indicators of HRV, and the respiratory rate in nurses (Table 2).

Real-time online intervention: Regarding outcomes related to mental health, the study by Dincer et al ,[ which used a brief, online form of EFT, found that the treatment group showed significantly lower distress assessed by SUD (P < .001) and anxiety state assessed by STAI (P < .001), compared with the no intervention group. In particular, the total STAI score of the EFT group after the intervention was reduced to 32.25 ± 4.67, confirming the improvement from severe to mild state anxiety. In summary, real-time online intervention was related to significant benefits in distress, anxiety, and burnout in nurses (Table 2).

Details of e-healthcare for the mental health of nurses

The type of e-healthcare for the mental health of nurses used in the included studies varied. Content that was directly related to nurses’ work was used in only one study. Yamagishi et al [ provided career identity training as its content. On the other hand, 4 studies provided content related to mental health improvement, including general mental health,[ stress management,[ positive thinking,[ and cognitive rehearsal.[ In addition, one study[ provided EFT, which is a type of mind-body therapy that uses the tapping of specific points of the body, and another study[ provided biofeedback training through smartphone technology. That is, out of the 7 included studies, in 5 studies,[ the therapies were used to improve the mental health or work functioning of nurses. In the other study,[ it was used for the purpose of educating a mind-body therapy that could potentially improve the mental health of nurses. Another study[ applied ICT complementarily to provide biofeedback training, but was not content-based (Table 3).

Table 3

Details of e-healthcare for mental health of nurse.

Study	Treatment intervention	Contents of the intervention
Yamagishi 2008	Web-based career identity training -Time per session: 60 min/session -Frequency: NR	1) The definition of career identity2) Cognition of the participants’ own career identity3) The characteristics of nurses’ career identity (including career alternatives, compatibility with life-cycle, and examples of career goals and planning) 4) Career goal management and planning.
Noben 2014	E-mental health condition -Time per session: NR -Frequency: NR	1) Psyfit: aimed at promoting mental fitness and wellbeing2) Strong at Work: aimed at learning skills to cope better with work-related stress3) Colour your Life: for coping with depressive symptoms4) Don’t Panic Online: to reduce symptoms of panic disorder5) Drinking Less: aimed at reducing risky alcohol consumption. Note. In any case, making use of the e-mental health interventions was strictly voluntary and nurses were free to reject the offer of using the interventions
Hersch 2016	Web-based stress management program for nurses -Time per session: mean 43 min/session -Frequency: NR	1) Welcome and Introduction: includes information on the stresses of nursing, and how stress impacts the body and affects daily life2) Assess Your Stress: provides assessments and feedback on personal stress and coping levels3) Identify Stressors: helps users recognize the symptoms of stress and their personal stressors and includes a tool for tracking their stress4) Manage Stress: provides a number of different stress management strategies and tools5) Avoid Negative Coping: addresses the problem of using alcohol and drugs to manage stress6) Your Mental Health: focuses on depression and anxiety and when to seek additional counseling
Motamed-Jahromi 2017	Smartphone application (positive thinking) -Time per session: NR -Frequency: NR	On average, 12 picture messages, 3 audios, and 1 video related to positive thinking were sent daily, as follow.1) Familiarity with positive thoughts and beliefs about ourselves, God, others, life, and future.2) Fighting against negative thoughts and stopping them through positive thinking.3) How to adapt to adverse events and changes in mood caused by their beliefs.4) Change of pessimistic explanatory style through practicing optimism and positive thinking.5) Familiarity with the process of synergy including summarizing the thoughts, actions, and plans for the future.6) Familiarity with the techniques of self-acceptance, self-esteem, focusing on limited capability, and treating disabilities.7) Optimism by reinforcing religious beliefs; familiarity with the role of faith and trust in God.
Kang 2019	Smartphone application (cognitive rehearsal intervention) -Time per session: NR -Frequency: NR	1) An introduction to non-violent conversation as a standard communication2) Six webtoons of workplace bullying situations, including (1) when a colleague nurse gets angry and shouts at me, (2) when he or she interrupts my work, (3) when he or she treats me as if I am invisible, (4) when he or she disregards me, (5) when he or she humiliates me in front of others, and (6) when pointing out personal issues3) A bulletin board for questions and answers
Dincer 2020	Brief online form of EFT -Time per session: 20 min/session -Frequency: NR	Basic steps of EFT1) Identify an anxiety-evoking issue and determine the SUD level.2) Creating a personal acceptance and reminder statement in the general form of “I accept myself despite this ∼”3) Tapping seven times on each acupressure point shown.4) After tapping these points, the affirmation/reminder statement is repeated.5) A sequence of physical movements and vocalizations called “The Nine Gamut Procedure” is carried out.6) Steps 3 and 4 are repeated.7) Another SUD rating is given.
Hsieh 2020	Smartphone-delivered BT -Time per session: NR -Frequency: 1 process/week	MP4 video file (guided shorter meditation practices and the processes of real-time BT) based on the conception of BT, including (1) self-guided muscle relaxation; (2) diaphragmatic breathing; (3) pursed-lips breathing; and (4) real-time respiratory sinus arrhythmia BT

BT = biofeedback training, EFT = emotional freedom technique, NR = not reported, SUD = subjective units of distress.

Discussion

Summary of evidence

The aim of this review was to comprehensively analyze the effectiveness of e-healthcare interventions on burnout and other mental health aspects in nurses. Seven RCTs were included, and overall, participants had moderate or higher levels of anxiety, depression, stress, anger, fatigue, distress, burnout, work limitation and/or occupational stress. However, in one study targeting workplace bullying,[ the baseline work bullying levels were only mild to moderate. The e-healthcare interventions could be classified into web-based, smartphone-based, and real-time online interventions.

Only data regarding the online form of EFT were available for burnout analysis, and the intervention significantly improved burnout severity compared with no intervention. The effects of web-based, smartphone-based, and real-time online interventions on several outcomes are summarized in Figure 3. Regarding web-based interventions, studies reported that the interventions showed significant benefits on career identity, but showed non-significant effects on work limitations, general mental health, distress, stress coping, substance use, and negative emotions. Mixed results were reported for job stress. Regarding smartphone-based interventions, studies reported that the interventions showed significant benefits on the quality of work life, workplace bullying, job stress, turnover intention, and resilience, but showed non-significant effects on depression, some indicators of HRV, and respiration rate. Although there was no significant difference in the CES-D total score between smartphone-based biofeedback and no intervention groups, the total CES-D score of the intervention group was reduced after the intervention from 15.08 ± 7.13 (significant depression) to 8.13 ± 6.33 (not significant depression), but the minimal clinically important difference (MCID) of this evaluation tool was not established.[ Lastly, regarding real-time online intervention, one study using EFT reported that the intervention showed significant benefits on distress, anxiety, and burnout. In particular, the STAI total score of the EFT group was reduced after the intervention from 67.68 ± 9.05 (severe state anxiety) to 32.25 ± 4.67 (mild state anxiety), and the change value exceeded MCID (i.e., 10) of this evaluation tool.[ In summary, studies using smartphone-based or real-time online interventions reported significant improvement in various mental health-related outcomes in nurses, but significant improvement after web-based interventions was rarely reported.

Figure 3

E-healthcare for mental health of nurse. Green, yellow, red, and grey, respectively, are represented with the following symbols: “statistically significant improvement compared to control group (mostly no intervention group)”, “statistically significant improvement in some subscales, but no statistical difference in other subscales”, “no statistically significant improvement”, and “not performed statistical analysis”. The number in each column indicates the number of references.

Overall, our findings were not conclusive, as the interventions and outcomes in the included studies were varied and heterogeneous, and their methodological quality was poor. Moreover, the exact efficacy of the e-healthcare interventions used could not be ascertained because most control conditions used in the included studies were without intervention. In other words, the contribution of the placebo effect to the effectiveness reported in the included studies could not be excluded. However, we believe that we have uncovered potentially relevant data for establishing a promising e-healthcare strategy for future clinical use or research.

Clinical implications

Mental health problems in nurses, including burnout, are very common and important,[ and there is an urgent need to address them. The development of ICT and the emphasis on non-face-to-face services due to COVID-19 are driving the adoption of ICT in the field of mental healthcare.[ Because ICT can be easily combined with other interventions already known to improve the mental health of healthcare professionals, such as mind-body modalities,[ its use may be a promising strategy for improving the mental health of nurses in the future.

The results of our review show that goal-oriented, web-based interventions (e.g., regarding career identity) may be associated with improvements in some outcomes related to work functioning or job stress. In addition, it has been reported that positive psychological techniques or cognitive therapy content based on smartphone application have positive effects on various mental health outcomes such as work functioning, job stress, turnover intention, depression, and resilience. Interestingly, there was a study that provided biofeedback to smartphone application, but it did not significantly affect biomarkers such as HRV or respiration rate. Finally, there was a report that provided EFT, a type of mind-body modality, online and in real-time, to improve distress, anxiety, and burnout. These results suggest that some work-oriented content, mental health contents, mind-body modalities, and even biofeedback can be combined with ICT to improve the mental health outcomes of nurses.

However, the wide heterogeneity of interventions and outcomes used makes it difficult to draw firm conclusions. Given the absence of a “golden standard” for mental health outcomes in the non-clinical population of nurses, the outcomes of future studies in this field should be more detailed, and these studies should be based on large-scale clinical observations of the mental health of nurses.

E-healthcare intervention for nurses in the COVID-19 era

Only one study,[ which used the online form of EFT to reduce burnout in nurses after the COVID-19 pandemic, showed that e-healthcare intervention could be used to improve nurses’ mental health in the COVID-19 era. Therefore, this study may be regarded as a type of outlier that distinguishes it from other included studies. Since the outbreak of COVID-19, mental health problems of health care workers including nurses have become a hot topic,[ and the use of mental health promotion strategies to reduce stress, anxiety, and depression symptoms of nurses is considered important in this era.[ Managing the mental health of healthcare workers can contribute to preserving organizational resilience, patient safety, and staff retention, potentially enhancing humanity's capacity to cope with the COVID-19 disaster.[ Although there has only been one study,[ it has shed light on new possibilities for e-healthcare intervention in the COVID-19 era, during which unprecedented levels of psychological pressure and physical exertion on healthcare workers including nurses has occurred.

Limitations and suggestions for further studies

This review presents the results of the most comprehensive systematic review available to date on e-healthcare interventions for burnout and other mental health aspects of nurses. However, the following limitations should be considered when interpreting the findings. First, because the number of studies included in this review was insufficient and the interventions used in each study varied, the evidence proving the effectiveness of e-healthcare interventions for most outcomes was based on one or 2 studies. Therefore, the evidence base for this finding is weak and may be influenced by the results of high-quality clinical trials in the future. As previously described, given that the mental health of nurses is important and e-healthcare interventions are a promising strategy for this issue, further research in this field is needed in the near future. Second, the methodological quality of the included studies was generally poor. In particular, most studies showed a high risk of selection bias, performance bias, and detection bias. These limitations may have resulted in an overestimation of the effect size of the interventions studied. Considering that the comparators of the included studies were mostly no intervention, a high risk of performance bias may be inevitable. However, even with such a comparison, selection and detection biases are methodological issues that can be resolved; future studies should take this into consideration and use more appropriate methodologies. Third, the interventions were classified as web-based, smartphone-based, or real-time, online interventions. This classification is based on the fact that the interventions used ICT; however, since the contents of the interventions were all different, the interventions were heterogeneous. These limitations suggest that the findings of this review should not be interpreted simply as web-based, smartphone-based, or real-time online interventions; rather, the content of the interventions used in each study should be interpreted more precisely. On a positive note, as research continues, further analysis of effective intervention elements will become possible, and will lead to the development of standardized e-healthcare interventions. Fourth, none of the included studies included a medical diagnosis as the inclusion criteria for participants. Although these participants showed increased levels of state anxiety, depression, stress, anger, fatigue, distress, burnout, work limitation, and/or occupational stress at baseline, they were not a substitute for a medical diagnosis, including psychiatric disorders. Therefore, it should be clearly emphasized that the results of this systematic review are aimed at the occupational group of nurses and not the clinical population. Fifth, the outcomes used in the included studies were heterogeneous, and in particular, the use of evaluation tools with verified cutoff scores and/or MCID was limited to 2 studies.[ This suggests that in order to prove clinically relevant effectiveness of e-healthcare interventions for nurses in future clinical studies, evaluation tools with validated cutoff scores and MCIDs should be used. Sixth, although one study[ included biomarkers related to mental health, HRV and respiration rate, other biomarkers such as serum stress hormone levels and blood pressure were lacking. Investigating and reporting the impact on biomarkers can provide an objective understanding of e-healthcare interventions for improving nurses’ mental health, as well as provide strategies for reducing the risk of overestimation, as described above. Seventh, despite a surge in research on e-healthcare since the COVID-19 pandemic,[ only one[ study included in this review reported the impact of e-healthcare intervention on the mental health of nurses during COVID-19. However, considering that studies on nurses’ mental health problems related to COVID-19[ and the applicability of e-healthcare in the COVID-19 era[ are emerging, research investigating the impact of e-healthcare interventions on improving nurses’ mental health related to COVID-19 should be more encouraged. Eighth, the purpose of this systematic review was to analyze the effectiveness of e-healthcare interventions on mental health aspects in nurses, based on only RCTs, the gold standard for effectiveness research. However, the included studies could not demonstrate its effectiveness robustly. Therefore, in future studies, clinical studies of various designs dealing with this topic, not limited to RCT, can be reviewed.[ To account for the heterogeneity of e-healthcare interventions found in this review as well as the effectiveness of e-healthcare interventions, it may be helpful not to limit the scope of the review to RCTs. However, even in this case, a potential risk of bias in the study design should be carefully considered. Additionally, the issue of including non-RCTs in future systematic reviews should be sufficiently addressed in the protocol along with its rationale.[Finally, most control conditions used in the included studies were not intervention, suggesting that potential placebo effects may affect the results of the included studies. Therefore, future studies are encouraged to use attention control conditions, such as the positive thinking book used in the Motamed-Jahromi et al .[ This may rule out potential placebo effects of intervention and help identify differences between e-healthcare and offline healthcare strategies.

Conclusion

This review investigated the impact of e-healthcare interventions on burnout and other mental health aspects of nurses. There was some evidence that the online form of the EFT significantly improved burnout severity compared with no intervention. In addition, other outcomes, such as career identity, quality of work life, workplace bullying, job stress, turnover intention, distress, anxiety, and resilience, were also reportedly improved by web-based, smartphone-based, and/or real-time online interventions. However, due to the poor methodological quality of the included studies and the wide heterogeneity of the interventions and outcomes, further high-quality clinical trials in this area should be conducted in the future.

Author contributions

Conceptualization: Chan-Young Kwon.

Funding acquisition: Chan-Young Kwon.

Methodology: Jung-Hyun Park, Su-Eun Jung, Da-Jung Ha, Boram Lee, Chan-Young Kwon.

Supervision: Chan-Young Kwon.

Writing – original draft: Jung-Hyun Park, Su-Eun Jung, Da-Jung Ha, Chan-Young Kwon.

Writing – review & editing: Boram Lee, My-Sung Kim, Kyo-Lin Sim, Yung Hyun Choi, Chan-Young Kwon.