Implementation of an educational program for nursing students amidst the Ebola virus disease epidemic.

BACKGROUND: The global Ebola virus disease (EVD) epidemic of 2014/2015 prompted faculty at Emory University to develop an educational program for nursing students to increase EVD knowledge and confidence and decrease concerns about exposure risk.
PURPOSE: The purpose of this article is to describe the development, implementation, and evaluation of the EVD Just-in-Time Teaching (JiTT) educational program.
METHODS: Informational sessions, online course links, and a targeted, self-directed slide presentation were developed and implemented for the EVD educational program. Three student surveys administered at different time points were used to evaluate the program and change in students' EVD knowledge, confidence in knowledge, and risk concern. DISCUSSION: Implementation of a JiTT educational program effectively achieved our goals to increase EVD knowledge, decrease fear, and enhance student confidence in the ability to discuss EVD risk. These achievements were sustained over time.
CONCLUSION: JiTT methodology is an effective strategy for schools of nursing to respond quickly and comprehensively during an unanticipated infectious disease outbreak. Copyright Â

Introduction

The Ebola virus disease (EVD) epidemic of 2014/2015 presented Atlanta-area health care providers, health care professions schools, and students a unique challenge to quickly prepare for the care of EVD-infected aid workers from African countries affected by this disease. The decision to accept these patients resulted in the activation and expansion of the serious communicable diseases unit (SCDU) at Emory University Hospital (Feistritzer, Hill, Vanairsdale, & Gentry, 2014). Intense public interest followed the decision and resulted in tremendous media coverage. Between July 31 and September 22, 2014, >42,000 stories went out on broadcast and >18,000 print stories were written mentioning Emory and Ebola (“Telling the Story,” 2014). Some of the attention heightened the fear and anxiety associated with caring for individuals in our community because of the highly infectious nature of EVD. People spoke out on social media, fearing that our caring for these patients put our larger community at risk. In response to the public outcry, Susan Grant, the Chief Nurse for Emory Healthcare wrote in the Washington Post, “We can either let our actions be guided by misunderstanding, fear, and self-interest or we can lead by knowledge, science and compassion. We can fear, or we can care.” (Grant, 2014).

The Emory University Nell Hodgson Woodruff School of Nursing (NHWSN) is located on the same campus as Emory University Hospital and is also adjacent to the Centers for Disease Control and Prevention (CDC). Both the CDC and Emory Healthcare are key partners for the clinical and public health education of our student nurses. The treatment of patients with EVD at Emory University Hospital, combined with our CDC colleagues' response to the EVD epidemic in Africa and the status of Atlanta being a major international transportation hub, necessitated a swift response by key public health faculty and administration of the NHWSN to educate our students and fellow faculty colleagues and staff members about EVD. EVD education needed to include modes of transmission, risk for exposure and transmission, signs and symptoms of infection, therapy, and counseling techniques to allay fear and anxiety associated with living in Atlanta and working or training within the health care facilities treating EVD-infected patients. It was our goal to increase EVD knowledge, decrease fear, and enhance students' confidence in their ability to discuss EVD risk with family and friends.

Just-in-Time Teaching (JiTT) is an online educational approach that can be used to rapidly disseminate important information in an efficient and effective way to address learning needs during a crisis (Chotani et al., 2003). JiTT approaches have been used to quickly disseminate information after large-scale disasters and public health epidemics, such as the global outbreak of severe acute respiratory syndrome (SARS) that occurred in the early 2000s (O'Connor et al., 2009, Yang et al., 2010). Providing information expeditiously during complex humanitarian emergencies, such as a disease outbreak, is essential to quelling the fears of nursing students, who may encounter affected patients during clinical rotations, and communities who are uncertain about essential facts and who might be influenced by media coverage that at times dwells on unpleasant details and fuels the public's apprehensions (Stirling et al., 2015, Teaching, 2015). To respond to the emergent EVD epidemic, we designed a comprehensive and targeted approach to educate our students. The purpose of this article is to describe the development, implementation, and evaluation of this educational effort.

Methods

Dissemination of EVD Education

Early in the Fall Semester of 2014, we arranged for lunch-and-learn presentations, inviting all community members to learn more about the EVD outbreak in Africa. We invited colleagues from the CDC to present information about their experiences in Sierra Leone, one of the EVD-affected countries. Interested students and faculty attended other educational events at our university's School of Public Health. Specific to information about the EVD patients being cared for at Emory's SCDU, many attended a town hall meeting held jointly with the medical school where attendees heard directly from the SCDU team that was caring for the individuals with EVD.

In addition to the opportunities provided to learn more about EVD across our campus, the faculty decided that because our undergraduate nursing students were engaging in clinical training within the health care facility caring for patients with EVD, a more comprehensive and targeted approach to educate our students was needed. Additional goals for providing education were to increase student knowledge of EVD risks and ways to mitigate exposure, decrease fear of EVD, and enhance students' confidence in discussing EVD with others, including family, friends, and patients. Faculty course coordinators of classes addressing professional role content for each cohort of undergraduate students created an Ebola information page on their electronic course sites. The Ebola information content included links to CDC, Emory Healthcare, and other Atlanta-area health care EVD policies and guidelines. In addition, a 23-slide PowerPoint presentation was developed using CDC guidelines and the newly developed Emory Healthcare Ebola Preparedness Protocols. The presentation, posted on the course sites, included an overview of the EVD outbreak, EVD facts, modes of transmission, signs and symptoms of early and later stage infection, Emory University's EVD-specific travel policies, Emory Healthcare's publically available Ebola Preparedness Protocols, and CDC's published “Frequently Asked Questions” and answers. The presentation was designed for students' self-directed viewing and learning.

Participants

Participants targeted for this educational program were all undergraduate student nurses enrolled in our prelicensure bachelor of science in nursing (BSN) program at NHWSN in Fall 2014 and Spring 2015. Inclusion criteria included all enrolled undergraduate students; there were no exclusion criteria. Sample size was determined by the size of the enrolled undergraduate student population. This target group consisted of a total of 320 undergraduate students who were 93% women, 36% white, 24% black, 11% Asian, 7% Hispanic, and 20% multiracial/ethnic or undeclared. Consultation with the Emory University Institutional Review Board confirmed that this project met exemption criteria.

Program Implementation

Early in December 2014, the project's data manager (B.B.) invited all students in the BSN program to participate in the EVD self-directed education program described previously, via e-mail. The data manager did not serve in a faculty capacity to any of the students at the time. The e-mail stated that faculty were interested in students' perceptions about educational information that had already been provided to them about EVD and their experiences and level of comfort with discussing EVD with others, such as family members. The e-mail invitation described that the new education program included completing a pretest, viewing a PowerPoint slideshow, and completing two post-tests (immediately after the training and five weeks later) and that participation was completely voluntary. The pre- and post-tests were identical. Students were enrolled in one of three classes, and faculty members teaching those classes agreed to offer a small incentive in the form of bonus points to students who participated in the study. Application of the bonus points was determined by the individual faculty member for each course.

The invitation e-mail included a link to the pretest, which was hosted on the Research Electronic Data Capture (REDCap) platform. REDCap allowed for tracking participants for comparison on pre- and post-test results by linking student identification numbers that were loaded into the system. Time to complete the pretest was estimated to be about 15 min. The pretest link remained active for 3 days after the initial e-mail was sent. The pre-test survey consisted of three demographic questions, one item related to who they may have already provided any EVD information to, two questions related to the student's confidence level providing education to others about EVD, one item asking if they felt they needed additional EVD training, 13 knowledge questions, two questions related to the student's level of concern about their risk to EVD, and one question about attendance at recent campus educational programs about EVD. A few of the students either did not receive the initial e-mail or did not receive a valid link to the pre-test; these issues were resolved by sending e-mails to these students individually. The survey link was e-mailed again to students who experienced technological difficulties; no duplicate surveys occurred as a feature of the RedCap system.

Three days after the pre-test survey closed, faculty of the students' classes posted the PowerPoint slideshow on their online course sites, and participants were directed where to find the slideshow for viewing. Participants did not have access to the PowerPoint slideshow before completing the pre-test. Time to completely view the slideshow was estimated to be about 15 minutes.

The PowerPoint slideshow was available to view for three days. Students who participated in the pretest were invited via e-mail to complete the post-test. The post-test was also hosted on the REDCap platform, and survey responses were linked by student identification numbers. The post-test link remained active for three days.

After the post-test, students were on an academic break for approximately five weeks. When classes resumed in January 2015, students who had participated in the pre- and post-test surveys were invited to participate in a follow-up post-test. The purpose of this follow-up post-test was to assess the retention of knowledge and any changes in confidence in addressing EVD concerns after a major school break. The REDCap system linked survey results across the three tests with student identification numbers. The data manager provided student identification numbers to faculty for the purpose of awarding bonus credit. Bonus credit was awarded to students who completed all three surveys.

Data Analyses

Statistical analyses were performed using the Statistical Package for the Social Sciences, version 22 (SPSS, Chicago, IL). Statistical significance was set at p <.05 a priori. Data were reviewed for completeness. Any skipped or missing items were summarized and reported for items not related to the knowledge test. Missing items on the knowledge test portion were treated as incorrect responses. Most of the data collected were categorical and ordinal in nature; thus, most items were summarized using percentages and frequencies. Age was normally distributed, so the mean and standard deviation were reported. Descriptive statistics were compiled for all student characteristics, demographics, knowledge test scores, training needs, comfort, and confidence items. Knowledge scores were computed as the percentage correct out of the 13 EVD content items. The two concern items and three confidence items were scaled with four response ordinal categories (not at all, somewhat, very, and extremely). The two concern items were averaged together, and the three confidence items were averaged together. Reliability was assessed for each of the averaged items using standardized Cronbach alpha and the split-half Spearman–Brown formula (Eisinga, Grotenhuis, & Pelzer, 2013). Average concern was significantly right skewed and still ordinal in nature and was dichotomized into subjects who were “not at all” concerned (score = 1) vs. those who were somewhat to extremely concerned (scores > 1). Average confidence was also right skewed and still ordinal in nature and was also dichotomized into subjects who were “not at all” or “somewhat” confident (scores ≤ 2) and those who were “very” or “extremely” confident (scores > 2).

Multilevel modeling (MLM) instead of repeated measures analysis of variance was used to test for changes over time for the three time points for the continuous knowledge scores because MLM uses all available data and adjusts for missing data over time (Hedeker & Gibbons, 2006). For the dichotomized items for concern, confidence, and needing additional training, generalized multilevel modeling (GzMLM) was used for these binary response variables with logit link function (e.g., logistic regression) to test for changes over time. Age was also included as a covariate since older students may have had more confidence or comfort levels. For consistency, age was included in all the MLM/GzMLM models. For all models, pairwise comparisons were made between responses at all three time points using Sidak Type I error rate–adjusted p values (“IBM SPSS Statistics for Windows, Version 22.0,” 2013). Pairwise differences between T1 and T2 evaluated the initial improvements immediately after training, between T1 and T3 evaluated the longer term improvements from baseline, and between T2 and T3 evaluated the sustained or retained effects from the training.

Findings

Baseline surveys were completed by 233 (73%) of 320 eligible undergraduate students. The age of students participating in this study ranged from 19 to 52 years with an average age of 26.2 (standard deviation = 7.2) years. The majority were female (93.1%) with 39.5% juniors, 34.8% seniors, and 25.7% accelerated undergraduates. When asked who the students had previously provided any information about the Ebola outbreak to, the majority (>83%) said friends and family, and slightly less than half (41%) said fellow students (Table 1 ). Of the 233 who completed the baseline surveys, 192 (82.4%) completed the immediate post-test survey and 145 (62.2%) completed the final post-test survey. The 88 who did not complete all three surveys were not significantly different from the 145 who completed all three in age, gender, baseline knowledge, concern, confidence, or wanting additional training.

Table 1

Demographics for 233 Students

Demographic	Category	Mean (Standard Deviation)	Min–Max
Age		26.2 (7.2)	19–54
Gender; count (%)	Female	217 (93.1)
	Male	16 (6.9)
Student status; count (%)	Junior	92 (39.5)
	Senior	81 (34.8)
	Accelerated	60 (25.7)
Information provided to; count (%)	Family	208 (89.3)
	Friends	194 (83.3)
	Coworkers	24 (10.3)
	Patients/families	26 (11.2)
	Fellow students	96 (41.2)
	Neighbors	24 (10.3)
	Community acquaintances	28 (12.0)

Initially, the students scored 75.9 (12.7) on the knowledge test and improved immediately after training with scores averaging 90.7 (12.4) which was significantly higher than baseline (p < .001; Table 2 , Figure 1 ). Their knowledge scores were well retained by the third time point with average scores of 89.8 (11.6) with no significant loss in knowledge scores from Time 2 (p = .514).

Table 2

Unadjusted Summary Statistics for Outcomes at Three Time Points

Characteristic	Time 1 (n = 233)	Time 2 (n = 192)	Time 3 (n = 145)
Ebola virus disease (EVD) knowledge test score; mean score (standard deviation)∗	75.9 (12.7)	90.7 (12.4)	89.8 (11.6)
Average concern count (%); not at all†	122 (52.4)	114 (59.4)	98 (68.1)‡
Concerned about your risk as a health care provider count (%); not at all†	125 (53.6)	119 (62.0)	99 (68.8)‡
Concerned about your risk as a resident count (%); not at all†	153 (65.7)	137 (71.4)	107 (74.3)‡
Need additional training about EVD count (%); yes†	110 (47.2)	69 (35.9)	37 (25.5)
Average confidence count (%; >2); very/extremely†	102 (43.8)	135 (70.3)	91 (62.8)
Confidence in discussing EVD with family/friends count (%); very/extremely†	68 (29.2)	118 (61.5)	72 (50.0)
Confidence in answering questions about EVD transmission count (%); very/extremely†	69 (29.6)	116 (60.4)	72 (49.7)
Confidence in conveying calm message about general public's risk for EVD count (%); very/extremely†	92 (39.5)	120 (62.5)	90 (62.1)

EVD knowledge test scores analyzed using multilevel modeling (MLM).

Dichotomous outcomes analyzed using generalized multilevel modeling (GzMLM) with binary responses with logit link functions. The categories indicated by the counts and percents reported were the target category for the binary response logit link GzMLM.

One subject skipped answering the concern items at Time 3.

Figure 1

Estimated percentages (means and 95% confidence intervals) from multilevel models. Note: All estimated means and 95% confidence intervals were adjusted for age as a covariate in the multilevel models. Test scores were analyzed using multilevel models (MLMs) and the other three outcomes (additional training [% yes], average concern [% not at all], and confidence average [% very or extremely]) were analyzed using generalized multilevel models (GzMLMs) with binary responses and logit link functions. p values are provided for each pairwise comparison between the three time points and were adjusted using Sidak pairwise error rate correction.

At baseline, only half (52.4%) were not at all concerned about their risk (averaged from concern as a health care provider and as an Atlanta city resident). These two items showed good internal consistency and reliability with a standardized Cronbach alpha and split-half Spearman–Brown coefficient of 0.865. This percentage of students not at all concerned did increase significantly over time with improvements from baseline to Time 3 (p = .001) with 68.1% not at all concerned by Time 3 (Table 2, Figure 1). When looking at the two individual concern items (concern as a health care provider and concern as a resident), the levels of not at all concerned was consistently lower for risk as a health care provider, but both showed steady increases over all three time points (Table 2).

At baseline, slightly more than half (52.8%) stated they did need additional training about EVD, but this decreased significantly over time with significant decreases from Time 1 to Time 2 (p = .010) and from Time 2 to Time 3 (p = .048) with overall decreases from Time 1 to Time 3 (p < .001) down to only 25.5% wanting additional training by Time 3 (Table 2, Figure 1). At baseline, only 43.8% of the students were very or extremely confident in their average ability to discuss EVD with family/friends, answer questions about EVD transmission, and convey a calm message about the general public risk for EVD. These three items showed high internal consistency and reliability with a standardized Cronbach alpha of 0.905. The average confidence increased significantly from baseline to immediate post-test to 70.3% (p < .001), but this confidence level decreased slightly by the final post-test at Time 3 down to 62.8% which was not significantly less than Time 2 (p = .135) and was still significantly higher than baseline (p < .001; Table 2, Figure 1). When looking at the individual confidence level items, the lowest confidence levels were for discussing EVD with family/friends and answering questions about EVD transmission. The confidence levels for conveying a calm message about the general public's risk for EVD were consistently higher across time (Table 2).

A final detailed summary of the percentage of correct answers to the 13 individual knowledge test items at all three time points is provided in Table 3 . Reviewing this table shows that the weakest knowledge areas were for knowing how the Ebola virus infection is diagnosed (Item 12 with baseline knowledge at 31.8%), knowing how long protection lasts for people who recover from Ebola (Item 14 with baseline knowledge at 33.5%), and knowing whether you can still contact the virus from a person not showing symptoms (Item 13 with baseline knowledge at 49.8%). These three items all showed improvement from baseline to immediate post-test at Time 2, but Items 13 and 14 showed the poorest retention by the third time point at the final post-test. Two additional items with lower levels of knowledge at baseline were Item 18 with only 62.7% knowing if supportive care was currently the only treatment available for Ebola patients and Item 20 with only 62.7% knowing how health care responders returning from other countries should be monitored. However, after training, both of these items showed significant and sustained improvement with knowledge levels above 80%. The remaining knowledge test items showed reasonable levels of knowledge at baseline above 80% that improved to 90% and higher over time.

Table 3

Percent Correct for 13 Knowledge Test Items

Question	Time 1 (N = 233)		Time 2 (N = 192)		Time 3 (N = 145)
	n	%	n	%	n	%
8. Where is the Ebola outbreak concentrated at this time: (correct = West Africa)	229	98.3	187	97.4	141	97.2
9. Ebola is caused by a: (correct = virus)	226	97.0	188	97.9	140	96.6
10. Ebola is transmitted to others through: (correct = infected body secretions)	216	92.7	188	97.9	144	99.3
11. The incubation period for Ebola is 21 days. (correct = TRUE)	216	92.7	179	93.2	139	95.9
12. Ebola virus infection is currently diagnosed by measuring: (correct = RNA levels in the blood)	74	31.8	160	83.3	118	81.4
13. If a person is not showing symptoms, you can still contact the virus from them? (correct = FALSE)	116	49.8	160	83.3	114	78.6
14. According to our best information, people who recover from Ebola have antibody protection for at least: (correct = 10 years)	78	33.5	143	74.5	89	61.4
15. Sudden onset of fever is one of the first signs of Ebola. (correct = TRUE)	227	97.4	187	97.4	143	98.6
16. According to CDC guidelines, a person is only considered at high risk if the person has been in direct contact with infected body fluids from a patient with Ebola virus infection. (correct = TRUE)	203	87.1	186	96.9	139	95.9
17. Currently, there are no FDA-approved treatments for Ebola. (correct = TRUE)	203	87.1	178	92.7	134	92.4
18. Supportive care is currently the only treatment available for Ebola patients. (correct = TRUE)	146	62.7	160	83.3	130	89.7
19. Ebola is spread by mosquitoes. (correct = FALSE)	220	94.4	188	97.9	140	96.6
20. Health care responders returning from other countries should be: (correct = monitored for symptoms according to a set protocol)	146	62.7	161	83.9	121	83.4

Note. CDC, Centers for Disease Control and Prevention; FDA, Food and Drug Administration.

Discussion and Recommendations

The West African EVD epidemic of 2014/2015 that brought Ebola-infected patients to the metro-Atlanta area and to a hospital in which our student nurses were completing clinical education rotations provided a unique opportunity for the faculty at the NHWSN to prepare our student nurses for a major, fear-provoking public health event and to test the effectiveness of an educational program. There is little study devoted to the response of health professional schools, particularly schools of nursing in the event of an unforeseen infectious disease outbreak such as EVD (Stirling et al., 2015). Furthermore, there is little guidance for how to swiftly and effectively prepare nursing students for such events, both in their roles as patient providers and community educators.

The implementation of a JiTT educational program effectively achieved our goals to increase EVD knowledge, decrease fear, and enhance students' confidence in their ability to discuss EVD risk. Furthermore, these achievements were sustained over time. This demonstration educational program highlights the effectiveness of self-directed learning, especially in times of a threatening disease outbreak.

Limitations to this educational program included a substantial decrease in the number of student participants who completed the final survey from the baseline measurement time point. This decrease aligned with the level of course credit or bonus points provided to students, indicating greater student motivation to complete the full program when credit was awarded in meaningful ways to students. Giving extra credit points could also be a limitation of the program findings as it may not be representative of students who did not need extra credit (i.e., students with better course grades). The challenge with implementing consistent bonus points was having differing courses over two separate semesters. Greater coordination among faculty and throughout the courses might have helped to encourage student participation. The student nurse population at Emory University is primarily female, reflecting common gender norms of the nursing profession. This may, however, limit the generalizability of these findings to other more gender-balanced student groups.

Conclusions

The JiTT methodology and self-directed learning are effective means of increasing knowledge and confidence and decreasing risk concern among student nurses. In this era of globalization, when any communicable illness is “only a plane ride away” and intense media coverage can increase fear and anxiety, JiTT is a successful method of delivering evidence-based information to students in a timely manner. Schools of nursing must have the tools and resources to respond quickly and comprehensively during an unanticipated infectious disease outbreak to protect their students and staff, to prevent disease, and to be empowered advocates of accurate information in the midst of an epidemic.