Adaptation of EPEC-EM Curriculum in a Residency with Asynchronous Learning.

OBJECTIVE: The Education in Palliative and End-of-life Care for Emergency Medicine Project (EPEC™-EM) is a comprehensive curriculum in palliative and end-of-life care for emergency providers. We assessed the adaptation of this course to an EM residency program using synchronous and asynchronous learning.
METHODS: Curriculum adaptation followed Kern's standardized six-step curriculum design process. Post-graduate year (PGY) 1-4 residents were taught all EPEC™-EM cognitive domains, divided as seven synchronous and seven asynchronous modules. All synchronous modules featured large group didactic lectures and review of EPEC™-EM course materials. Asynchronous modules use only EPEC™-EM electronic course media for resident self-study. Targeted evaluation for EPEC™-EM knowledge objectives was conducted by a prospective case-control crossover study, with synchronous learning serving as the quasi-control, using validated exam tools. We compared de-identified test scores for effectiveness of learning method, using aggregate group performance means for each learning strategy.
RESULTS: Of 45 eligible residents 55% participated in a pre-test for local needs analysis, and 78% completed a post-test to measure teaching method effect. Post-test scores improved across all EPEC™-EM domains, with a mean improvement for synchronous modules of +28% (SD=9) and a mean improvement for asynchronous modules of +30% (SD=18). The aggregate mean difference between learning methods was 1.9% (95% CI -15.3, +19.0). Mean test scores of the residents who completed the post-test were: synchronous modules 77% (SD=12); asynchronous modules 83% (SD=13); all modules 80% (SD=12).
CONCLUSION: EPEC™-EM adapted materials can improve resident knowledge of palliative medicine domains, as assessed through validated testing of course objectives. Synchronous and asynchronous learning methods appear to result in similar knowledge transfer, feasibly allowing some course content to be effectively delivered outside of large group lectures.

INTRODUCTION

Over 2.4 million Americans die each year,1 with approximately 379,000 deaths occurring in emergency departments (ED).2 Unfortunately, the quality of care experienced by patients who are nearing death is inconsistent and sometimes poor, as physicians across multiple specialties report feeling inadequately trained to manage the special needs of patients at the end-of-life. 3–14 Terminally ill patients need health providers with excellent communication skills and knowledge of palliative medicine.14–17

The Education in Palliative and End-of-life Care for Emergency Medicine Project (EPEC™-EM) is a federally-funded, multi-center curriculum design and clinical research study that has developed a comprehensive training course in palliative and end-of-life care for emergency providers.18 Although EPEC™-EM teaching materials are designed for easy dissemination to physician and nurse learners, emergency medicine (EM) educators may still encounter challenges to using this curriculum when teaching emergency palliative care to novices. To provide the EPEC™-EM course in its entirety, significant instruction time must be added to an already full resident or staff development curriculum. Also, the nature of palliative care education requires the simulated practice of new skills, which can be difficult for faculty to facilitate in large group settings.19

One suggested method of incorporating new course material into an existing educational program is through curriculum adaptations that use both synchronous and asynchronous learning.20–21 Synchronous learning refers to the teaching of a group of students who all attend the same instructional session at the same time. Asynchronous learning is defined as individualized, self-directed learning that occurs away from groups of similar level learners.20 Although asynchronous learning may afford greater flexibility when redesigning a curriculum, it is still unclear if such educational experiences are as effective as traditional synchronous instruction. Specifically, we are not aware of any studies comparing the effectiveness of synchronous and asynchronous learning in an EM residency didactic program in the United States.

The objectives of this study are (1) to describe a local adaptation of the EPEC™-EM curriculum in an EM residency program and (2) to compare asynchronous and synchronous learning in our curriculum adaptation.

METHODS

Study Design

This curriculum development project used a six-step design approach defined by Kern et al22, as outlined in Figure 1 and further described in the study protocol. We conducted targeted program evaluation for impact objectives reflecting knowledge acquisition through a prospective case-control crossover study, with synchronous learning serving as the quasi-control.

Figure 1

Summary of EPECTM-EM adaptation using Kern’s Curriculum Development for Medical Education: A Six Step Approach.22

Study Setting

This EPEC™-EM curriculum adaptation was designed and implemented for EM residents at Northwestern University, Feinberg School of Medicine. Northwestern Memorial Hospital is the primary clinical training site for the residency program, with an urban ED population of approximately 81,000 patient visits per year. An inpatient palliative medicine service is on site with 16 available beds. Terminally ill patients who routinely present to the ED for emergency and end-of-life care generally receive treatment from one of four highly active specialty services at the hospital: geriatrics, hematology-oncology, transplant surgery and heart failure. Hospice patients are occasionally transported to the ED as well.

Study Population

Subjects included a convenience sample of all EM residents in academic year 2007–08 at the Northwestern Emergency Medicine Residency, a PGY 1–4 training program. We excluded subjects if they were unavailable or unwilling to participate in an unannounced pre-test or if they were unable or unwilling to participate in the post-test. Residents were informed that de-identified pre-test and post-test performance data may be used for educational research purposes.

The EPEC™-EM Curriculum18

The EPEC™-EM course was developed by a national, multi-disciplinary advisory board of experts in EM, hospice and palliative medicine, geriatrics and oncology. The curriculum is comprised of 14 learning modules that reflect core cognitive domains of emergency palliative care. Replication of the entire course requires a total of 16 hours of instruction time. Course material is disseminated nationally through the “Become an EPEC™-EM Trainer” Conference, using a train-the-trainer approach in which conference attendees are taught both the course material, as well as methods to teach that material at their home institutions. The EPEC™-EM Project provides each trainer with an electronic copy of the detailed course syllabus, instructors’ manual and modifiable slides sets for all lectures. Trainers are encouraged to adapt course materials to the educational needs of learners at their individual institutions. Conference attendees participate in a written examination at the beginning and end of the conference used to assess knowledge acquisition. This examination is comprised of at least three multiple choice questions per module, selected from a bank of items written and validated by the faculty of the EPEC™-EM Project.

Study Protocol

We obtained local institutional review committee approval to train EM physicians in EPEC™-EM at our institution. Our study team used Kern’s six-step curriculum design approach to adapt the EPEC™-EM course into our local residency didactic program.22 [Step 1] Problem identification and a general needs assessment were completed through a review of the existing residency curriculum for palliative and end-of-life care topics, which was compared to the learning objectives of each module in the EPEC™-EM course.18 [Step 2] Local needs assessment of targeted learners was performed with a written, multiple choice examination (pre-test) given to a convenience sample of all residents who attended conference on a single day in November 2007. The pre-test was the same 42-item examination administered to all attendees of the national EPEC™-EM conference held in Chicago, IL in August 2007. Subjects were not told of the timing or content of the pre-test before it was distributed. [Step 3] The same goals and objectives of each individual module of the national EPEC™-EM course were chosen for the new residency curriculum.18 [Step 4] The educational strategies selected included: synchronous presentation for half of the EPEC™-EM modules using large group teaching (featuring didactic lectures, case based discussions and role playing) and asynchronous learning for half of the modules through resident review of EPEC™-EM course materials. [Step 5] The curriculum was implemented over a four-week period in March 2008. Electronic copies of the EPEC™-EM course syllabus, lecture slides and a self-study examination (post-test) were distributed at the beginning of the month. The post-test was collected in April 2008, at which time an answer key was provided to all residents, regardless of their participation in the pre-test or the post-test. The post-test was the same 42-item multiple choice examination administered at the national EPEC™-EM conference held in New Orleans, LA in February 2008. Though the pre-test and the post-test use different questions, the items tested similar learning objectives and course content. The seven modules presented for synchronous learning were chosen based on faculty preference for ease of presentation in a large group setting. Six of the seven synchronous modules were taught by physicians who had completed EPEC™-EM trainer certification at a national conference. The seven synchronous modules were each presented during an hour of conference in the last two weeks of the month to allow resident review of the EPEC™-EM course materials prior to the didactic sessions. Asynchronous modules consisted of resident self-study of unaltered electronic copies of the EPEC™-EM course media (course syllabus, instructors’ guide and lecture slide sets).18 Asynchronous modules were offered throughout the four-week curriculum and could be reviewed by the resident at any time. Each module required approximately one hour of self-study. Residents who were unable to attend a synchronous module lecture due to schedule conflicts were expected to review the corresponding EPEC™-EM course media in an asynchronous fashion. [Step 6] The curriculum adaptation was evaluated through a comparison of group performance on the pre-test and the post-test. The residency coordinator de-identified individual tests prior to data analysis.

Measurements

All EPEC™-EM course materials and examinations were used with permission of the EPEC™-EM Project. We compared resident group performance for each test item on both the pre-test and post-test with group performance by attendees at the respective national EPEC™-EM conferences for which the exams were originally administered. We removed items that performed differently between the two groups from the data analysis to reduce the effect of variability among individual lecturers who may have taught course material in a way that could skew responses for any single item. A significant difference in item performance was defined as a majority of respondents correctly or incorrectly answering an item that was observed to perform in the opposite way in the comparison group. Psychometric experts employed by the EPEC™-EM Project reviewed all item performance in a meeting with the study investigators to insure accuracy of testing.

Data Analysis

Examination items were grouped by module topic. We used descriptive statistics to report group scores for the entire residency program on the pre-test and posttest for each module topic. A difference in pre-test and post-test scores was calculated for each module. We categorized modules by learning method as asynchronous or synchronous. Our primary outcome was the aggregate mean difference in pre-test and post-test scores between learning methods. We compared this difference with 95% confidence intervals with significance defined as an interval that crosses 1.0.

RESULTS

Figure 1 summarizes the major findings of each of the six curriculum development steps used in this protocol.

Of 45 eligible residents in our program, 25 (55%) were present at conference on the day of the unannounced pre-test that was used for local needs analysis; all of these residents participated in the pre-test. Thirty-five residents (78%) completed the post-test. Item analysis revealed only one post-test question that performed differently between the resident cohort and the EPEC™-EM conference cohort; we removed this post-test item from module #7 from further data analysis.

Tables 1 and 2 summarize the percent difference in pre-test and post-test scores observed for the entire resident cohort across all 14 learning modules, grouped by asynchronous or synchronous learning methods. Mean test scores of the residents who completed the post-test were: asynchronous modules 83% (SD=13); synchronous modules 77% (SD=12); all modules 80% (SD=12). Those modules that used asynchronous learning demonstrated a range of test score improvement between 5% and 57%, with a mean improvement of 30% (SD=18%). Modules presented using synchronous learning had test score improvement in a range of 11% and 41%, with a mean of 28% (SD=9%). The aggregate mean difference in pre-test and post-test scores grouped by learning method was 1.9% (95% CI, −15.3 and +19.0), demonstrating a lack of inferiority for asynchronous learning in our curriculum adaptation.

Table 1

Pre-test and post-test results by teaching method and module number

Teaching Method:	Asynchronous Modules
Module Number	Subject	Pre-Test % Correct	Post-Test % Correct	Difference (% Post-test - % Pre-test)
1	Prognostication and Trajectories	57	85	+ 28
2	Rapid Palliative Care Assessment	40	77	+ 37
3	Goals of Care	80	85	+ 5
4	Advance Directives	78	91	+ 13
5	Caring for Hospice Patients	13	57	+ 44
6	Grief and Bereavement	71	98	+ 27
11	Chronic Pain	34	91	+ 57
Teaching Method:	Synchronous Modules
Module Number	Subject	Pre-Test % Correct	Post-Test % Correct	Difference (% Post-test - % Pre-test)
7	Withdrawing and Withholding Care	67	97	+ 30
8	Family Witnessed Resuscitation	74	85	+ 11
9	Death Disclosure	67	93	+ 26
10	Common Symptoms	33	74	+ 41
12	Malignant Pain	40	74	+ 34
13	Complications of Cancer	26	58	+ 32
14	Last Hours of Living	38	62	+ 24

Table 2

Group performance by teaching method

Teaching Method:	Asynchronous Modules
	Pre-Test % Correct	Post-Test %Correct	Difference (% Post-test - % Pre-test)
Mean (SD)	53 (25)	83 (13)	30 (18)
Median (IQR)	57 (34, 78)	85 (77, 91)	28 (13, 44)
Teaching Method:	Synchronous Modules
	Pre-Test % Correct	Post-Test % Correct	Difference (% Post-test - % Pre-test)
Mean (SD)	49 (19)	77 (15)	28 (9)
Median (IQR)	40 (33, 67)	74 (62, 93)	30 (24, 34)

DISCUSSION

Our study illustrates a successful model for the local adaptation of a national curriculum in palliative and end-of-life care to an existing EM residency didactic program using both asynchronous and synchronous learning. We believe that our study is the first of its kind to suggest parity of knowledge transfer between asynchronous and synchronous learning in a United States EM residency training program.

Several developments led to the need for our residency curriculum redesign. In 2006, the American Board of Emergency Medicine (ABEM) co-sponsored the creation of the American Board of Hospice and Palliative Medicine.23 Subspecialty certification in Hospice and Palliative Medicine is now an option for all ABEM-certified physicians, and with this opportunity came an institutional desire to expose residents to new educational content. 23 In addition, training in palliative and end-of-life care has become a standard recommendation in numerous guidelines for competency-based education that address the goals of the Accreditation Council of Graduate Medical Education (ACGME) Outcome Project.7,10,13,15,24–35 General needs analysis of our residency didactic program revealed a paucity of exposure to core cognitive domains of emergency palliative care prior to this curriculum adaptation.

The challenge of incorporating a large amount of new course material within an existing residency didactic program is not a trivial concern to educators tasked with curriculum redesign. As a recent comparison, similar challenges were encountered with the emergence of ultrasound as a necessary educational requirement for EM training programs.36–38 Novel curricula often require the recruitment of additional faculty or the expert training of existing faculty at significant monetary and time costs. Competing educational objectives complicate the inclusion of new didactic material, especially if the required curriculum change is voluminous and inherently skills-based.

Asynchronous learning offers many potential benefits for residency directors when they redesign their didactic programs. Both a Council of Emergency Medicine Residency Directors subcommittee and the Residency Review Committee in Emergency Medicine (RRC-EM) have recommendations for the use of asynchronous learning.20,39 Advantages of asynchronous instruction include: customization of a curriculum to an individual learner’s schedule and needs; reduction in the burden of additional scheduled conference time for both resident learners and course faculty; an alternative learning style that may be preferred by younger generations of students; delivery of material to large numbers of learners in a systematic and consistent manner; practice of the life-long learning principles required of physicians; the use of faculty at other institutions when local expertise does not exist; and instant access to electronic media that can be easily revised and updated.20,40 Disadvantages of asynchronous learning include lack of social interaction among students and limited methods of real-time feedback.20 While there may be many compelling reasons to employ asynchronous learning, reports of the effective use of this methodology in EM education are limited.21,40–42

Asynchronous learning proved to be a useful method of delivering some novel palliative care education needed in our residency program. Our data suggests that asynchronous learning was similar to synchronous learning for selected course material in our population. Specifically, there was no meaningful difference in post-test scores between asynchronous and synchronous modules, with an aggregate mean difference between teaching methodologies across the entire cohort of only 1.9% (95% CI −15.3, +19.0).

Based on our general needs assessment, our population of learners was thought to be uniform in their lack of didactic and clinical exposure to core cognitive domains of emergency palliative care. Therefore, we studied the effect of our curriculum design on the overall cohort rather than each individual learner. This is an especially practical model for residency directors who need to introduce new material to an entire residency program for the first time.

Though education literature is replete with descriptions of various curricula used to teach palliative care to residents7,24–25,28,32, none describe a method for incorporating an existing course to an EM residency program. The use of locally adapted EPEC™-EM conference materials appears to be an effective method of delivering asynchronous self-study modules for our curriculum adaptation. In our population, post-test scores improved across all EPEC™-EM domains with a mean of +29%.

As palliative and end-of-life care represents a relatively new area of focus within EM, many program directors may have difficulty identifying faculty members with the expertise to create appropriate instructional media. The EPEC™-EM curriculum provided a comprehensive and useful collection of teaching materials at a low cost of faculty training time and conference participation. In specific, we implemented the entire four-week curriculum adaptation with the assistance of three faculty members and three senior residents who were funded to attend a two-day EPEC™-EM trainer course. External faculty training via EPEC™-EM costs $745 for conference registration at 2009 rates, plus travel and requisite protected time per individual.43 Maintenance of curriculum may be significantly less expensive if intradepartmental faculty training can occur prior to future cycles of local dissemination.

It is important to note that the method of asynchronous learning used in our study differs from that described in the ACGME program requirements for EM. As defined in the RRC-EM guidelines, “emergency medicine programs may utilize individualized interactive instruction for up to 20% of the planned educational experiences.”39 We delivered 50% of our curriculum as asynchronous learning in order to have equal numbers of asynchronous and synchronous modules for ease of data analysis in this study. Programs that adopt our approach to teaching the EPEC-EM™ curriculum should plan for no more than 20% cumulative asynchronous instruction, in compliance with current RRC-EM requirements. Also, we did not individualize our instructional material according to resident year, as our general needs assessment indicated that our population of learners was uniform in their lack of exposure to emergency palliative medicine content. Future cycles of our curriculum can be edited for learner experience based on observed changes in our local needs assessment. It is expected that after a single cycle of instruction, the educational content would no longer be novel to all learners; the curriculum could then be tailored to intermediate and novice learners, or senior and junior residents. Lastly, the EPEC™-EM course media used in our study was not interactive, however we distributed post-test answer keys after the curriculum in an effort to provide feedback and encourage continuous self-study. The EPEC™ Project offers online versions of the original EPEC™ and EPEC™-Oncology courses that are interactive and may serve as a model for asynchronous learning that better comply with the RRC-EM guidelines.43

LIMITATIONS

The most significant limitation of our study is the potential for selection bias that results from both a small sample size of eligible subjects at a single residency program, as well as our protocol for pre-/post-testing. The pre-test was administered to a convenience sample of residents available on a single day of conference; the response rate might have been higher if we tested over multiple conference dates. The post-test was distributed several weeks ahead of its required due date, and it was returned by respondents at variable times before a final due date. A higher response rate was observed for the post-tests collected in this manner compared to the pre-tests, however, there is the potential that post-tests were completed either without review of the asynchronous materials or in an “open-book” or shared format by some respondents. Tests were not paired by individual resident, but rather the scores were averaged to observe aggregate group performance. We do not know which residents participated in either the pre-test or post-test, nor do we know if individual residents demonstrated any benefit from the curriculum or the pre-test itself. In addition, our analysis lacked comparison to a separate, nonintervention true control group that might have tested differently had they not received the curriculum or pre-test at all. While pre-test and post-test examinations tested learning objectives specific to a particular module, it is possible that content taught in synchronous modules may have helped subjects answer questions from asynchronous modules as well.

The testing materials that we used only assessed for knowledge acquisition. Our curriculum included goals and objectives for both new knowledge and skills; future cycles of our adapted curriculum will need to include skills assessments. We do not know if gains in knowledge were retained longer than the interval of time between the curriculum and the post-test. As comparison, we adapted our study from the national EPEC-EMTM course, which administers their validated post-tests immediately after each hourly lecture during the two-day conference. In our study the post-test was collected one month after the educational intervention, a significantly longer latency period than that used by the EPEC-EMTM Project. Future cycles of the curriculum might reveal higher pre-test scores if content was retained by the learners still in the residency program. Lastly, we did not assess health outcomes measures of patients with palliative or end-of-life care needs that might have been changed due to our educational curriculum.

CONCLUSION

EPEC™-EM adapted materials improved knowledge of core palliative medicine domains for a cohort of residents, as assessed through validated testing of course objectives. Asynchronous and synchronous learning appeared to result in similar knowledge transfer, feasibly allowing some course content to be effectively delivered outside of large group lectures. As palliative and end-of-life care emerges as an educational priority for emergency physicians, this curriculum adaptation model may be useful to other residency programs.