Development, implementation and evaluation of a pain management and palliative care educational seminar for medical students.

BACKGROUND: Despite calls for the development and evaluation of pain education programs during early medical student training, little research has been dedicated to this initiative.
OBJECTIVES: To develop a pain management and palliative care seminar for medical students during their surgical clerkship and evaluate its impact on knowledge over time.
METHODS: A multidisciplinary team of palliative care and pain experts worked collaboratively and developed the seminar over one year. Teaching methods included didactic and case-based instruction, as well as small and large group discussions. A total of 292 medical students attended a seminar during their third- or fourth-year surgical rotation. A 10-item test on knowledge regarding pain and palliative care topics was administered before the seminar, immediately following the seminar and up to one year following the seminar. Ninety-five percent (n=277) of students completed the post-test and 31% (n=90) completed the follow-up test.
RESULTS: The mean pretest, post-test and one-year follow-up test scores were 51%, 75% and 73%, respectively. Mean test scores at post-test and follow-up were significantly higher than pretest scores (all P<0.001). No significant difference was observed in mean test scores between follow-up and post-test (P=0.559), indicating that students retained knowledge gained from the seminar.
CONCLUSIONS: A high-quality educational seminar using interactive and case-based instruction can enhance students' knowledge of pain management and palliative care. These findings highlight the feasibility of developing and implementing pain education material for medical students during their training.

Research suggests that the prevalence of chronic pain in Canada is approximately 19% (1). Individuals with chronic pain experience myriad challenges within physical, psychological and social domains. Reports indicate interference with daily activities and employment (2), higher levels of depression and anxiety disorders (3), and difficulties maintaining relationships (4). Not only do these issues significantly impact the quality of life of individuals with chronic pain, but they also place a considerable burden on the Canadian economy and health care system (1,5–7). Canada spends at least $6 billion annually on managing chronic pain, and the costs associated with reduced productivity due to chronic pain (eg, sick days, job losses) is estimated to contribute an additional $37 billion per year of economic burden (6). Researchers argue that, as the demographics change over time and middle-age and older adults represent a larger percentage of the population, chronic pain conditions will pose an even greater economic problem to our society (1,5). With this changing demographic, it is reasonable to assume that competent palliative and end-of-life care pain management will become even more important to reduce the socioeconomic impact on our society.

Despite the obvious need for health professionals to cultivate competent and compassionate pain management practices to provide high-quality care for chronic pain patients, research shows that the coverage of pain topics and formal pain training is scarcely available or not well integrated into the medical curriculum in North America (8–13) or overseas (14). With respect to the palliative and end-of-life care literature, a previous review demonstrated evidence for inadequate training for medical students (15). Research continues to document poor coverage on palliative care topics for Canadian undergraduate medical students (16) as well as a lack of opportunity for students to participate in palliative care rotations across North American and Western European undergraduate medical programs (17). Not surprisingly, the deficiency in pain and palliative care education has resulted in poor competence among undergraduate and postgraduate students as well as physicians for managing the complexity of chronic pain, and palliative and end-of-life care conditions (15,18,19).

As a result of the lack of pain and palliative care training in medical education, and in an effort to improve the life of those experiencing chronic pain, recommendations to include or enhance educational training in pain assessment and treatment for undergraduate and postgraduate students across all health professions has been continuously emphasized by researchers and health care organizations (20–22). To enhance competency in these areas, researchers, educators and medical practitioners have initiated the development, implementation and evaluation of pain and palliative care seminars, online webinars and integrated curricula (7,9,23–29).

In an effort to evaluate the effectiveness of these programs, investigators have included assessments of knowledge retention in pain management and practices, shifts in attitudes or beliefs about pain, and improved patient care or pain management practices. Studies have demonstrated short-term improvement in these areas immediately following the end of the program (7,9,23,27,28), as well as long term (24–26,29). Researchers have emphasized the importance of providing students with opportunities to engage in practical case-based exercises rather than exclusively providing didactic lecture formats (14,30,31) and that these programs should be a required component of medical training, not optional (21).

As a result of a review of the undergraduate curriculum in the Faculty of Medicine at the University of Toronto (Toronto, Ontario), the opportunity presented itself to address chronic pain management and palliative care. The Department of Surgery created a mandatory 4 h seminar in pain management and palliative care for medical students during their clerkship. At the time of the present study, the seminars were provided to the initial group of students during the fourth and final year, but were moved into the third, penultimate year of study for the later group of students due to change in the duration of the core clinical clerkship at the university.

An adaptation of Kirkpatrick’s model of training evaluation criteria for assessment in higher education (32) was used as a theoretical framework in the current study. The four levels include: reaction (student reactions and utility judgements of the program); learning (measures of learning outcomes such as knowledge tests); behaviour (measures of student use of knowledge and skills gained from the program and application to other settings); and results (evaluating the impact of education on individual accomplishments and contributions to society). We focused on evaluating the first two levels of Kirkpatrick’s adapted model.

Specifically, to address the reaction criterion, students were asked whether they perceived the seminar as useful for their practice. The learning criterion was addressed by administering a test of pain and palliative care knowledge immediately before and following the seminar, and up to one year later. Due to limited resources, a control group was not included. We recognize the limitations of using a pre-post design without a control group, particularly regarding the ability to determine that improvement in student learning is a result of the seminar and not some other factor. However, our main objective for the present study was to focus on developing and implementing a feasible quality seminar and to reach as many medical students in training as possible.

It was hypothesized that students would demonstrate a significant improvement in knowledge on pain and palliative care topics immediately following the seminar and at follow-up one year later, compared with pretest scores. We further hypothesized that students would retain the knowledge gained from the seminar after a delay; therefore, no differences were expected in test scores between post-test and follow-up. Finally, we expected that both third- and fourth-year students would benefit equally from the seminar and, therefore, no differences in test scores were expected between third- and fourth-year students.

METHODS

Participants

Undergraduate medical students from the University of Toronto completing their third- and fourth-year surgical clerkship participated in a 4 h seminar on pain management and palliative care (n=292), completing the pretest immediately before the seminar. Ninety-five percent (n=277) of students completed the post-test immediately following the seminar and 31% (n=90) completed the follow-up test via e-mail.

Procedure

In response to curriculum review, prompted by updates to the Liaison Committee on Medical Education requirements, the authors’ faculty determined there was insufficient teaching about chronic pain, and palliative and end-of-life care. In addition, the Educating Future Physicians in Palliative and End-of-Life Care project, a federally funded initiative to ensure education regarding palliative and end-of-life care, determined specific content areas that should be required of all graduating medical students. It was determined that didactic content was being provided in the preclerkship years but was missing from the clerkship years. Areas of missing content relating to chronic pain management, and palliative and end-of-life care were identified as contiguous with teaching within the surgical clerkship. Faculty from pain medicine, surgery and palliative care developed and presented these seminars, revising and updating content, for three years before the model used in this research was finalized.

The Mount Sinai Hospital Research Ethics Board (Toronto, Ontario) approved the present study. Informed consent was obtained from all students, and the first seminar was presented in October 2010 and repeated every six weeks throughout the year, so that all groups of surgical clerks attended the pain and palliative care seminar during their surgical rotation.

A 10-item test examining knowledge of pain management was administered in the classroom before the seminar, immediately following the seminar and at follow-up approximately one year later via e-mail. The course instructors (n=4) collaborated on the development of all test items. Using the seminar objectives and content template as a starting point, the instructors created a total of 20 multiple-choice questions. Each item was designed to assess a fact or apply a concept covered during the seminar. The goal of the item development process was to create items that reflected the breadth of seminar content. An education scientist and evaluation expert (NW) provided guidance on length of items and testing format. The final items were randomly divided into three sets of 10 questions to be administered before the seminar, immediately after the seminar and at follow-up. In both of the post-tests, five repeat items were mixed in with five new questions. This was included to ensure that the post-testing captured recall of content and not purely memory of test items. Although the repeat of a subset of items on delayed testing and the random assignment of items to tests limits the possibility that the tests varied in difficulty, they were not statistically examined for equivalence, which represents a limitation of the present study.

To assess students’ reactions regarding the seminar, they were asked to comment whether, and how, the information they learned from the seminar influenced their practice. Specifically, students were asked: “Did the information you learned in the Pain and Palliative Care seminar have an impact on your practice? If so, how? Please provide concrete examples.” The follow-up test was administered only after all 11 seminars were completed. Therefore, the duration between post-test and follow-up varied for participants (between two months and one year), depending on when the seminar was held during the students’ surgical rotation.

Seminar content

The primary purpose of the seminar was to enhance student learning about chronic pain management, and palliative and end-of-life care. The seminar focused on defining pain and its burden, assessing, diagnosing and treating pain, as well as considering common comorbidities. A spiral curriculum approach was used, building on related material presented in the Pain or Palliative Care Weeks during Foundations of Medical Practice, as well as complementary material during anesthesia, family medicine and pediatrics rotations. The purpose was to offer students the opportunity to integrate didactic and clinical cases within the context of their surgical rotation.

The seminars consisted of two sections. The first included a 20 min to 25 min interactive minilecture regarding an approach to chronic pain, followed by an exercise in writing legal narcotic prescriptions in Ontario. The second section consisted of a further 20 min to 25 min interactive minilecture about palliative and end-of-life care, with a specific focus on malignant bowel obstruction (see Table 1 for educational objectives of sections I and II). The seminars concluded with the students breaking up into smaller groups to address case-based questions building on these materials and reporting their responses back for a large group discussion.

TABLE 1

Educational objectives

Section I: Chronic pain

Section II: Palliative and end-of-life care

Outline the Five Pillars of pain management using the mnemonic ADDOP (33): Assessment; Defining the underlying condition; Diagnosis and establishing a treatment path; Other treatments embracing the biopsychosocial model and treating comorbidities; Personal management Use the concept of the Five Pillars in recommending pain management for selected patients Discuss the role of adjuvant medications in pain management Propose appropriate use of adjuvant medications in pain management

Review the pathophysiology of malignant bowel obstruction Review management options for malignant bowel obstruction Establish appropriate treatment options aimed at symptom control and reversing bowel obstruction Discuss the physician’s duty for self-care when providing assistance with pain management, and palliative and end-of-life care

Statistical analysis

Students’ scores were not matched across testing time points and student year of education was only recorded during the follow-up test via e-mail (n=44 in third year and n=39 in fourth year). As a result, a 2 (year of education: third and fourth year) × 3 (time: pretest, post-test and follow-up) repeated-measures ANOVA could not be conducted. A one-way ANOVA was conducted to test for differences in scores over time.

RESULTS

Of the total sample (n=292), 277 (95%) completed the post-test and 90 (31%) completed the follow-up test via e-mail. A one-way ANOVA was conducted to test for differences in scores (percentages of correct answers) across time. Mean test scores were significantly different among pretest, post-test and follow-up (F[2, 657] = 314.29, P<0.001, ηp2=0.488). Bonferroni post hoc comparisons indicated that mean (± SD) post-test scores (75±11) and follow-up scores (73±10) were significantly higher than pre-test scores (51±14; all P<0.001). Students’ test scores did not significantly decrease between post-test and follow-up (P=0.559) (Figure 1).

Figure 1)

Test score means as a function of time. Error bars represent SEM

An independent-samples t test was conducted to compare mean test scores between third- and fourth-year students at follow-up. No significant difference in test scores was observed between third-year (73±11) and fourth-year students (74±9) at follow-up (t[81]=−0.37; P=0.71).

In addition to the 10-item test administered at follow-up, students were asked: “Did the information you learned in the Pain and Palliative Care seminar have an impact on your practice? If so, how? Please provide concrete examples.” The total student response rate on this qualitative component was 45 of 90. These responses were subject to content analysis. Of 45 students who responded to this item, 20 responded ‘yes’, they did find the seminar had a positive impact on their practice. Responses fell into four main categories: prescription writing; pain and symptom control; improved use of pain and palliative medicine consults; and inspiration to continue pain and palliative education and experience. Fifteen students did not believe that the seminar positively impacted their practice. While no explanation was offered for most respondents, three reasons were provided: medical students are required to follow the attending physician’s instructions; opioids are overprescribed and dangerous; and the lecture was too long and should have been repeated. There were an additional 10 participants who either could not remember the lecture or responded ‘not applicable’.

DISCUSSION

The objective of the present study was to develop and evaluate the effectiveness of a pain and palliative care educational seminar on third- and fourth-year medical students’ knowledge during their surgical clerkship. A 10-item test was administered immediately before a 4 h seminar to assess baseline knowledge in pain, and palliative and end-of-life care topics. To measure improved knowledge over time, students were tested in class immediately following the seminar and at follow-up up to one year later via e-mail. Consistent with our predictions, we demonstrated that students’ mean test scores significantly improved from pretest (mean score 51%) to post-test (75%) and were sustained up to one year following the seminar (73%). These results extend previous findings of improved student knowledge immediately following pain education programs (7,9,23,27,28), and demonstrate that these effects can be sustained at least two months after a short seminar. We administered the follow-up test at one timepoint following all 11 seminars, rather than one year after each of the seminars, and we were unable to identify students who completed the seminar two months previously versus one year previously. Therefore, it is unclear which students participated in the follow-up test. Regardless, our results provide support that knowledge can be maintained at least two months and up to one year following the seminar.

Moreover, although we did not collect information regarding year of study during the pretest and post-test, no significant difference was obtained in test scores between third- and fourth-year students at follow-up. This finding supports our hypothesis that the seminar is useful for both third- and fourth-year medical students. This was not surprising given that studies have shown improvements throughout medical students’ undergraduate and postgraduate years (7,9,23,27,28). Indeed, researchers argue that pain management training should begin during the early years of medical training (21).

The qualitative component of the final test was limited to an open-ended question, which asked participants to indicate the impact of the seminar on their practice. Keeping this section short was in the interest of the medical students’ time to facilitate participation. It is unclear why the response rate for this item in the follow-up was so low (50%). Of the students who did respond, approximately one-half viewed the seminar as positive, one-third did not find it to be helpful and one-fifth could not recall the seminar. For some students, a significant length of time had passed between the seminar and follow-up (ranging from two months to one year), which may explain the number of students who failed to recall their attendance.

Participants indicated that the seminar had a positive impact on their practice and responses fell into four main categories: prescription writing; pain and symptom control; improved use of pain and palliative medicine consults; and inspiration to continue pain and palliative education and experience. Participant responses were well aligned with the material covered in the seminar. It is positive to note that students were inspired to continue education in end-of-life care and pain management following participation in the seminar. Of those who believed that the seminar did not positively impact their practice, only three reasons were provided and, therefore, examination for themes was not possible.

The present study had several limitations. First, we did not include a control group to compare students’ knowledge on pain and palliative care topics following the seminar with students who were not exposed to the seminar to rule out other factors that may be contributing to the improvement in pain and palliative care knowledge. Second, the attrition rate at post-test was 5% (n=272), but at follow-up test this increased to approximately 67% (n=90). Because the follow-up test was administered via e-mail between two months and one year after the delivery of the seminar, this attrition rate is not surprising. Given the large sample size at pretest, the number of responses at the follow-up test remained adequate. Third, it is possible that students’ knowledge on pain and palliative care topics were sustained at the follow-up test due to other reasons, rather than the seminar itself. For example, over the course of the study, students may have been exposed to similar pain management material as that covered during the seminar in their standard medical curriculum. In particular, students may have participated in a 1 h seminar during third year in medicine, one half-day in third-year family medicine, 2 h during anesthesia, elective courses in pain clinics or the Palliative Care Week during Foundations of Medical Practice. Therefore, the follow-up data should be interpreted with caution.

One of the primary challenges in developing the seminar was selecting topics deemed to be most important for improved knowledge in palliative care and pain management. Also, allocating time to develop and deliver the seminar was difficult. To optimize future success in this endeavour, we urge collaborations among pain and palliative care experts, medical education committees and health care organizations. Given that political and institutional barriers, and limitations in time and resources are often cited as obstacles for the successful development and implementation of palliative care and pain management education (21,34,35), joining efforts will reduce the burden on all those invested. Support from funding organizations is also required to improve the feasibility of conducting large-scale studies with more sophisticated designs, such as randomized controlled trials, that examine not only outcomes pertaining to the reaction (eg, utility judgements) and learning (eg, tests of knowledge) criteria from the adapted Kirkpatrick’s model of training evaluation, but also assessments of how students incorporate their knowledge in their practice and the degree to which this translates into improved patient care. This will allow for the identification of specific education program components that lead to improved outcomes.

Drawing on the results from the present study, we plan to modify the seminar and better target attitudes and beliefs about pain such as one concern expressed that “opioids are overprescribed and dangerous”. In fact, other studies have highlighted that, although improved pain knowledge is important for better-quality pain management services, student attitudes and beliefs about pain must also be addressed for this knowledge to be translated into clinical practice (20,24). To this end, we encourage future research to include more comprehensive qualitative components that assess current deficiencies in pain and palliative care education from the perspective of faculty and students. This will help to direct areas of focus for well-designed education programs, and process and outcome evaluations.

Our results demonstrate the feasibility of developing and implementing a high-quality educational seminar using interactive and case-based instruction for medical students during their training. In addition, we found significant improvement in students’ knowledge of pain management and palliative care practices following the seminar. Feedback from students and seminar facilitators will be discussed and incorporated for improved future delivery of this seminar. We hope that the present study will encourage future collaborations between pain and palliative care experts, faculty and researchers to develop similar educational components for their medical programs, and evaluate student competency and patient care practices. These initiatives are critical to provide effective and compassionate care for patients experiencing pain, and for reducing the burden on health care professionals.