Exploring assessment of medical students' competencies in pain medicine-A review.

INTRODUCTION: Considering the continuing high prevalence and public health burden of pain, it is critical that medical students are equipped with competencies in the field of pain medicine. Robust assessment of student expertise is integral for effective implementation of competency-based medical education.
OBJECTIVE: The aim of this review was to describe the literature regarding methods for assessing pain medicine competencies in medical students.
METHOD: PubMed, Medline, EMBASE, ERIC, and Google Scholar, and BEME data bases were searched for empirical studies primarily focusing on assessment of any domain of pain medicine competencies in medical students published between January 1997 and December 2016.
RESULTS: A total of 41 studies met the inclusion criteria. Most assessments were performed for low-stakes summative purposes and did not reflect contemporary theories of assessment. Assessments were predominantly undertaken using written tests or clinical simulation methods. The most common pain medicine education topics assessed were pain pharmacology and the management of cancer and low-back pain. Most studies focussed on assessment of cognitive levels of learning as opposed to more challenging domains of demonstrating skills and attitudes or developing and implementing pain management plans.
CONCLUSION: This review highlights the need for more robust assessment tools that effectively measure the abilities of medical students to integrate pain-related competencies into clinical practice. A Pain Medicine Assessment Framework has been developed to encourage systematic planning of pain medicine assessment at medical schools internationally and to promote continuous multidimensional assessments in a variety of clinical contexts based on well-defined pain medicine competencies.

1. Introduction

Competency-based medical education begins with a careful consideration of competencies required by medical graduates to address health care priorities and provides a mechanism to integrate the health needs of the country with the values of the medical profession.[54,57] Competencies describe the complex set of knowledge, skills, and attitudes a medical student requires upon graduation.[21,72] Articulation of these competencies is guided by the expectations and requirements of the health care system, including professional organizations, medical colleges, government, and, importantly, consumers. Competency-based education focuses on defining learning outcomes that can be measured to determine mastery of a specified competency upon graduation.[54] These learning outcomes influence the core educational activities of teaching, learning, and assessment.[57]

Robust assessment and documentation of student expertise is integral for the effective implementation of competency-based medical education.[74] Assessment in medical education can be defined as the processes used to make judgements about the achievements of students over a course of study.[59] Important changes have been made to methods of assessment of medical students over the last 5 decades.[102] Historically, assessments were designed to determine students' performance through an end of year examination. However, many medical schools now undertake continuous assessments of students' mastery of educational objectives.[10] A number of standardised methods and instruments have been developed to provide accurate, reliable, and timely assessments of medical students' competencies across a comprehensive curriculum.[32,102]

Pain medicine is the medical discipline that aims to prepare students to care for patients living with pain.[8] It deals with the science, prevention, evaluation, treatment, and rehabilitation of persons living with pain from a multimodal perspective. Pain medicine is a developing field of medical practice that recently gained recognition as a medical speciality in a number of countries such as Australia, Canada, New Zealand, United Kingdom, and United States.[36,83,98,116] Acute pain is one of the most common reasons for patients to seek treatment at an emergency department, and severe pain after surgery remains a major problem.[51] Chronic pain affects approximately 20% of the population and is a significant public health problem with important biomedical, psychological, social, and economic implications.[17,27,62,65,78,100] It is critical, therefore, that medical students are equipped with competencies in the field of pain medicine, and that these competencies are adequately assessed, so that the students are prepared for the clinical environment upon graduation. However, pain medicine education is poorly addressed in medical school curricula internationally.[104]

Pain medicine competencies may be defined as the observable abilities of medical students (and practitioners) to integrate knowledge, skills, and attitudes related to pain medicine, into effective clinical practice.[41,43] Traditionally, pain medicine for new medical graduates was conceptualised as a knowledge-based subject rather than a set of learned competencies in which students must demonstrate proficiency.[13] The International Association for the Study of Pain (IASP) is the leading global professional forum for science, practice, and education in the field of pain.[66] Pain medicine curricula for medical undergraduates were first developed by the IASP in 1988,[91] and a number of editions have been published over the intervening years, the most recent in 2018[23,67] (appendix 1, available at http://links.lww.com/PR9/A36). The objective of the IASP Curriculum Outline on Pain for Medicine is to provide the knowledge and skills necessary for new graduates to advance the science and management of pain as part of an interprofessional team.[67] Desired outcomes of pain medicine education emphasize the learner's capacity to successfully and compassionately perform tasks in the real world, such as pain assessment, collaborative approaches to treatment options, and application of pain competencies across the lifespan in the context of various settings, populations, and care-team models.[67] More recently, core competencies for pain management with measurable learning outcomes were developed by an Expert Interprofessional Pain Competencies Consensus Group to provide guidance related to prelicensure health professional education (Fishman et al.)[41] (appendix 2, available at http://links.lww.com/PR9/A36). Internationally, there is poor documentation of pain medicine competency assessment methods for medical students.[19,81,104] Questions about how best to assess these competencies remain unanswered.[13]

Consequently, the aim of this review was to identify and summarise the literature regarding methods for assessing pain medicine competencies in medical students. In particular, the following questions were addressed:

What was the purpose of the assessment? (eg, was it to evaluate the effectiveness of a course or to evaluate students' performance in an examination?)

How were the assessment instruments developed? (eg, were they developed with attention to specific learning objectives/competencies; with reference to the IASP curriculum and/or interprofessional learning?)

What level of clinical competency was assessed? (eg, knows, knows how, shows how, and does[82])

What type of assessment methods was used? (eg, multiple choice questions [MCQs], Objective Structured Clinical Examination [OSCE])

What topics of pain medicine education were assessed?

What attention was paid to the reliability and validity of the assessment tools?

What learning domains were assessed? (eg, knowledge, skills, and/or attitudes)

These data will be used to develop a framework for the rigorous assessment of pain medicine competencies for medical students.

2. Methods

2.1. Literature search

Pubmed, Medline, Embase, ERIC, Google Scholar, and BEME were searched for articles relating to the assessment of medical students' competencies related to pain medicine. The following search terms were used: pain, student, undergraduate, medical, education, knowledge, attitudes, skills, and curriculum. The term “pain” was included in all the searches (appendix 3 for the Medline search strategy, available at http://links.lww.com/PR9/A36). Secondary hand-searching of reference lists and relevant publications known to the researcher identified 3 further studies, which were also included.

2.2. Screening and inclusion/exclusion criteria

Articles were selected by reviewing titles and abstracts to determine whether they might meet the inclusion criteria. Two reviewers independently screened all identified articles after deduplication. The full text of the article was retrieved where relevance was unclear. Consensus was sought through discussion where the decision regarding eligibility of an article was unclear. The article was not included if both reviewers agreed the article was not relevant to the research question.

The main inclusion criteria for this review were as follows:

Medical students (defined as undergraduate or graduate-entry students undertaking a course of study at a medical school leading to the primary qualification of medical practitioner);

Pain medicine (defined as the discipline within the field of medicine that is concerned with the science, prevention, evaluation, treatment, and rehabilitation of persons living with pain);

Empirical studies published between January 1, 1997, and December 31, 2016, describing assessment of any domain of pain medicine competencies (knowledge, attitudes, or skills) in medical students.

Exclusion criteria included:

Studies that were not focussed primarily on assessment of knowledge, attitudes, or skills related to pain medicine;

Articles that focussed solely on palliative care education without referencing pain medicine for medical students;

Studies that investigated knowledge, attitudes, and skills in pain medicine amongst groups other than medical students;

Continuing medical education and medical courses offered to practitioners who have graduated from their medical degree; and

Editorials and review articles.

2.3. Data extraction

All articles selected were analysed according to the following headings: Details of study (name of first author; date of publication; university/institution; country of study; study population; and course); study purpose; assessment instrument development (eg, relationship to education course content, published literature, and IASP curriculum); level of assessment; method of assessment; pain medicine topics assessed; psychometric properties of the instrument (eg, validity and reliability); and domains of learning measured.

3. Results

Eight hundred thirty-four citations were retrieved from the initial database search after removal of duplicates. A total of 41 published studies described 53 assessment instruments that were used to examine pain medicine competencies of medical students. The details of the selection process are shown in Figure 1.

Figure 1.

Flow diagram of the search and selection process.

3.1. Characteristics of studies

Studies included 7599 medical students across the different years of study, from first to final year. The number of participants in a study ranged from 12[46] to 1181,[63] with a median of 114 participants.

Just over half of the studies (51%) were published before 2010 and 98% were in English. One study was published in German. Twenty-two studies (54%) were from North America (United States 44%[9,24,25,34,55,60,71,80,85,92,99,107-110,127,128,132]; Canada 10%[63,90,114,125]); 12 (29%) from Europe[3,6,12,46,56,69,73,84,87,94,95,101]; 3 (7%) from Australia;[18,118,120] and 1 each from Saudi Arabia,[68] Philippines,[76] Thailand,[88] and Taiwan.[50]

3.2. Purpose of pain medicine assessments

The purpose of the assessments as described in the articles reviewed is outlined in Table 1. More than half of the studies (56%) assessed medical students who were exposed to a specific pain medicine module within the medical curriculum, such as a comprehensive pain medicine course (20%),[12,34,46,63,85,110,125,132] cancer pain module (15%),[50,71,76,88,92,108] paediatric pain course (5%),[9,24] acute pain module (5%),[55,99] or low-back pain module (5%).[56,127] Four studies (10%) described an instrument that was used for assessment of interprofessional pain education.[34,63,99,125]

Table 1

Purpose of assessment.

3.3. Resources used to develop assessment instruments

Information on the resources used to design assessment tools were provided in 28 (68%) studies (Table 2). Most studies based test content on literature review (34%) or consultation with faculty experts (29%) rather than on defined learning objectives or competencies.

Table 2

Resources used to develop assessment instruments.

Four studies referenced published schemes that describe educational competencies in more detail.[9,34,90,132] Only one study cited the core competencies for pain management for prelicensure health professionals published in 2013 by Fishman et al.[41] Two other schemes were referenced: namely, the Accreditation Council for Graduate Medical Education[1] core competencies and the Educating Future Physicians in Palliative and End-of-Life Care[2] core competencies both of which were adapted for use in a pain medicine setting.

3.4. Assessment of levels of clinical competence

Using Miller's pyramid of clinical competence, assessment methods can be classified in terms of what level of competence they require of the student such as knows, knows how, shows how, and does.[82] Most studies (80%) in this review assessed “knows”—straight factual recall of knowledge using MCQs and short answer tests, and “knows how”—the application of knowledge to problem-solving and decision-making using written assignments, oral questioning, and development of management plans. Eight (20%) studies focused on “shows how,”[55,56,73,101,108-110,127] examining integration of learning skills into a successful performance with a standardised patient (SP) or simulated experience. No studies examined students' actual performance in a real-life clinical setting on the wards or in the consulting room.

3.5. Standard setting

Standard setting is the process adopted to define levels of proficiency, typically but not limited to agreement on a defensible pass score.[86] Although standard setting is generally accepted as good practice in medical education, only 5 studies (12%) gave details of a defined standard of competence (eg, pass/fail) to evaluate the students' knowledge, attitude, or clinical skills.[63,76,85,110,127]

3.6. Methods used to assess pain competencies

Table 3 provides information on the methods used in the studies to assess pain medicine competencies. Thirteen studies (32%) used more than one instrument or method of assessment, with 11 studies using 2 instruments[9,56,60,63,69,71,95,99,101,114,128] and 2 studies using 3 instruments.[18,85]

Table 3

Methods used to assess pain medicine competencies.

Nineteen studies used existing instruments or modified existing instruments for assessment of medical students' knowledge, attitudes, or skills. The Pain Knowledge and Beliefs Questionnaire,[63,125] the Cancer Pain Objective Structured Clinical Examination,[108,109] the Back Pain Beliefs,[18,69,112] Health Care Providers' Pain and Impairment Relationship Scale,[18,84,96] the Cancer Pain Questionnaire,[50,68,129] and Paediatric Pain Knowledge Questionnaire[9,24] were each used in 2 studies. Most of these existing instruments were developed before 2005 (n = 17 89%). New instruments were designed in 78% (n = 32) of studies.[6,9,12,24,25,34,46,55,56,60,63,68,71,73,80,85,87,88,90,92,94,95,99,101,109,110,114,118,120,125,127,132]

3.6.1. Written examination

Written examinations were used in 31 studies to assess students' pain medicine competencies. The number of questions per test ranged from 1 to 59, with a mode of 13. Three studies did not report the number of questions in the assessment tool.

3.6.1.1. Multiple choice questions and Likert rating scales

Multiple choice questions were used to assess knowledge in 17/18 studies (94%). Single best answer format of the MCQ was used in 11 studies[6,9,12,24,68,85,88,90,94,95,101] and 10 studies used the true/false format.[3,12,50,63,68,76,90,118,125,132] Likert rating scales were used to measure pain-related knowledge and attitudes, and the number of response options in the rating scales ranged from 3 to 7. There was limited use of open-ended short answer questions and structured essays.

3.6.1.2. Group assignments

A small-group assignment, which involved completion of a comprehensive pain management care plan, was evaluated as an indication of transfer of learning to the real-life patient context.[63] One study used a paired-work assignment to assess clinical applications of cancer pain.[85]

3.6.1.3. Images, vignettes, or written description of a clinical scenario

Still facial images (1 study), vignettes (3 studies), and written description of a clinical scenario (2 studies) were used to examine medical students' treatment recommendations and attitudes towards patients with pain,[18,60,94,101,120] in particular, the influence of contextual variables on clinical recommendations. A “key features” problem was used in one study, which involved the use of brief clinical scenarios of patients in pain, to evaluate the students' pain diagnostic and management skills.[101] In these 5 studies, students recorded answers using a 0 to 100 visual analogue scale, by choosing answers from a list of possible choices or using a Likert scale. The number of clinical scenarios ranged from 1 to 16 with a median of 8.

3.6.1.4. Reflective journals

Reflective journals were examined in one study to explore students' attitudes to working with patients with chronic pain.[25] First-year students described their experiences during a 4-week clinical immersion rotation. Entries were considered as students' perceptions of pain encounters if they included expressions of thoughts and feelings engendered by pain-related clinical experiences.[25] The journal entries were coded and recoded using an iterative process.[25]

3.6.2. Clinical simulation

3.6.2.1. Objective Structured Clinical Examination

The OSCE was used in 8 studies,[55,56,73,101,108-110,127] primarily to assess clinical skills (88%), but also to assess knowledge (37%)[55,108,109] and attitude to pain (13%).[101] One OSCE focussed primarily on regional anaesthesia techniques,[55] 2 studies assessed clinical competence in the area of cancer pain management,[108,109] and 2 studies assessed clinical skills in examining chronic low-back pain.[56,127] The number of OSCE stations ranged from 1 to 10, with a median of 3. One study included a global rating scale for clinical and communication skill checklists.[73]

3.6.2.2. Simulation-based group assessment

In one study, medical-nursing student dyads were challenged to assess and manage acute pain in a simulated hospitalized patient using effective collaborative interprofessional skills.[99] The students completed presurvey and postsurvey measuring confidence in their ability to assess and manage acute pain.

3.6.3. Assessment by supervising clinicians

3.6.3.1. Focus groups

Focus groups were used in 2 studies to explore the knowledge and attitudes of students to pain management, retention of material presented in the course, and to identify gaps in knowledge with respect to pain education.[71,114] Students were asked a set of questions on specific topics such as the meaning of pain (opiophobia and subjectivity), assessment of pain, clinical management of pain (particularly with regard to opioids), communication and the therapeutic relationship, ethical considerations, and the institutional context influencing the students' attitudes to providing pain management. Both studies used a semistructured format.

3.6.3.2. Interprofessional workshops

Medical and nursing students collaborated to develop a best-practice pain management plan for 2 case studies.[34] A scoring checklist for the interprofessional pain management plans was used to assess and compare learning outcomes between student groups.

3.6.3.3. Group-based oral assessment

One study included an oral examination to assess students' learning.[95] The examination took the format of a small-group discussion during which clinical cases were presented and teachers evaluated students' knowledge.

3.6.4. Multisource assessment

3.6.4.1. Portfolios

Written portfolios were used to assess students' pain attitudes in one study.[85] The portfolios involved a compilation of short exercises including writing a brief pain narrative, describing the pain depicted in a fine-art image, assessing personal responses to the experience of pain during a common psychophysical test (the 30-second cold pressor test), drafting personal learning objectives for the pain course, defining the role of empathy and compassion in medicine, a description of the relationship between pain and addiction, and a reflection of lessons learnt from a pain expert panel and positive personal qualities exhibited by the pain expert clinicians.[85] The portfolios were graded by one examiner using a 4-point system (fair, good, very good, and excellent).

3.6.5. Self-assessment

Six studies described medical students' self-assessment of pain knowledge competencies and confidence in their clinical capabilities.[9,46,56,76,92,99] Students rated their competencies with an agreement or disagreement response option towards a given statement, using a 4- or 5-item Likert response format.

3.7. Pain medicine topics included in assessment tools

There was considerable variation in the attention given to different topics (Fig. 2) (appendix 1, available at http://links.lww.com/PR9/A36). The IASP core curriculum for medical students was used to benchmark the pain content topics that were evident in the reviewed studies.[67]

Figure 2.

Topics of items included in the assessment tool.

Clinical pharmacology was the most commonly assessed topic (n = 31; 76%), which included opioids (n = 22; 71%), adjuvant analgesics (n = 4; 13%), local anaesthetics (n = 2; 7%), and nonsteroidal anti-inflammatory drugs (n = 2; 7%). Some studies did not specify details of pharmacology topic assessed.

3.8. Validity in assessment

Details of the psychometric properties of the assessment tools varied. Validity was discussed to some degree in 16 studies (39%),[6,18,50,60,63,68,69,71,84,99,108-110,118,125,127] with content validity being most frequently reported (15%).[6,50,63,76,118,125] Validity of OSCE assessments were reported as being improved by strategies such as the incorporation of SPs,[55,56,73,101,108-110,127] use of a checklist,[55,56,73,101,108-110,127] inclusion of 3 or more stations,[73,108-110,127] and blinded observers.[55,101,127] Fourteen studies (34%) discussed the reliability of assessment instruments. Information on the psychometric properties of instruments used was not provided in 13 studies (32%).[9,12,24,46,85,88,90,92,94,95,120,128,132]

3.9. Pain medicine learning domains assessed

Studies assessed the following pain medicine domains: knowledge (n = 24; 59%), attitudes of medical students towards patients in pain and to the discipline of pain medicine (n = 23; 56%), and clinical skills necessary for the practice of pain medicine (n = 14; 34%) (Table 4). Most assessment methods evaluated core knowledge, attitudes, and clinical skills in isolation. Comprehensive evaluation of learning encompassing all 3 dimensions of knowledge, attitudes, and skills was described in 3 studies (7%).

Table 4

Pain medicine learning domains assessed.

4. Discussion

This review provides an overview of methods used to assess medical students' competencies in pain medicine internationally. Studies were predominantly focussed on assessment of pain medicine knowledge and attitudes of North American medical students before 2010. Most studies described summative assessments using written tests. Only one-third of studies used assessment tools based on internationally recognised pain medicine curricula/learning objectives (such as the IASP Curriculum Outline on Pain for Medicine)[67] or core competencies for pain management (Fishman et al.).[41] The most common pain medicine education topics assessed were pain pharmacology (in particular, opioid prescription) and the management of cancer and low-back pain. Validity of the assessment tools was poorly addressed in general. Most studies focussed on assessment of the cognitive level of learning (such as knowledge and understanding of pain medicine) as opposed to the more challenging domains of demonstrating skills and attitudes, and developing and implementing pain medicine management plans.

Worldwide, entry-level medical education programmes are tertiary courses undertaken at a medical school.[130] The duration of these courses varies between 4 and 6 years.[130] In many countries, newly graduated doctors are specifically required to undertake a period of clinical practice (internship, foundation programme, or housemanship) lasting 1 to 2 years before full registration or unrestricted practice.[130] During this bridging period after medical school, new graduates are generally required to practice the key skills and knowledge learnt during their medical education, so as to gain proficiency in the basic clinical skills that will prepare them for the context in which they will be expected to work.[48,131] Pain and inadequate pain management remain a significant problem in hospitals.[35,51,106,111,122] New graduates are likely to encounter patients experiencing acute, chronic noncancer and cancer pain in hospital emergency, surgical, and medical departments.

A recent review of pain medicine education internationally has revealed that, in general, comprehensive pain medicine content is not mandatory in the medical curriculum.[103] Pain medicine is taught in a fragmented way within modules in specific areas of medicine (such as anaesthesia) as opposed to a dedicated pain module.[103] Minimal learning time is allocated to the topic of pain medicine.[103] In Germany, education on pain became compulsory in 2012 within federally defined medical school curricula.[19] The Pain Management Core Curriculum for German Medical Schools has been developed to provide the content for a defined set of teaching sessions in interdisciplinary pain management for medical students.[67] This curriculum formed the basis of the recently updated Pain Management Code Curriculum for European Medical Schools.[67] The IASP Curriculum Outline on Pain for Medicine is based on these curricula.[67]

A paucity of research into the assessment of pain medicine competencies indicates a general lack of emphasis on pain medicine assessment internationally.[19,81,104] Assessment provides a reflection of institutional values.[33] Explicit value can be judged by the amount of time and resources that are allocated to the teaching of a topic and the weighting of the topic in the assessment process. The absence of attention to assessment of pain medicine competencies implies a prevailing lack of value in the topic by teaching institutions.[32] The current approach to medicine is based largely on the disease model.[105] Historically pain was regarded as a passive symptom of a disease process.[105] However, there is accumulating evidence that pain is a disease entity with its own pathology, symptoms, and signs.[105] In addition, most medical schools focus on a biomedical model and emphasize knowledge-based learning.[75] Although neurobiological and pathology of pain mechanisms are important aspects of the pain curriculum, the psychosocial dimensions of pain development, maintenance, and management are critical to pain medicine.[75] This broader conception of pain medicine sets it apart from other fields of medicine such as genetics, radiology, and surgery.

4.1. Purpose of assessment

This review demonstrated that assessments of medical students' pain medicine competencies were undertaken for a limited number of purposes. Most studies were performed from the perspective of assessing efficacy of a pain course or identifying gaps in the medical curriculum. There are a number of diverse personal, institutional, and societal reasons why assessment of pain medicine competencies of medical students should be an integral part of a competency-based medical curriculum (Box 1).[33] The goal of the assessment will determine the approach to the assessment process, that is, summative assessments of learning and/or formative assessments for learning. These assessment goals are not necessarily discrete or opposed to one another, as in some instances, they may be interdependent and complimentary.[121]

For the student

Facilitates a desire to learn and develop pain medicine skills

Focuses the learner's attention on what is considered core pain knowledge and competencies

Provides feedback about performance in pain medicine assessments to identify strengths and weaknesses

Encourages students to reflect on their own performance and attitudes to pain medicine

For the teacher

Demonstrates achievement of pain competencies

Identifies educational gaps in the curriculum with respect to pain medicine

Promotes curriculum development

Recognises standards of competence for students at different levels

For the institution

Ensures that the medical workforce is able to practice pain medicine in an interprofessional model of care

Meets the demands of the accrediting and professional bodies with respect to defined graduate outcomes

Reflects the institutional value placed on pain medicine by determining how pain medicine is assessed

Recognises the “hidden” curriculum

Justifies progression within the medical curriculum

For the health system

Determines which practitioners are competent and safe to enter the workforce

Optimises careful use of limited resources

Promotes best-practice pain care for the patients

For the public

Provides accountability in terms of medical professionals' abilities to integrate pain-related knowledge, skills, values, and attitudes into clinical practice

4.1.1. Assessment of learning

Most studies in this review described summative assessments to pilot a new pain course, to improve an existing pain module, or to formulate pain education resources for students. There was a significant lack of high-stakes assessments to determine progression through a course (7%) or achievement of a set standard of competence (12%). The public expects medical schools to certify the competence of future practitioners to meet the health care needs of people experiencing pain.[40] There have been calls for professional bodies to increase the number of required entry-to-practice pain competencies to improve the capacity for medical practitioners to alleviate the suffering of unrelieved pain.[126] More direction is needed to define criteria or standards for these assessments to support high-stake decisions regarding readiness for entry into unsupervised medical practice and to promote best-practice pain care for patients.

The review revealed limited examples of pain medicine competency assessments based on the defined IASP Curriculum objectives (28%) or core competencies for pain management for prelicensure health professionals (Fishman et al.) (2%). A possible reason for this lack of integration may be that these core competencies in pain management have only recently been published in 2013.[41] Second, these core competencies provide general guidance related to pain medicine education for all major health care professionals and are not specific for medical education.[41] Competency-based medical education is explicitly oriented to the desired performance characteristics of medical graduates that should largely be determined by stakeholders outside of the educational program.[44,54] This review has highlighted the need for more defined assessment processes based on learning outcomes as defined by the IASP, which incorporate the interprofessional core competencies for pain management prelicensure health professionals (as described by Fishman et al.).[41]

4.1.2. Assessment for learning

It is generally accepted that assessments that include formative feedback can enhance learning.[123] Although few studies in this review focussed on assessments aimed at driving learning, 3 studies described assessment methods that were used in a formative capacity to promote interprofessional learning.[34,63,99] In these studies, a number of different health professionals worked together to integrate knowledge, skills, and attitudes to develop a pain management plan. The group sessions were facilitated by trained faculty staff during or after the education experience to provide useful opportunities for self-reflection and feedback.

Self-assessment of pain knowledge competencies and confidence in clinical capabilities was described in 15% of studies. Self-assessment and reflection methods are more suited to formative-type assessment, as self-reported abilities may vary from actual abilities and are susceptible to recall and social desirability biases.[52] Two studies described methods (such as portfolio work and journals) that were shown to be useful for fostering habits of self-reflection and self-remediation.[25,85] The portfolio work resulted in increased awareness of the depth of suffering experienced by patients in pain and enhanced a commitment to excellence in pain care. The reflective journals brought to light the fact that 1st-year medical students identify pain as a major concern in their early clinical experience and, in general, have a negative perception of pain medicine. Assessment for learning was shown to be an integral part of a pain medicine curriculum to foster emotional strength (eg, empathy, compassion, and caring), emotional intelligence (eg, awareness of the impact of pain on mood and perception of emotion in others), emotional resilience (eg, capacity for emotional self-repair and ability to tolerate frustration), and emotional regulation (eg, tolerance of difficulties and sense of professional duty).[85]

4.2. Appropriate methods of assessment

The diversity of methods used in this review reflects the range of the assessment purposes and settings. Written exercises, in particular, MCQs were the most common method used to assess medical students' knowledge and attitudes of pain medicine. This was not surprising because the current medical school environment with large student numbers and a pressurised curriculum favours the MCQ-type assessment.[37] Multiple choice questions require less time to administer and mark than open-ended short answer questions and structured essays.[49] Multiple choice questions have been shown to be more reliable in testing knowledge than traditional essay questions.[5]

True/false and single best answer-type questions were predominantly used to assess cognitive knowledge. More recently, medical schools are encouraged not to use true/false question formats because dichotomous choices are rare in medical practice, and it is difficult to frame true/false questions that are unambiguous and sufficiently challenging to discriminate between students.[47] Questionnaires scored using a Likert rating scale were mostly used for assessing attitudes for research or for self-evaluation purposes rather than high-stakes summative assessments.

If well-constructed, MCQs can also be used to assess learning objectives that require higher-order thinking (such as applying knowledge to a clinical situation).[28,37] Validity of MCQs can be increased by sampling the entirety of the domain and including sufficient numbers of items in the assessment instrument (in general, at least 30 items).[28] The use of well-crafted items that address a range of difficulty levels and the careful selection of distractors to reduce the cueing effect MCQ testing will also increase validity of written tests.[7,28] Extended-matching questions, in which the format of the question starts with a vignette and the student is required to make a diagnosis choosing an answer from a list of options, were not used in any studies in this review but have increasingly been used as a medical examination format.[22,30] Because well-constructed MCQs and extended-matching questions are time-consuming to develop, there may be opportunities for agencies and universities to work collaboratively to construct a bank of questions related to pain medicine for medical educators to draw from, such as the Get Through Series.[26,28,37]

Higher-order assessment of medical students' pain competencies, for example, assessment of clinical skills, was undertaken using the OSCE format with simulated patients.[55,56,73,101,108-110,127] The OSCE assessment was used effectively for assessing competencies such as attitude, empathy, and communication as part of a pain assessment in a variety of contexts (acute, low-back, and cancer pain).[55,73,109,110,124,127] The OSCE may currently be the most appropriate summative assessment for educators to use to identify students who may be at risk of poor performance in a clinical workplace environment and is used extensively within clinical medical education.[32,70,79,115] Careful attention needs to be paid to test content, design, and implementation factors such as wide sampling of cases, large numbers of stations and examiners, and good standardisation of patients to improve validity and reliability.[119,124] This is especially relevant when the results will be used for high-stakes decision-making.[119] Checklists have become standard in many OSCE programmes as was evidenced in this review, but global rating scales scored by experts (that show higher interstation reliability and better construct validity than do checklists) were only used in one study.[119,124] Theoretical difficulties associated with OSCE-type assessments were discussed in 4 studies, such as the time taken to design an OSCE and deliver education for the actors, as well as the costs associated with actors. Three studies used volunteer patients, and one study used students as actors to avoid these costs, and it was felt that the positive effects of the intervention on patients and students validated the time spent by clinical staff. Debriefing is an essential component of simulation that allows the student to understand their decision-making processes and was discussed in 2 of the studies in this review.[53] Examples of high-fidelity simulation experiences (involving SPs or computer-programmed mannequins) to broaden the assessment of pain-related core competencies for health professional students have been described elsewhere.[13]

Although the use of portfolios and multisource (360°) pain medicine assessments was poorly documented in this review, these methods have the potential for providing both formative and summative feedback on pain medicine competence.[61] Multidimensional pain portfolios were described in one study to engage the students in active, structured reflection about pain-related experiences.[85] Portfolios can be extended to encompass reporting on work done, progress made as well as goals and plans on how to further improve competence.[29,89] Their format and content may differ considerably.[29,89] Although these assessment methods can be time-consuming to prepare and review, they are most effective when goals and procedures are clearly communicated; there is a flexible structure underpinning the development of the portfolio; support is provided through mentoring, and there is robust integration of the assessment into the curriculum.[29] Portfolios should ideally be judged by multiple assessors who should be encouraged to substantiate their judgments.[89] Multisource (360°) assessments involving assessment by patients, peers, and other members of the clinical team have been shown to provide valuable insight into students' work habits and capacity for teamwork.[4,31,32,38,58,117]

Validity focusses on “whether a test actually succeeds in testing the competencies that it is designed to test.”[124] Ideally, assessment of clinical competency should provide some measure of actual performance in a real-world environment and capacity to adapt to different environments.[32] There were no published studies of direct observation of students' encounters with patients by supervising clinicians to assess the impact of pain medicine education on students' clinical performance. Demonstration of pain competencies in “real-world” settings favours formative assessment opportunities. These assessments require flexibility, as not all patients will provide adequate teaching/learning opportunities, and quality supervision is necessary to reduce patient risks.[13] Refining of learning can take place during feedback to structure further learning.[13] Creating valid and reliable assessment tools to predict real-world clinical performance of pain competencies is challenging, and further work is needed in this area to assist educators with this important task.[32,54,124]

This review has described multiple methods of pain medicine assessment in a variety of settings and contexts. It is clear from this review that there was no one gold standard for the assessment of the broad range of pain medicine competencies. Assessment methods had unique strengths and flaws. The use of multiple approaches over time can compensate for deficits in any one method.[32] Three studies stood out as examples of multifaceted evaluation of the effectiveness of pain education modules by assessing the 3 main domains of learning, that is, knowledge, attitudes, and skills.[9,63,85] Further work is needed to encourage medical schools to implement similar multidimensional assessments and adapt the tools to their unique local situations.

4.3. Implications for practice and future research

Only 5 studies included examination of students' empathy towards patients with pain.[46,73,85,87,95] This review suggests that little attention is being paid to documented assessment of students' ability to display empathy, considering that the development of compassionate and empathetic practices has been considered a critically important aspect of pain management.[14]

Different approaches to assessment of interprofessional skills were explored in this review, such as medical student/nursing student dyads in a clinical simulation experience, case-based workshops for medical and nursing students, and group assignments (medical, nursing, dentistry, pharmacy, physical therapy, and occupational therapy students).[34,63,99] It is widely accepted that to maximize optimal pain management, health professionals need to work together in interprofessional teams.[15,42,45,64,97] Continued work is needed to develop and refine formal assessment of interprofessional pain knowledge, attitudes, and clinical skills to enhance the seriousness and importance of this aspect of pain medicine education.[20,81]

This review has shown that there is a lack of attention being paid to exploring assessment of topics such as psychological and physical therapy approaches to pain management, as well as types of pain such as headache, visceral pain, and chronic primary pain. The needs of special populations such as children, the elderly, developmentally challenged, and patients with opioid tolerance were also neglected. More than half of the studies in this review were focussed on opioid therapy rather than on evidence-based advances in pharmacological management of pain, such as the use of tricyclic antidepressants, selective noradrenaline reuptake inhibitors, antiepileptics, topical lidocaine, and capsaicin.[39] These are topics listed in the IASP Curriculum Outline on Pain for Medicine, which should be covered if medical graduates are to be adequately prepared for the management of pain.[67] Emerging concepts in pain theory, such as central sensitisation, neuroplasticity, allodynia, cortical modulation, nociception being neither sufficient nor necessary to evoke pain, and the concept of pain as a disease entity were seldom included in the assessment of pain medicine competency.[3,113]

Assessment tools need to be kept up to date as scientific knowledge grows. A clear example of assessment tools becoming outdated was demonstrated in the area of opioid prescription for pain. Earlier studies (1998–2008) focussed on opiophobia, or the reluctance of medical students to prescribe opioids for patients in pain.[50,76,80,107,128] The current teaching at that time was that patients had the right to pain relief, and that the risks of opioid prescription in terms of dependency and overdose were low.[93,128] Pain experts are now warning against the liberal use of opioids especially for chronic non-cancer pain (CNCP).[77] Medical schools have recently incorporated core competencies into their curricula such as safe prescribing for acute short-term opioid use and the management of long-term opioid prescriptions for the treatment of CNCP and have committed to assessing students' competence in these areas.[11]

4.3.1. How should pain medicine competencies be assessed?

Current assessment practices can be enhanced by the recognition and application of the principles summarised in Box 2.[32,33,123,124] Ideally, assessment of pain medicine competencies requires a purposely built combination of individual instruments, each being optimally designed in terms of assessment for learning and assessment of learning.[102]

Be clear on the purpose of assessment

Encourage self-reflection

Provide timely feedback and mentoring

Ensure continuous and frequent assessments (formative and summative)

Incorporate multiple methods of assessments in a variety of contexts

Define the endpoint of assessment and set appropriate standards

Plan assessments using a developmental perspective in terms of levels of learning

Ensure that assessments are grounded on evidence-based content

Design authentic work-based assessments

4.3.2. Framework for assessments

This review has highlighted a lack of comprehensive pain medicine assessment models. In part, this reflects the low priority given by medical schools to the delivery of pain medicine education internationally. In addition, detailed pain medicine competencies specifically designed for medical students have yet to be developed. The resources available at present are the pain management core competencies for prelicensure health professionals (Fishman et al.),[41] which are designed to be used in conjunction with the IASP curriculum outline on pain for medical students.

Pain medicine is a complex and multifaceted discipline that is mostly taught as a topic integrated throughout the entire medical curriculum. An assessment framework would, therefore, be useful to provide a structured conceptual map of essential content components and core competencies for assessment in the pain medicine curriculum as a whole. Hence, a Pain Medicine Assessment Framework (Table 5) has been developed to provide a systematic and comprehensive approach to designing pain medicine assessments for medical students.[33,67] This framework incorporates the assessment of pain management core competencies recommended for prelicensure health professionals as described by Fishman et al.[41] and the IASP curriculum (appendix 1 and 2, available at http://links.lww.com/PR9/A36). The framework can also act as a point of reference for the various members of the pain medicine education community and can be used as a blueprint for universities wishing to incorporate pain medicine into the curriculum. It allows for planning and recording of continuous and frequent assessments, and the sequencing of assessments in terms of level of learning. It ensures that there is consistent assessment across the spectrum of clinical contexts and reduces bias in assessment practice by combining the various methods of assessments. The Pain Medicine Assessment Framework is a useful record of assessment for the student because it encourages self-reflection, but it is also a valuable resource for supporting critical reflection and evaluation of the pain medicine curriculum by medical educators.

Table 5

The pain medicine assessment framework.

4.4. Limitations

Pain medicine is embedded into the discipline structure of medicine. It is therefore likely that pain medicine assessment occurs within medical schools but remains unpublished and as a result was not included in this review. Second, although many studies described individual assessment methods, quality data that evaluated and compared assessment methods were lacking.

5. Conclusions

This review highlights the diversity of instruments used to assess the outcomes of pain medicine education and competencies in medical students. There is no one “gold standard” method for assessing knowledge, attitudes, and skills of pain medicine. In general, the studies reviewed did not reflect contemporary theories of educational assessment.[16] Most assessments were performed for low-stakes summative purposes and were not constructed using specific pain competencies or learning objectives. Assessments were predominantly undertaken using written tests or clinical simulation methods. Important topics that are intrinsic in the pain medicine curriculum were neglected, and current evidence-based pain medicine knowledge was not reflected in many assessments. Some medical schools demonstrated innovative assessment methods (such as reflective journals, vignettes, and portfolios) and multifaceted assessment processes to measure multiple domains of competence in clinical pain medicine practice. Future research could prioritise the development of robust assessment tools that can effectively measure the abilities of medical students to integrate pain-related knowledge, skills, and attitudes into clinical practice.

Assessment plays an integral role in competency-based education and should evolve in concert with delivery of the educational objectives. A framework of assessment has been developed to encourage systematic planning of pain medicine assessment at medical schools internationally to validate the objectives of the curriculum and to provide effective feedback to the students. Importantly, this framework will promote fundamental aspects of competency-based pain medicine assessment such as continuous multidimensional assessments in a variety of clinical contexts based on well-defined pain medicine competencies.

Disclosures

The authors declare that they have no competing interests.