Scope and Nature of Pain- and Analgesia-Related Content of the United States Medical Licensing Examination (USMLE).

Background: "The ongoing opioid crisis lies at the intersection of two substantial public health challenges-reducing the burden of suffering from pain and containing the rising toll of the harms that can result from the use of opioid medications" [1]. Improved pain education for health care providers is an essential component of the multidimensional response to both still-unmet challenges [2,3]. Despite the importance of licensing examinations in assuring competency in health care providers, there has been no prior appraisal of pain and related content within the United States Medical Licensing Examination (USMLE).
Methods: An expert panel developed a novel methodology for characterizing USMLE questions based on pain core competencies and topical and public health relevance.
Results: Under secure conditions, raters used this methodology to score 1,506 questions, with 28.7% (432) identified as including the word "pain." Of these, 232 questions (15.4% of the 1,506 USMLE questions reviewed) were assessed as being fully or partially related to pain, rather than just mentioning pain but not testing knowledge of its mechanisms and their implications for treatment. The large majority of questions related to pain (88%) focused on assessment rather than safe and effective pain management, or the context of pain. Conclusions: This emphasis on assessment misses other important aspects of safe and effective pain management, including those specific to opioid safety. Our findings inform ways to improve the long-term education of our medical and other graduates, thereby improving the health care of the populations they serve.

Background

Pain is pervasive among those seeking health care, but it is often inappropriately addressed by clinicians [1,2]. Inadequate education about pain, particularly as regards the safe and effective use of opioids for chronic noncancer pain, has been identified as contributing to the ongoing epidemic of opioid misuse and abuse [3-5]. The need for improved pain education for all health care providers, to address the dual crises of undertreated pain and inappropriate opioid use, has been raised in many recent governmental [6] and nongovernmental [7,8] white papers.

Historically, pain has been an orphan subject within crowded health education curricula [6,7,9,10]. The 2016 National Pain Strategy (NPS) from the US Department of Health and Human Services states, “Improvements are needed in discipline-specific core competencies, including basic knowledge, assessment, effective team-based care, empathy, and cultural competency. Educational program accreditation bodies and professional licensure boards can require pain teaching and clinician learning at the undergraduate and graduate levels” [6].

The 2016 Centers for Disease Control and Prevention (CDC) Guideline for Prescribing Opioids for Chronic Pain [3] included numerous recommendations for prescribers to take greater responsibility for assessing, monitoring, and mitigating risks of opioid misuse and abuse when prescribing opioids, and providing patients with ongoing education. However, a national survey of 2,626 US medical residents completing training in specialties where pain management is an essential component reported that approximately 50% felt only “somewhat prepared” to counsel patients about pain management and approximately 25% felt “somewhat unprepared” or “very unprepared” [11]. Other more recent studies have reflected these findings [12,13]. Data indicating a similar lack of perceived preparedness among fourth year medical students, particularly as regards chronic pain, were recently replicated in one New England medical school [14].

A 2017 National Academy of Medicine report on the need to balance legitimate pain control with containment of opioid abuse asserted that “any meaningful effort to improve pain management will require a fundamental shift in the nation’s approach to mandating pain-related education for all health professionals who provide care to individuals with pain” [15,16]. One of many avenues toward improving pain management education is to mandate that competency in pain management is a criterion for accreditation of health professional schools and licensing of graduates [7,9]. For example, mandatory core competencies for prevention and management of prescription drug abuse, including pain assessment and treatment using nonopioid and nondrug modalities, were recently developed through a collaboration of the four Massachusetts medical schools, led by its Department of Public Health [17].

The present study evaluates the scope and nature of pain content within the United States Medical Licensing Examination (USMLE), a sequence of tests that must be successfully completed by allopathic physicians to obtain an unrestricted state medical license in the United States [18]. The USMLE consists of three separate examinations that test candidates on basic science and clinical knowledge, as well as the clinical competencies necessary for the unsupervised general practice of medicine [18]. In cooperation with the National Board of Medical Examiners (NBME) and the Federation of State Medical Boards (FSMB), a panel of leaders in pain management from academic pain management programs and professional organizations evaluated USMLE test questions for the quantity, scope, and nature of content related to pain competencies. To ensure that our assessment of pain and pain-related content was comprehensive, we benchmarked our findings using distinct, albeit overlapping, sets of curricular content previously developed and published by pain educators [9,19].

Methods

To our knowledge, other than a single 1997 article comparing Parts I and II of the 1986 vs 1993 USMLE [20], there has been no previously published empirical methodology. Therefore, we developed novel assessment criteria to examine the quantity, scope, and nature of pain-related questions within the USMLE. This structured review involved an on-site, secure review of questions from Step 1, Step 2, and Step 3 examination forms.

Panel Composition

Experts in pain management were selected by the Pain Core Competency Executive Committee at the University of California, Davis (http://www.ucdmc.ucdavis.edu/advancingpainrelief/Projects/Core_Competencies.html), for participation in this review. Criteria included leadership roles within academic pain management programs, expertise in the pain field, as evidenced by peer-reviewed publications, and/or holding a leadership position within stakeholder professional organizations. Twelve individuals accepted the invitation to participate in this detailed inventory (Supplementary Data).

Question Review Methodology

Panelists met in teleconferences from May to September 2014 to discuss the project, guide development, and finalize the review methodology. Project leaders at UC Davis (SMF, HMY, JMM) incorporated recommendations and revised and updated the methodology under the guidance of the UC Davis Pain Core Competency Executive Committee.

Panelists utilized the developed data collection instrument to categorize and rate USMLE test questions (Supplementary Data). First, questions including the word “pain” were identified for further analysis. Each question that included the word “pain” was assessed for how closely the question focused upon pain (fully, partially, or not at all). Questions that were deemed fully or partially related to pain were further evaluated for quality, association with published pain domains and competencies (Table 1), and other content areas (Table 2). Questions were examined for pain content and inclusion in one or more of four broad areas: 1) overarching domains related to general pain competency (Table 1) [19]; 2) specific pain core competencies (Table 1) [19]; 3) major topics in pain (Table 2; Supplementary Data) [9]; and 4) key pain-related public health issues (Table 2). The first two areas were previously developed through an interprofessional consensus summit for prelicensure clinical education for all health professionals [19]. The third was based on pain topics from published consensus recommendations [9], and the fourth included topics of compelling interest prepared by the expert reviewers.

Table 1

Pain domains and core competencies

Domain 1: Multidimensional Nature of Pain: What Is Pain?

1.1. Explain the complex, multidimensional, and individual-specific nature of pain.

1.2. Present theories and science for understanding pain.

1.3. Define terminology for describing pain and associated conditions.

1.4. Describe the impact of pain on society.

1.5. Explain how cultural, institutional, societal, and regulatory influences affect assessment and management of pain.

Domain 2: Pain Assessment and Measurement: How Is Pain Recognized?

2.1. Use valid and reliable tools for measuring pain and associated symptoms to assess and reassess related outcomes as appropriate for the clinical context and population.

2.2. Describe patient, provider, and system factors that can facilitate or interfere with effective pain assessment and management.

2.3. Assess patient preferences and values to determine pain-related goals and priorities.

2.4. Demonstrate empathic and compassionate communication during pain assessment.

Domain 3: Management of Pain: How Is Pain Relieved?

3.1. Demonstrate the inclusion of patients and others, as appropriate, in the education and shared decision-making process for pain care.

3.2. Identify pain treatment options that can be accessed in a comprehensive pain management plan.

3.3. Explain how health promotion and self-management strategies are important to the management of pain.

3.4. Develop a pain treatment plan based on benefits and risks of available treatments.

3.5. Monitor effects of pain management approaches to adjust the plan of care as needed.

3.6. Differentiate physical dependence, substance use disorder, misuse, tolerance, addiction, and nonadherence.

3.7. Develop a treatment plan that takes into account the differences between acute pain, acute-on-chronic pain, chronic/persistent pain, and pain at the end of life.

Domain 4: Clinical Conditions: How Does Context Influence Pain Management?

4.1. Describe the unique pain assessment and management needs of special populations.

4.2. Explain how to assess and manage pain across settings and transitions of care.

4.3. Describe the role, scope of practice, and contribution of the different professions within a pain management care team.

4.4. Implement an individualized pain management plan that integrates the perspectives of patients, their social support systems, and health care providers in the context of available resources.

4.5. Describe the role of the clinician as advocates in assisting patients to meet treatment goals.

Depicted are domains and specific pain-related competencies falling within the four major domains previously developed by an interprofessional consensus summit [19].

Table 2

Major topics in pain and key public health issues (Supplementary Data)

Major Pain Topics	Key Public Health Issues
1. Human and social costs of pain	1. Disparities
2. Basic science of pain	2. Infants
3. Clinical assessment of pain	3. Adolescents
4. Pharmacological pain management	4. Childbirth
5. Nonpharmacological pain management	5. Older adults
6. Acute pain	6. End of life
7. Chronic pain, including types and forms of pain	7. Prescription safety, abuse, addiction, and misuse
8. Pediatric pain	8. Mental illness
9. Geriatric pain	9. Chronic disease comorbid with chronic pain
10. Cancer pain/palliative care	10. Military-related pain
11. Pain ontology (meaning culture/ethnicity)	11. Disability
12. Interventional approaches to pain care	12. Cancer treatment and cancer survival
13. Medicolegal	13. Chronic pain after surgery
14. Visceral, pelvic, abdominal pain	14. Patient-reported outcomes
15. Gynecological and obstetric pain	15. Other

Depicted are major content areas including major topics recommended in a comprehensive pain-related curriculum developed by the Committee on Education, American Academy of Pain Medicine [21], and key public health issues, as recommended by the expert reviewers from this study.

Onsite Review

The expert review occurred at the NBME headquarters in Philadelphia, Pennsylvania, on November 3 and 4, 2014, with the 12 panelists. Panelists were apprised of and complied with all NBME security policies. Three review sessions were conducted for three hours each. Panelists were randomly paired and rotated at the end of each session, so that each reviewer worked with three different partners.

USMLE staff provided the panel with four randomly selected USMLE examination forms for each of the three steps, reflecting basic to advanced medical knowledge. Each form contained more questions to review than the allotted time allowed. Reviewers were instructed to accurately review and complete as many questions as possible in the given time frame, starting with the first question and without skipping any. No question was reviewed more than once. Contents from the Step 2 Clinical Skills exam and the computer-based case simulations (CCS) were not included in the scope of this review. Reviews were entered into Excel spreadsheets for analysis.

Data Analysis

NBME staff provided the panelists’ ratings to the project leaders (SMF, HMY, JMM, LB, JW) in 12 spreadsheets (four from each of the three steps). For data analysis, the 12 spreadsheets were merged into a single comprehensive file structured with one question per row and one column per rating variable.

Results

Expert pairs reviewed 1,506 questions (40.4%) from a total of 3,728 questions included on the examination forms. Analysis began with frequencies of categories of each rating variable. Questions classified as fully or partially focused on pain are reported as “related to pain.” Questions classified as “not related to pain but includes pain-related terms or content” were not analyzed further because pain knowledge was not tested as part of such questions. Figure 1 illustrates how the USMLE questions were screened and further reviewed. Total percentages for domains, competencies, major topics, or key public health issues could be greater than 100% because a single question could have multiple assignments across categories.

Figure 1

Flow diagram of USMLE question review process. Depicts the reviewed questions from all provided, initially reviewed, and those that were further reviewed for full or partial pain content.

Of the 1,506 questions scored by raters, 28.7% (432) were identified as including the word “pain” and 71.3% (1,074) were rated as not including pain. The vast majority of fully reviewed questions were found to be of high quality; however, for proprietary reasons, the NBME requested that specific quality ratings not be shared publically. Items related to pain comprised 15.4% (232) of the 1,506 USMLE questions reviewed. An additional 13.3% (200) of the questions included the word “pain,” but these questions were not categorized as related to or testing pain knowledge.

Domains

All four domains within the pain competencies were represented in the three steps. Domain 2 (“How is pain recognized”) was the most common, found in 88.4% of the pain-related questions reviewed across the three steps. The other three domains of pain were substantially less represented in pain-related questions: Domain 1 (“What is pain?”) was found in 2.8%; Domain 3 (“How is pain treated?”) was found in 3.7%; Domain 4 (“The context of pain”) was found in 2.4% (Figure 2).

Figure 2

Pain competency domains within the USMLE. Findings represented in questions fully or partially related to pain (x/232) by pain competency domains (Table 1) [19].

Competencies

Addressing “Use of valid tools for assessing pain” (Competency 2.1) was the most frequently tested pain competency, observed in 79.7% of the pain-related questions. Other competencies were noted less frequently as a percentage of the pain-related questions: “Development of a pain treatment plan” (Competency 3.4) in 19.0%, “Nature of pain” (Competency 1.1) in 11.6%, and “Needs of special populations” (Competency 4.1) in 7.3% of the pain-related questions. Other competencies were less represented but were tested in at least one question in at least one step, though some were not covered in all steps (Figure 3).

Figure 3

Pain competencies within the USMLE. Findings represented in questions fully or partially related to pain (x/232) by individual pain core competencies (also see Table 1) [19].

Major Topics

In the review of major topics, “Clinical assessment of pain” (Topic 3) eclipsed all others at approximately 84.0% of the 232 pain-related questions. In descending frequency as a percentage of the pain-related questions: “Acute pain” appeared in 31.9%, chronic pain in 17.2%, “Pharmacological pain management” in 15.1%, “Geriatric pain” in 12.5%, and pediatric pain in 11.6% (Figure 4, Table 2). All other individual major topics were found less frequently, including nonpharmacological pain management and end of life care, among others.

Figure 4

Major topics. Major topics identified by the Education Committee of the American Academy of Pain Medicine, represented in questions fully or partially related to pain (Supplementary Data) [21].

Key Public Health Issues Including Prescription Drug Safety and Opioid Use Disorder

Review of key public health issues as a percentage of the 232 pain-related questions found the following, in order of highest to lowest frequency: “Older adults” at 17.2%, “Disparities” at 8.9, and adolescents at 6.5%. Other public health issues covered at lower rates included “Prescription drug safety and misuse” at 2.6% of the 232 pain-related questions, “Mental illness” at 1.7%, and “chronic disease with comorbid chronic pain” at 0.4%. As shown in Figures 2–5, regardless of the curricular classification used to gauge the scope and nature of pain and related topics among the USMLE questions, only a small proportion of these questions addressed prescription drug safety and opioid use disorder. Several key public health issues were not mentioned in the questions examined. These topics included “Labor pain,” “End of life pain,” “Military-related pain,” “Pain-related disability,” “Cancer pain treatment and cancer survival,” “Chronic pain after surgery,” and “Patient-reported outcomes” (Figure 5, Table 2).

Figure 5

Key public health issues. Key public health issues identified by expert reviewers participating in the present study, and their representation in questions fully or partially related to pain (x/232).

Discussion

The results of this study reflect pain-related knowledge and competencies within the three steps of the USMLE examination. These findings suggest that, although there is adequate testing of some pain-related areas, there are important gaps.

The overall percentage of questions within the USMLE that tested for pain content appears to be a reasonable proportion of the examination (15.4%; 232 of 1,506). Questions about how pain is recognized vastly outnumbered those appraising the fundamental understanding of pain as a biopsychosocial process or safe and effective pain management. Only a small minority of questions addressed what to do once pain was recognized. Topics other than pain assessment and recognition, such as understanding pain, safe and effective pain treatment, and needs of special populations, are essential to safe and effective pain management and merit greater representation in testing for licensure. Similarly, evaluation of pain concepts in USMLE questions also revealed disproportionate emphasis on pain assessment. Critically important topics, such as the medico-legal aspects of pain management, including risk mitigation in opioid prescribing, were represented in a small fraction of questions. This correlated with the small fraction of questions relating to prescription safety and misuse found through reviewing the “key public health issues” appraisal of the questions, suggesting internal validity in this study. Review of USMLE questions through the lens of “key public health issues” highlighted a number of additional areas where the content of the exam could be augmented. Examples include pain-related disability including back pain [22], chronic pain resulting from acute pain after surgery and trauma [23], military injuries [24] and their sequelae in veterans [25], and pain related to chronic disease [6,7].

The landmark National Academy of Medicine (NAM; previously Institute of Medicine [IOM]) reports (2011 and 2017, respectively) and the recently released strategic plan for implementation of pain-related recommendations (the National Pain Strategy [NPS]) emphasize the importance of professional education and training as one of the pillars of a comprehensive, integrated approach to improving pain care [6,7,16]. As the 2017 NPS states:

Education and training must allow learners to achieve discipline-specific core competencies, which include empathy and cultural sensitivity across a broad range of disciplines, and prepare them to provide high quality team-based care for pain. Demonstration of competency in pain assessment, safe and effective pain care (including specific training on safe opioid prescribing practices), the risks associated with prescription analgesics, communication of these risks to patients, and prescriber education should be a requirement for licensure and certification of health professionals and should be considered in curriculum review for accreditation of health professional training programs.

The results of the present study are fully aligned with these recommendations and provide a baseline by which to evaluate future progress toward achieving them. Moreover, the present study indicates a relative scarcity of questions testing examinees’ knowledge and competency in identifying patients at risk for aberrant opioid use or other harmful outcomes of public health interest during chronic opioid therapy [3].

The present review of USMLE test question content was structured to identify content on opioid use disorder in three of the curricular assessments employed: specific competencies, major topics, and public health. The paucity of findings in these categories may have been expected as responses from State and Federal government agencies to the ongoing epidemic of prescription opioid misuse, abuse, and unintentional drug overdose deaths increasingly acknowledge educational deficiencies in practicing clinicians in the focal areas of addiction and opioid prescribing [3,6,17,26,27].

For the past decade, great concern has centered on the seemingly incongruous co-occurrence of high rates of chronic pain and excessive prescribing of opioid medications [3,6,7,17]. The current urgent concerns over pain treatment and drug abuse are striking in contrast to decades of efforts to raise awareness of pain and its consequences. For instance, “pain as the fifth vital sign” was embraced by the US Department of Veterans Affairs (VA) in the late 1990s [28]. Recognition of pain intensity as a fifth vital sign has been challenged as too simplistic, inadequately capturing the complexity of the multidimensional pain experience and an understanding of pain theory and comprehensive management [29,30]. Likewise, the Joint Commission required that pain must be assessed in all patients until amending this requirement in 2009, in line with the recognized need for multidimensional identification and management of pain [31,32]. Reliance on assessment of pain as a principal means of improving pain management may rest upon an assumption that if pain were only recognized, effective treatment would simply follow. The paucity of comprehensive pain education for clinicians at all levels of prelicensure and postgraduate health education and the current public health crises related to pain, as well as the excessive use and abuse of prescription opioids, suggest that this assumption is ill founded [9,19].

Some limitations temper the interpretation of our findings. Time constraints limited the number of questions reviewed, and the questions reviewed were a nonrandom sample of the overall USMLE following the order of the 2014 content outline (Supplementary Data) [33]. There is a possibility that a heavier weighting of certain pain topics or public health issues could have been detected in questions at the end of each reviewed test set that were not assessed due to time limitations. It is possible that characterizing questions in sequential order would underemphasize topics represented late in the USMLE content outline. These might include pregnancy and female reproduction, as well as end of life care. Although certain topics may have been subject to greater ascertainment had questions been reviewed in a random order, a review of the content outline does not suggest that the high frequency of questions on pain assessment would have changed (Supplementary Data).

Competency in the broad spectrum of pain knowledge and treatment should help equip the next generation of clinicians to treat pain more safely and effectively (and prevent pain when feasible). With such a shift in the focus of medical education, excessive reliance on opioid analgesia may be expected to diminish [34]. Conversely, multimodal individualized treatment plans, including nondrug (e.g., behavioral and physical) therapies and nonopioid pharmacotherapy, would be expected to become more common. The present findings can be used to inform stakeholders, such as the organizations that certify and license students or accredit health professional schools, and ensure that new clinicians are better prepared to manage the dual public health burdens of inadequately treated pain and the epidemic of opioid misuse.

Conclusions

The need to assure competency in safe and effective pain care is now well described in numerous governmental reports, including the NPS and the CDC guidelines for prescribing opioids [3,19] and nongovernmental white papers [7,8]. The present evaluation found that despite ample numbers of questions related to pain in the USMLE, pain assessment was disproportionately represented compared with the nature and context of pain, or how pain can be safely and effectively treated. The present findings should help guide future versions of the USMLE as well as promote fundamental reform of pain education for health professionals [4].

Click here for additional data file.