Evaluation of the International Classification of Diseases-11 chronic pain classification: study protocol for an ecological implementation field study in low-, middle-, and high-income countries.

INTRODUCTION: The purpose of the present ecological implementation field study is to evaluate the new classification of chronic pain as implemented in the 11th revision of the International Classification of Diseases (ICD-11) with regard to clinical utility and interrater reliability. To evaluate the classification in a variety of settings, the study will be implemented in different low-, middle-, and high-income countries.
METHODS: The study will be conducted in 2 phases. Participating pain clinics of the first phase are located in India, Cuba, and New Zealand. Two or more clinicians per study center will use the ICD-11 classification of chronic pain to diagnose 75 to 100 consecutive new chronic pain patients per center. A structured classification algorithm will guide the diagnostic process. Interrater reliability will be analyzed for the first 20 consecutive new patients per center. Before the coding, a training workshop will introduce the clinicians to the new classification. The main outcome parameter of the ecological implementation field study is clinical utility. More specifically, this entails clinical utility ratings, interrater reliability, as well as the exhaustiveness of the classification and the mutual exclusiveness of the new chronic pain categories. Differences between countries with different cultural backgrounds and income levels will be analyzed. PERSPECTIVE: The ecological implementation field study presented here will be implemented in several countries with different income levels. This increases the generalizability of the results and allows initial insight into the global applicability of the new chronic pain classification. A positive evaluation can facilitate the implementation of the classification.

1. Introduction

Chronic pain is a highly prevalent condition affecting up to 20% of the global population[6] and contributing significantly to the global burden of disease.[25,33] Despite its significance, chronic pain is not represented adequately in the current version of the International Classification of Diseases (ICD-10).[26,27] To overcome these problems, an international task force of the International Association for the Study of Pain (IASP) developed a new classification of chronic pain for the ICD-11.[31,32]

In this new classification of chronic pain for the ICD-11 (hereafter termed “the ICD-11 chronic pain classification”), chronic pain is defined as pain that persists or recurs for longer than 3 months.[31,32] The classification distinguishes 7 categories of chronic pain: chronic primary pain,[19] chronic cancer-related pain,[3] chronic postsurgical or posttraumatic pain,[29] chronic neuropathic pain,[28] chronic secondary headache or orofacial pain,[4] chronic secondary visceral pain,[1] and chronic secondary musculoskeletal pain.[20] Each category of chronic pain comprises several subcategories or diagnostic levels to allow diagnosis on a more granular level (eg, chronic widespread pain as a sublevel or level 2 diagnosis of chronic primary pain).[19] Table 1 gives an overview of the classification with its different levels, including the new diagnostic codes.[35] Furthermore, the classification allows any chronic pain condition to be described further by assigning so-called extension codes.[31] That is to say, patients rate the intensity of their pain as well as their pain-related distress and pain-related interference on a 0 to 10 numerical rating scale (NRS) or on a visual analogue scale. The combination of these ratings represents the specifier of pain severity. The temporal course of the chronic pain can be coded as persistent, recurring with pain-free intervals, or persistent with pain attacks, and forms a second specifier. Finally, the presence or absence of pain-related psychosocial factors such as pain catastrophizing or fear avoidance can be recorded as well.[31]

Table 1

Overview of the ICD-11 chronic pain classification.

The ICD-11 chronic pain classification has been added to the ICD-11 platform[36] and is now part of the frozen version of the ICD-11 Mortality and Morbidity Statistics (MMS) for preparing implementation by member states.[35] In May 2019, the World Health Assembly agreed to adopt the ICD-11, which will come into effect in 2022.[37]

Any new diagnostic classification needs to be evaluated. Ecological implementation field studies allow evaluation of the reliability of a new classification as well as evaluation of its clinical utility in a realistic clinical setting.[14] Furthermore, ecological implementation field studies enable evaluation of the implementation of the new classification into the clinical setting.[23] Clinical utility refers to the degree by which a classification contributes to the communication of clinical information, to adequate treatment decisions, to facilitated documentation, and to patient management.[9,10,18] The World Health Organization (WHO) emphasized the improvement of clinical utility as one of the main goals of the ICD-11 revision, alongside global applicability.[12] A pilot evaluation study of the ICD-11 chronic pain classification, conducted in different primary care as well as specialty pain treatment centers in 4 countries, showed that clinicians rated the clinical utility of the 7 new main categories as good to very good.[2] A following online field study in cooperation with the WHO revealed that pain specialists assigned the correct ICD-11 code to the majority of chronic pain diagnoses after having received minimal training (Barke, Korwisi, Jakob, Konstanjsek, Rief & Treede, manuscript in preparation.) For most diagnoses, ICD-11 performed better than ICD-10 with regard to correct code assignment. Furthermore, the clinical utility of the ICD-11 chronic pain classification was rated as very useful.

It is essential for a classification system that it is clinically useful in a variety of settings, including primary care and countries with fewer resources than high-income countries.[21] Furthermore, conducting ecological implementation field studies in multiple countries with different income levels and different cultural backgrounds enhances the generalizability of the results, thus contributing to the knowledge regarding the global applicability of the classification.[11,22]

Ecological implementation field studies for the evaluation of new classification systems that have to be based on self-reported information rather than observable biological markers usually include clinical interviews or assessments conducted by at least 2 independent raters to establish interrater reliability.[7,17,23,24] Commonly, participating clinicians of ecological implementation field studies are familiarized with the new diagnostic guidelines to be evaluated before data collection.[7,14,17,23,24]

The goal of this study ( Chronic Pain Codes Ecological Testing and Assessment: ICE TEA) is to evaluate the ICD-11 chronic pain classification in terms of clinical utility and interrater reliability. Furthermore, the exhaustiveness of the classification and mutual exclusiveness of the diagnostic categories will be analyzed. It is expected that conducting the study in countries with different cultural backgrounds and with different income levels will enable conclusions about the global applicability of the classification.

2. Methods

The study protocol presented here is guided by the description of the official WHO ecological implementations field studies for mental and behavioral disorders by Keeley et al.[14] The study will be implemented in 2 phases of data collection. For the first phase, ethical approval has been obtained before data collection from the Department of Psychology at the University of Marburg, Germany (approval number 2018-41k) as well as the following participating study centers: Havana, Cuba (approval number 13 on November 18, 2018), Kolkata, India (approval number 010/2018), and Dunedin, New Zealand (approval number H19/105). The study center in Hyderabad, India, did not require an on-site ethical approval because the approval from Marburg as coordinating center was accepted there. For the second phase, ethical approval will also be obtained from all participating study centers before data collection.

2.1. Study setting

The ICE TEA study will be conducted in pain clinics (mostly outpatient) in low-, lower-middle-, upper-middle-, and high-income countries as defined by the World Bank.[34] As was done in similar studies, the study centers are selected based on their interest in participation as well as available resources needed for implementation of the study.[11] For the first phase, pain clinics in India (lower-middle-income country), Cuba (upper-middle-income country), and New Zealand (high-income country) have been selected as study centers. Table 2 gives an overview of some characteristics of the study centers of the first phase. The second phase will include at least 2 clinics per income category, covering all WHO regions (eg, Iran, Thailand, Germany, and United States) as well as low-income countries. Furthermore, specialty treatment centers other than pain clinics as well as primary care centers will be irecruited in this second phase to ensure that all 7 main chronic pain categories are represented in the final sample (eg, palliative care to include chronic cancer-related pain, internal medicine to include a variety of chronic visceral pain.)

Table 2

Characteristics of the study centers of the first phase of data collection.

2.2. Participants

The ICE TEA study will include 2 sets of participants: (1) 2 or more participating clinicians per study center who will evaluate the new classification of chronic pain after using it for their diagnoses (hereafter termed “clinicians”); and (2) 75 to 100 consecutive new chronic pain patients per study center (hereafter termed “patients”) who will be diagnosed by the clinicians.

2.2.1. Clinicians

Participating clinicians will work at pain centers. They will have a specialist education in pain medicine or have worked with patients with chronic pain for more than 3 years. This includes specialists in different disciplines such as physicians and clinical psychologists licensed to perform psychological treatment, who are standardly involved in multidisciplinary diagnostic processes of chronic pain with additional physical examinations performed by physicians. In the second phase, clinicians of other specialties (eg, primary care, palliative care, oncology, and internal medicine) with extensive experience with chronic pain will be included. Before the coding, all participating clinicians will take part in a brief on-site training workshop. At least 2 clinicians per study center will participate to establish interrater reliability. This will also serve to prevent that data can be connected to an individual clinician. All clinicians will provide their informed consent before their participation.

Clinicians will be eligible for participation if they meet the following inclusion criteria:

(1) Participation in the training workshop

(2) Very good level of English

(3) Working at a pain clinic (physician or licensed clinical psychologist)

(4) Available at the study center until the end of the data collection.

2.2.2. Patients

Each consecutive new patient who presents at the study center (inpatients and outpatients) will be invited to participate in the study. Only patients who provide their informed consent will be included in the study. New patients are defined as patients who consult the respective pain clinic or specialty center for the first time for the current chronic pain problem.

Patients will be eligible to participate in the study if they meet the following inclusion criteria:

(1) Aged at least 18 years

(2) Pain for longer than 3 months

(3) Able to communicate in English or another language spoken by the clinician (eg, Bengali, Hindi, Spanish, or Telugu for the first phase)

(4) Able to participate in a structured diagnostic process.

2.3. Study material

Clinicians will provide basic information at the beginning of the study. All patients will complete a set of questionnaires before the diagnostic assessment. The clinician will use a standardized classification algorithm to establish the chronic pain diagnoses for each patient. After this diagnostic process, the clinician will complete a Code Assignment and Evaluation Form (CAEF) for each patient. This evaluation form is intended to measure the main outcome. Because the ICD-11 is only available in English until it will come into effect in 2022, all study material for the clinicians will be in English. The clinician measures, patient measures, and CAEF are available in the supplemental digital content (SDC 1, available at http://links.lww.com/PR9/A64). The classification algorithm is currently being prepared as a separate publication.

2.3.1. Classification algorithm

The assessment of each patient will follow a standardized classification algorithm. It will be documented on the algorithm introduction form. The algorithm is a linear decision tree that will guide the clinician through the new diagnostic criteria for all chronic pain conditions. Where necessary, clinicians may refer to existing medical records (eg, referral documentation) if these are judged still to be conclusive. The diagnostic codes listed in the classification algorithm are based on the ICD-11 MMS, 2018 version.[35]

2.3.2. International Classification of Diseases-11 classification handout

In addition to the classification algorithm, the definitions and diagnostic criteria for all ICD-11 chronic pain conditions will be available during the diagnostic assessment and the following code assignment. Due to limited internet access at some of the participating study centers of the first phase, all raters will have access to an ICD-11 classification handout. This is a printed or PDF version of the definitions and diagnostic criteria of the ICD-11 chronic pain classification as implemented in the current version of the ICD-11.[35]

2.3.3. Clinician measures

During the training workshop, clinicians will provide basic demographic information as well as information regarding their professional experience (eg, years of experience working with chronic pain patients) and an initial evaluation of the ICD-11 chronic pain classification (baseline measure before its application) on an 11-point NRS from 0 (not useful at all) to 10 (very useful).

2.3.4. Patient measures

The patient measures will include basic demographic data, a pain history questionnaire, a pain localization chart, items to assess the chronic pain specifiers, as well as a set of questionnaires to assess pain-related variables such as pain-related disability and other psychological symptoms.

2.3.4.1. Pain Disability Index

The Pain Disability Index[30] is a 7-item questionnaire that assesses how much the pain interferes with different daily activities, such as occupation and social activities. All items are rated on an 11-point NRS from 0 (no disability) to 10 (total disability). The Pain Disability Index will be included as a measure of pain-related interference.

2.3.4.2. Brief Symptom Inventory-53

The Brief Symptom Inventory[8] is a 53-item questionnaire that lists different problems and complaints, such as nervousness or dizziness. Patients indicate how much they were bothered by these symptoms in the past week on a 5-point NRS from 0 (not at all) to 4 (extremely). The Brief Symptom Inventory-53 allows calculating a global symptom index and will be included as a measure of pain-related distress.

2.3.5. Code Assignment and Evaluation Form

The clinician who conducted the diagnostic assessment will complete the CAEF within 24 hours of the assessment. Analysis of the CAEFs will form the basis for answering the main research questions. During the interrater reliability coding, both clinicians who are present for the assessment complete the CAEF, blind to each other's answers. The CAEF includes the following:

(1) All ICD-11 chronic pain diagnoses assigned to the patient (ICD-11 MMS 2018 version[35])

(2) All chronic pain diagnoses as usually assigned in the clinic (see Table 2 for details)

(3) In the case of chronic secondary pain, the name of the underlying disease (second phase only: ICD-11 code of the underlying disease)

(4) Time taken to complete the diagnostic assessment and code assignment

(5) Five items on diagnostic tests, if required (eg, whether a test was required by the diagnostic criteria, whether it could be performed, reasons why a test was not performed)

(6) Presence or absence of psychosocial factors

(7) Clinical utility rating on an 11-point NRS from 0 (very difficult/not confident at all/not useful at all) to 10 (very easy/very confident/very useful) (ease of use, diagnostic confidence, overall utility, specific utility regarding: communication with colleagues and patients, data collection, documentation, patient management, treatment selection, improvement of outcome)

(8) Clinical utility rating of the current diagnostic system on the same 0 to 10 NRS

(9) Evaluation of the classification algorithm in its pilot version regarding difficulty, confidence, and utility on the same 0 to 10 NRSs (first phase only)

2.4. Procedure

Both phases of data collection will follow the same procedure. The first part of the ICE TEA study for each study center will consist of an introductory training course. Then, the actual coding of consecutive new patients for this center will begin. This is divided into 2 parts: the first part will be an interrater reliability coding with 2 raters present. The second part will be a continued consecutive coding by just one clinician. B.K. will be present during the training workshop as well as for the interrater reliability phase, to facilitate implementation of the study.

Patients will provide their informed consent and complete the questionnaires before the diagnostic assessment. Then, the clinician will diagnose the patient using the standardized classification algorithm. After the diagnostic assessment, the clinician will assign the respective diagnostic codes (ICD-11 and code as usual, eg, textbook diagnosis[5,13,16]), and evaluate the chronic pain classification and the algorithm with the CAEF. Figure 1 gives an overview of the procedure.

Figure 1.

Study procedure. This procedure does not differ between the first and the second phase of data collection. ICD-11, International Classification of Diseases-11.

2.4.1. Training workshop

At each study center, a brief training workshop for participating clinicians will be held before the actual coding by one of the authors (B.K.). This workshop will last approximately 4 to 6 hours and will be mandatory for the participating clinicians. The plan of the training workshop follows the clinician training of previous ecological field studies.[7,17,23,24] The training workshop will comprise a review of the study protocol and all study material, a presentation of all ICD-11 chronic pain diagnoses and the chronic pain specifiers, as well as an introduction to the classification algorithm. During this last part, clinicians will have the opportunity to use the algorithm with a chronic pain patient for practice purposes. At the end of the training workshop, all clinicians will provide their informed consent for study participation and complete a brief knowledge test.

2.4.2. Interrater reliability coding

In a first step, an interrater reliability coding will take place. Due to the limited amount of resources, it will not be feasible to assess each patient by 2 clinicians. However, a limited interrater reliability coding comprising the first n = 20 consecutive new patients per study center will take place after the training workshop to establish measures of interrater reliability. During this coding, one assessment will be conducted per patient with 2 clinicians (Clinician A and Clinician B) being present simultaneously. Clinician A will assess the patient by referring to the classification algorithm. Clinician B will attend as observer. The assessment will be conducted in the patient's language (ie, English, Bengali, Hindi, Spanish, or Telugu). B.K. will be present for guidance. At the end of the assessment, Clinician B and B.K. may ask additional questions, if needed.

This follows the procedures as implemented in similar field studies.[14,17,23] Both clinicians will assign the respective chronic pain codes independently and blind to each other's assignments. If more than 2 clinicians participate in a study center, these trios will be changed alternately. Clinicians will take turns regarding the role of interviewer and observer.

2.4.3. Continued consecutive coding by one clinician

After the interrater reliability coding, the following n = 55 to 80 consecutive new patients per study center will be assessed. During this part of the coding, only one clinician will be present for the diagnostic assessment and to complete the CAEF.

2.4.4. Code assignment and evaluation

After each diagnostic assessment, the clinician will complete a CAEF. On this form, the clinician will assign the pain diagnoses he or she would routinely use (eg, textual diagnosis) as well as the new ICD-11 chronic pain diagnoses (level 3 diagnoses wherever possible, see Table 1 for examples). See Table 2 for an overview of the current diagnostic systems used at the study centers. The ICD-11 codes and diagnostic criteria will be available during the coding. Furthermore, the CAEF will include questions regarding the clinical utility of the classification and the classification algorithm (see above).

2.5. Outcome

The ICE TEA study will focus on 4 aspects of clinical utility: clinical utility ratings, interrater reliability, exhaustiveness of the classification, and mutual exclusiveness of the categories of the ICD-11 chronic pain classification. Furthermore, differences between countries will be analyzed.

(1) Clinical utility ratings: How do clinicians perceive the clinical utility of the ICD-11 chronic pain classification regarding ease of use, diagnostic confidence, communication, treatment selection, patient management, documentation, data collection, and improved outcome? A general clinical utility rating of the current diagnostic system will be obtained as well. (Mean values of the clinicians' ratings)

(2) Interrater reliability: If 2 clinicians have the same diagnostic information on a given patient, do they assign the same chronic pain code? (Measure of inter-rater reliability)

(3) Exhaustiveness: Can all patients with chronic pain be classified according to the classification? (Proportion of patients who fall into the unspecified residual category)

(4) Mutual exclusiveness of the categories: Can the chronic pain conditions of all patients be classified into exactly one of the new categories? (Number of patients who cannot be clearly assigned a category)

(5) Influence of available resources and cultural background: Does the clinical utility as operationalized in the above variables differ between low- (only second phase), lower-middle-, upper-middle-, and high-income countries?

In addition, the first phase of data collection will also focus on feasibility aspects of the study implementation.

2.6. Statistical analyses

To establish interrater reliability, Kappa coefficients will be calculated for each diagnosis that is present in at least 15 patients who have been coded by 2 clinicians.[24] Kappa coefficients will be computed in a cascading way, first for level 1 diagnoses, followed by computations for level 2 and 3 diagnoses whenever possible.

For each study center, the mean clinical utility ratings will be computed. Differences in the clinical utility ratings between the countries will be analyzed with separate one-way analyses of variance. To examine the exhaustiveness of the classification, the percentage of pain syndromes that are classified as “unspecified” will be analyzed per study center. The mutual exclusiveness of the new diagnostic categories will be computed as the percentage of chronic pain syndromes per study center for which more than one diagnosis applies, excluding cases of comorbid chronic pain conditions.

2.7. Sample size

The sample size was determined in cooperation with the participating study centers as the maximum number of patients for whom the study procedure can be implemented during routine clinical practice.

3. Discussion

The ICE TEA study is an ecological implementation field study that aims to evaluate the ICD-11 chronic pain classification in 2 phases of data collection. This evaluation will investigate the clinical utility and interrater reliability of the classification as well as the exhaustiveness and mutual exclusiveness of the new categories. The study will be conducted in several countries with different resources and a variety of cultural backgrounds, which increases the generalizability of the results and enables initial analysis of the global applicability of the classification.[11,23] Although the study does not focus on the validity of the new chronic pain classification, an important aspect of validity may still be inferred from the mutual exclusiveness of the different chronic pain categories.[15]

The study protocol presented here is guided by the procedure of the official WHO field tests for mental and behavioral disorders,[14] thereby enabling its integration with other ICD-11 field testing efforts and facilitating the comparison and interpretation of the results. It is another strength of the study proposed here that differences in clinical utility between countries will be analyzed as well, which has been done only for a small number of similar previous field studies.[22] The results of the first phase of data collection will enable an initial analysis of country differences. These results will be corroborated in the second phase, where the amount of countries and settings will increase substantially. Furthermore, there might be ways to introduce a quantifiable outcome measure apart from self-reported information (eg, a time measure for documentation) in the second phase of data collection if possible at the study centers.

As was done in similar field studies of classification systems with multiple raters and patients potentially presenting with several diagnoses,[7,17,23,24] kappa coefficients will be computed as a measure of interrater reliability. Due to the limited resources available for this type of study in the context of routine patient care and the resulting limited sample size of the interrater reliability coding, it probably will not be possible to compute kappa for each of the 7 main categories after the first phase of data collection. The inclusion of specialty treatment centers and more centers per income category will enable computations for less frequent level 1 diagnoses (eg, chronic cancer-related pain) and more level 2 and 3 diagnoses. Furthermore, depending on the final sample size per diagnostic category in each study center, it might be possible to compare kappa coefficients between countries or income levels for the most prevalent diagnoses after the second phase of data collection.

The ICE TEA study builds on previous field testing efforts of the ICD-11 chronic pain classification while aiming at overcoming some of the limitations encountered there.[2] The 2016 pilot field study[2] only assessed the 7 main chronic pain categories, causing a diagnostic bias of double diagnoses, which would have been resolved on the next diagnostic level. By coding patients with more specific diagnoses on the third diagnostic level (see Table 1 for examples), this study enables a more accurate analysis of mutual exclusiveness and a more realistic coding compared to the 2016 study. Furthermore, the participating clinicians of this study will receive training before using the new classification, which was not possible in the 2016 pilot study. Although only one global clinical utility rating with regard to the main categories was obtained in 2016, this study will assess clinical utility more extensively and with regard to the more specific diagnoses. This increases the validity of the clinical utility ratings.

The 2017 online field study (Barke, Korwisi, Jakob, Konstanjsek, Rief, Treede, manuscript in preparation) was conducted as part of the official WHO field testing efforts. Here, clinicians assigned ICD-11 codes to diagnostic statements and rated the clinical utility of the new diagnoses when applied to short case vignettes. A brief online training was provided, but it could not be controlled whether participants completed it before their study participation. Although the use of standardized case vignettes enabled control over reported patient characteristics, this study will provide important additional aspects of evaluation by analyzing the implementation of the new classification in routine clinical practice, involving real patients. Furthermore, participation in the training will be controlled in this study, ensuring that all participating raters have the same amount of knowledge with regard to the new classification.

Although the diagnostic spectrum of participating patients in the first phase of data collection cannot be controlled, the inclusion of specialty treatment centers for underlying diseases that are often associated with chronic pain (such as palliative care) in the second phase will ensure that all categories of chronic pain will be represented in the final sample. The increased number of study centers from all WHO world regions in the second phase will corroborate the interrater reliability further and ensure the generalizability of the results in a global context. Strengths and limitations as well as obstacles from the first phase will directly influence the implementation of the second phase. However, it is likely that the amount of patients per category will differ between the study centers nevertheless.[2] These differences between the study centers may confound country differences and will need to be taken into consideration when interpreting the results.

An important strength of ecological implementation field studies is that they allow to evaluate the new classification in the clinical setting where it will be used later.[14] This ecological validity of such implementation studies comes at the price of less control over other aspects such as the patients' chronic pain conditions and other influencing factors. Here, case-controlled field studies involving standardized case vignettes can provide additional information and should also be conducted in the future.[14]

In conclusion, the ecological implementation field study presented here will assess the clinical utility and interrater reliability of the new ICD-11 classification of chronic pain in a global context. The exhaustiveness and mutual exclusiveness of the categories will be analyzed as well as quality indicators of the new classification. High clinical utility can facilitate the classification's global implementation, as clinicians are more likely to use a new classification consistently if it is perceived as useful.[22]

Disclosures

B. Korwisi reports other from IASP (NGO), outside the submitted work; R.-D. Treede reports grants from Bayer, personal fees from Grünenthal, Hydra, Bayer, decision Res, GSK, and Pfizer, and grants from EU, DFG, and BMBF, outside the submitted work; W. Rief has nothing to disclose; A. Barke reports other from IASP (NGO), outside the submitted work.

The IASP has financially supported the development of a chronic pain classification system for the ICD-11, by providing research grants and travel grants to the authors. This support was not part of the hereby-submitted work.

Supplemental digital content associated with this article can be found online at http://links.lww.com/PR9/A64.