Mixed-methods development of a new patient-reported outcome instrument for chronic low back pain: part 2-The Patient Assessment for Low Back Pain-Impacts (PAL-I).

We describe qualitative and quantitative development and preliminary validation of the Patient Assessment for Low Back Pain-Impacts (PAL-I), a patient-reported outcome measure for use in chronic low back pain (cLBP) clinical trials. Concept elicitation and cognitive interviews (qualitative methods) were used to identify and refine symptom concepts. Classical test theory and Rasch measurement theory (quantitative methods) were used to evaluate item-level and scale-level performance of the PAL-I using an iterative approach between qualitative and quantitative methods. Patients with cLBP participated in concept elicitation interviews (N = 43), cognitive interviews (N = 38), and assessment of paper-to-electronic format equivalence (N = 8). A web-based sample of self-reported patients with cLBP participated in quantitative studies to evaluate preliminary (N = 598) and revised (n = 401) drafts and patients with physician-diagnosed cLBP (N = 45) participated in preliminary validation of the PAL-I. The instrument contained 9 items describing cLBP impacts (walking, sitting, standing, lifting, sleep, social activities, travelling, climbing, and body movements). Item-level performance, scale structure, and scoring seemed to be appropriate. One-week test-retest reproducibility was acceptable (intraclass correlation coefficient 0.88 [95% confidence interval, 0.78-0.94]). Convergent validity was demonstrated with PAL-I total score and Roland-Morris Disability Questionnaire (Pearson correlation 0.82), MOS-36 Physical Functioning (-0.71), and MOS-36 Bodily Pain (-0.71). Individual item scores and total score discriminated between numeric rating scale tertile groups and painDETECT categories. Interpretation of paper and electronic administration modes was equivalent. The PAL-I demonstrated content validity and is potentially useful to assess treatment benefit in clinical trials of cLBP therapies.

1. Introduction

Low back pain (LBP) is described as “pain, muscle tension, or stiffness localized below the costal margin and above the gluteal folds, with or without leg pain.”[1,5,20] Low back pain is a common and painful condition that has been reported to have a global prevalence up to 84% depending on the case definition.[2] The prevalence of chronic LBP (cLBP), defined as LBP with duration >12 weeks,[21] is estimated to be ∼23%.[2] Low back pain causes more global disability than any other condition[10] and is frequently work-related. Treatments for cLPB include nonsteroidal anti-inflammatory drugs, acetaminophen, muscle relaxants, opioid pain medications, and physical therapy, which are effective for most patients.[11] For moderate-to-severe pain, epidural steroid injections, facet injections, and radiofrequency ablations are used.[11] Several promising new therapies are in phase 2/3 clinical trials for the treatment of cLBP, including ethanol gel for use in chemonucleolysis protocols (a process of injecting proteolytic enzymes into the intervertebral disk), platelet-rich plasma, stem-cell therapy for disk regeneration, tanezumab (an anti–nerve growth factor antibody) for nociceptor modulation, artemin (a neurotrophic growth factor) for neuronal regrowth,[11] and cebranopadol, a nociceptin/orphanin FQ peptide and opioid peptide receptor agonist.[6]

Patient-reported outcome (PRO) measures are used to evaluate patient perspective in clinical trials. Patient-reported outcome measures capture the health experience of patients and can be used to document treatment benefit and support labeling claims.[17,19] Many PRO measures/instruments are currently used in clinical trials of cLBP, including the Neuropathic Pain Symptom Inventory (NPSI), painDETECT, Roland-Morris Disability Questionnaire (RMDQ), Pain Quality Assessment Scale–Revised, Revised Short-Form McGill Pain Questionnaire, Low Back Pain Impact Questionnaire, Oswestry Disability Index, Pain Disability Index, Brief Pain Inventory and Brief Pain Inventory–Short Form, Musculoskeletal Outcomes Data Evaluation and Management System Spine Module, Orebro Musculoskeletal Pain Questionnaire, and the West Haven-Yale Multidimensional Pain Inventory Interference Scale. Most of the instruments measure pain and/or disability, hallmarks of cLBP.

The U.S. Food and Drug Administration (FDA) has published a guidance for drug sponsors outlining requirements for the development of PRO measures to ensure that they reliably measure the claimed concept in the patient population enrolled in the clinical trial.[19] We conducted a literature search to identify PRO instruments that could be used for a label claim for cLBP and found that most available measures were not developed in accordance with the U.S. FDA guideline,[18] particularly the requirement to demonstrate relevance to the intended patient population. This requires initial qualitative evidences, but these were not found in the published literature for the measures that were evaluated. Several measures lacked specificity to a cLBP population, and others had difficulties that were apparent in the clarity of item construction and the singularity of concepts presented. Therefore, we have developed and conducted preliminary validation studies on 2 PRO instruments for use in cLPB: the Patient Assessment for Low Back Pain–Symptoms (PAL-S)[14] and the PAL–Impacts (PAL-I). We now report the mixed-methods development of the PAL-I.

2. Methods

2.1. Study design and development steps for PAL-I questionnaire

The PAL-I was developed in parallel with the PAL-S, and the study design and development have been fully described.[14] Briefly, a mixed-methods approach, which includes both qualitative and quantitative methods, was used to finalize the questionnaires (Fig. 1). Qualitative methods were used to generate and refine the concepts included in the PRO, and quantitative methods were used to evaluate the concepts in the PRO. Because of the iterative nature (alternating use of qualitative and quantitative methods as they inform each other) of this approach, the methods and results are presented in chronological order.

Figure 1.

Steps in mixed-methods development of the PAL-I. Qualitative methods (blue) and quantitative methods (green) are shown for chronological steps in the development of the PAL-I. (Reproduced from Martin et al.[14])

A survey of the LBP literature was conducted to identify impact concepts and to review currently available instruments that could be relevant to patients with cLBP. Although many instruments were identified, none was sufficient for use in labeling claims[18] based on FDA guidance.[19] Qualitative methods used to develop the PAL-I included concept elicitation (CE) interviews and cognitive interviews, and the quantitative development was based on administration of the instrument to a large sample population with cLBP. The equivalence of paper and electronic platforms was also established using cognitive interviews.

This study was performed in accordance with Good Clinical Practice and applicable regulatory requirements. The study was approved by the Quorum Review Institutional Review Board for sites in the United States. For sites in the EU, market research facilities in Germany and the United Kingdom recruited patients from their databases and used their standard consent form process. No medical information or records were accessed or used with these groups, and therefore, no formal ethics review was required.

2.2. Patient population and recruitment

Patient recruitment included patients with cLBP across a spectrum of pain severity as well as demographic characteristics, and the sample populations have been described.[14] The PAL-I is intended for use in global clinical trials. To capture global patient experiences, patients were recruited from 5 clinical sites in the United States, 1 site in Germany, and 1 site in the United Kingdom. Seven cohorts of patients were recruited for (1) CE to identify impact concepts (N = 43); (2) first round of cognitive interviews to evaluate the instrument (N = 30); (3) second round of cognitive interviews to evaluate modifications to the instrument (N = 8); (4) cognitive interviews to evaluate paper to electronic equivalence (N = 8); (5) first quantitative administration (N = 598); (6) second quantitative administration (a subset of the first quantitative administration; n = 401); and (7) preliminary psychometric validation (N = 45).

Patients with a current pain score ≥4 on a 0- to 10-point numeric rating scale (NRS; moderate-to-severe pain) were eligible to participate in CE, cognitive interviews, and preliminary psychometric validation, and patients with pain NRS scores of 0 to 10 were eligible to participate in quantitative administrations. Pain likely to be neuropathic LBP was identified based on a painDETECT[8] score >19 during screening[9]; painDETECT scores were descriptive only and used to guide recruitment targets (at least 40% with neuropathic pain component in the quantitative population).

2.3. Concept elicitation interviews

Concept elicitation interviews were conducted using an interview guide that was designed to obtain both spontaneous and prompted input from patients about the impact of cLBP symptoms, as previously described for the PAL-S.[14] Concept elicitation interviews were audio recorded and transcribed, and the transcripts were coded and organized using Atlas.ti software[15] to develop a coding framework. The coding framework was used to organize and group information with similar content and was revised as needed to fit the concept information from the interview transcripts.

2.4. Quality indicators for qualitative data

Saturation of concept occurs when no new information is forthcoming from the CE interviews. Transcripts were chronologically ordered and grouped into quartiles of 10 and 11 transcripts. New concept codes for each subsequent transcript group were compared with codes from the preceding group until no new information was forthcoming from subsequent interviews, indicating saturation of concept.

Codes were identified and coded by 2 coders. The degree of consistency between coders was evaluated by dual coding of 10% of the transcript database and comparing each pair of coded transcripts for differences.

2.5. Item generation

Items to be included in the instrument were identified from expert input, a review of existing cLBP PRO instruments,[18] and CE interviews. Draft items were constructed to be at the appropriate reading level using patient-derived language and terminology.

2.6. Cognitive interviews (round 1) for instrument refinement

Cognitive interviews were conducted to evaluate the accuracy and consistency of patient comprehension of concepts presented in the draft items.[16] Patients were also asked about instrument instructions, response options, the recall period, and specific terminology.

2.7. First quantitative administration (pilot data collection)

The first quantitative administration was conducted to evaluate item performance and determine whether any items required revision. Classical test theory was used for item reduction, and included evaluations of missing data, ceiling and floor effects, item-to-item correlations, item-to-total correlations, factor analysis, and estimations of reliability. Rasch measurement theory (RMT) analyses were used to examine the measurement model and scoring of the PAL-I instrument, and to assess item-level performance.

The PAL-I instrument was administered using an existing web-based panel through Ipsos Observer (https://www.ipsos.com/en-us/online-research). Individuals who had previously participated in the Ipsos research panel and had reported cLPB were invited to complete a web-based survey to confirm their eligibility. Eligibility criteria included as follows: confirmation of clinical diagnosis and current cLBP, duration of cLBP, pain intensity, absence of recent low back surgery or planned low back surgery in the next 30 days, and recent epidural injections or spinal cord stimulation therapy. Participants completed the PAL-I and a questionnaire that included demographic characteristics, clinical characteristics (severity and location of back pain, pain movement, and sciatica/neuropathic pain assessment), and items regarding treatment (medication type, current treatment or not, duration of treatment, satisfaction with current treatment, and other nonmedication treatment).

Analyses included descriptive statistics and floor/ceiling effects for individual item responses. An item-to-item correlation matrix based on Pearson's r was constructed for each item; coefficients >0.70 suggested a potential redundancy between items. Item-to-total score correlations based on bivariate Pearson's r were evaluated (excluding the item of interest from the total score); coefficients <0.40 suggested potential nonassociations with the remaining items in the hypothesized scale. Items were evaluated for appropriate psychometric scaling based on RMT analysis, and required that item response options were ordered and the items formed a unidimensional construct (root-mean-square error of approximation). Items that did not fit the RMT model were considered candidates for item reduction or revision. Category probability curves (item characteristic curves) were used to identify items that did not demonstrate monotonically increased responses. Simulation analyses were conducted for items that exhibited disordered thresholds (inconsistent responses) by collapsing response categories to assess potential improvements in item characteristics. Consistency of response was examined using a person–item distribution map, which displays persons and items on a logit scale with the most able persons and most difficult items on one side and the least able persons and easiest items on the other side. To avoid large gaps in measurement, the distance between items should be <0.30 logits.[3] Cronbach's alpha was used to assess internal consistency.

2.8. Cognitive interviews (round 2) to test modifications

After the first quantitative administration, the PAL-I was revised and evaluated in 2 waves of cognitive interviews.

2.9. Second quantitative administration and preliminary psychometric validation

After evaluation in the second round of cognitive interviews, the revised PAL-I instrument was tested using 2 separate U.S.-based cohorts of patients with cLBP. The first sample to complete the web-based survey of the revised instrument was a subset of the participants in the first quantitative administration (401 patients of the original 598 patients). The second sample was a clinic-based cohort of patients with physician-diagnosed cLBP who were recruited to conduct preliminary psychometric validation of the instrument. This cohort comprised 45 patients with cLBP who were identified by patient records. These patients completed (on paper) the PAL-I, painDETECT, MOS-36 (a multi-item scale that assesses 8 health concepts of which we used 2: limitations in physical activities because of health problems and bodily pain, scored on a 0-100-point scale), RMDQ (a 24-item health status checklist designed to assess physical disability due to LBP, scored by summing the number of items checked by the patient with a range of 0-24), and NPSI (a self-administered questionnaire specifically designed to evaluate different symptoms of neuropathic pain with descriptors reflecting spontaneous, ongoing, or paroxysmal pain, evoked pain [ie, mechanical and thermal allodynia/hyperalgesia], and dysesthesia/paresthesia; with each item quantified on a 0-10 numerical scale). Intraclass correlation coefficients were used to evaluate test–retest reproducibility. Pearson correlations were used to assess convergent validity of the PAL-I with the painDETECT, MOS-36, RMDQ, and NPSI instruments. Known-groups validity was assessed by 2 methods: comparison with pain NRS tertiles and painDETECT groups.

2.10. Cognitive interviews to assess paper to electronic equivalence

Cognitive interviews were conducted per ISPOR recommendations (Ref. 7 page 423) to confirm that the intent and meaning of the items, response options, and instructions were unaffected by how the instrument was administered. The interview guide was specifically intended to capture the patient's comprehension of items and ability to complete the PAL-I instrument using a paper or electronic format. Questions in the interview process asked about: the comprehension and relevance of the individual items and how they may have differed between the paper or electronic versions; the fit of the response scales; the language used; and any lack of clarity of items, terminology, instructions, or sentence structure.

3. Results

3.1. Concept elicitation results

A total of 43 patients participated in CE interviews to identify cLPB impacts (Table 1). Saturation of concept was achieved by the end of the second transcript group. Interrater agreement ranged from 82.5% to 86.9% between the coders for the identification of concepts being expressed by patients, and ranged from 97.2% to 99.2% for the assignment of specific concept codes to concepts identified.

Table 1

Demographic and clinical characteristics of patients participating in concept elicitation and cognitive interviews (reproduced from Martin et al.[14]).

Based on the number of times a particular concept was expressed in the overall transcript data set, the predominant impact-related concepts were impacts on coping behaviors, household activities, and work activities (Table 2). The most common spontaneously offered impacts were those on walking (reported by 65.1% of participants), sitting (62.8%), exercise (58.1%), and leisure activities (58.1%). The most difficult impacts were on climbing stairs (mean difficulty rating 9.5 on NRS 0-10), coping behaviors (8.5), standing (8.4), and sports (8.4).

Table 2

Summary of impact concept code frequencies.

3.2. Item generation results

A total of 14 items were identified as appropriate for inclusion in the preliminary version of the PAL-I instrument (Table 3).

Table 3

Low back pain concepts selected for PAL-I instrument.

3.3. Cognitive interviews round 1 results

Thirty patients participated in the first round (4 waves) of cognitive interviews (Table 1). Patient input from these interviews led to changes in the instructions for the instrument (to better highlight the response procedure and recall period) and changes to the wording of some items.

3.4. First quantitative administration results

A total of 598 patients participated in the first quantitative administration of the instrument (Table 4). All PAL-I items were endorsed in the first quantitative administration. Item mean scores ranged from 2.15 (needed assistance with daily activities; SD 1.50) to 5.32 (affected your sexual activity; SD 2.23). High floor effects (participants selecting least impactful option) were seen with 4 items (affected your climbing up or down stairs, caused you to feel sad or down, caused you to feel worried or nervous, and needed assistance with daily activities). No item-to-item correlations were problematic. Item-to-total correlations ranged from 0.29 to 0.61. Cronbach's alpha for the 13 items was 0.81, with alphas remaining at 0.79 to 0.82 if any item was deleted.

Table 4

Demographic and clinical characteristics of patients participating in quantitative analyses (reproduced from Martin et al.[14]).

The RMT analysis showed only 2 items with an ordered threshold (affected your lifting, affected your turning, twisting, and bending); the remaining items were disordered. Disordering seemed to be based on how the response categories were presented: some items were not linear (eg, walking) and some items mixed impacts from the activity with impacts from the cLBP (eg, climbing stairs). These findings indicated that the response options in the preliminary version were not ideal, and the instrument was modified to expand the distributions for better scaling. Four items were removed (caused you to feel sad or down, caused you to feel worried or nervous, affected your sexual activity, and needed assistance with daily activities). Response options for the remaining 9 items were changed to a 4-point rating scale with a fifth option available for not doing the activity for reasons other than cLBP.

3.5. Cognitive interviews round 2 results

Modifications made to the PAL-I based on item-level analyses in the first quantitative administration were evaluated in a second round of cognitive interviews with 8 patients (Table 1). Additional changes were made to the instrument based on the results of these interviews. Instructions were changed from “Mark one box,” to “Select one,” to better facilitate migration from paper to electronic formatting. The order of items was adjusted so that the physical activity–related items were organized by increasing level of exertion necessary, which was followed by items assessing impacts in other areas of daily life. The walking item was altered to specify “during daily activities,” to clarify the intent of the item and alleviate potential confusion among individuals who walk or hike as a form of exercise outside their usual daily activity.

3.6. Quantitative findings from second administration

A second quantitative data collection was conducted with 401 newly recruited subjects from the same commercial sample. All PAL-I items and response options were endorsed. Item mean scores ranged from 0.93 (limited your traveling; SD 0.97) to 1.50 (limited your lifting; SD 0.92) (Table 5). No data were missing. High floor effects were seen with 2 items (limited your travelling and limited your social activities). Only 2 PAL-I items were strongly correlated with each other: limited your walking and limited your standing (r = 0.65). Item-to-total correlations ranged from 0.56 to 0.82. Cronbach's alpha for the 9 items was 0.92, with alphas remaining at 0.91 to 0.92 if any item was deleted. The item threshold map (Fig. 2) showed that all items fit the model with no items exhibiting a disordered threshold.

Table 5

Descriptive characteristics of PAL-I items from second quantitative administration.

Figure 2.

Item threshold map for items within the PAL-I Wave 2 web sample (n = 401).

3.7. Preliminary psychometric validation

A total of 45 patients with physician-diagnosed cLBP participated in the preliminary psychometric validation analyses (Table 4). All PAL-I items and response options were endorsed. Item mean scores ranged from 0.98 (limited your social activities; SD 0.78) to 1.51 (limited your sleep; SD 0.73). Test–retest reproducibility at 1 week was acceptable; the intraclass correlation coefficient was 0.88 (95% confidence interval 0.78-0.94). In convergent validity assessments, strong associations were seen between the PAL-I total score and the RMDQ (Pearson correlation coefficient 0.82), MOS-36 Physical Function (−0.71), and MOS-36 Bodily Pain (−0.71). Known-groups assessments based on pain NRS tertiles showed that the PAL-I total scores were able to significantly discriminate between levels of pain (Table 6).

Table 6

Known-groups validity of the PAL-I: mean item scores (SD) stratified by NRS tertiles.

3.8. Cognitive assessment of paper and electronic equivalence

Eight patients participated in the cognitive assessment of paper and electronic versions of the instrument (Table 1). There was no indication that understanding of the instructions, items, or response options was affected by differences in mode of administration. Some formatting issues to be considered for both paper and electronic versions were noted by the patients, including how close response options were to the question item, how crowded text was to the left margin, and where a sentence broke at the end of a line of text.

4. Discussion

The PAL-I and its companion instrument to measure pain symptoms (PAL-S)[14] were developed in accordance with U.S. FDA guidance for PRO measures to be used for label claims of patient-reported improvements in symptoms and impacts for medications to treat cLBP. The PAL-I demonstrated content validity; ie, the items in the instruments are relevant and representative of the cLBP experience based on input from patients with cLBP. In addition, the PAL-I demonstrated 1-week test–retest reliability, convergent validity with relevant components of existing instruments, and appropriate item-level performance, scale structure, and scoring.

The PAL-I provides a single total impact score on a scale of 0 to 3, with higher scores indicating greater impacts due to cLBP. Patients score each of the 9 items with response options of “Not at all limited,” (score = 0), “Limited a little,” (score = 1), “Limited a lot,” (score = 2), or “Did not do because of my LBP,” (score = 3), or patients can opt-out for an item (“Did not do for other reasons”). The mean score for all scored items (excluding opt-out items) represents the single total impact score.

Although many instruments have been developed to assess pain from the patient perspective, a previous search revealed that none was fully compliant with the FDA guidance and none could be used for label claims.[18] In our review of existing instruments, the most common reason for failing to comply with FDA guidance was lack of patient input. Patient input is particularly important for pain assessments, as there are no measurable signs or laboratory tests to estimate or quantify pain severity, and physicians cannot easily or accurately assess the various dimensions of pain. For example, in emergency departments, where the most common presenting symptom is pain, physicians frequently rate pain as less severe than patients.[4,13] Similarly, considerable nonconcordance between general practitioners' and patients' assessments of pain intensity has been observed for patients with chronic pain (Spearman correlation coefficient 0.20).[12] For the development of the PAL-I and the PAL-S, cohorts of patients with cLBP representing the spectrum of disease severity were recruited to elicit symptom and impact concepts that related specifically to cLBP. Although generic pain instruments are useful for clinical trials of medications to treat conditions associated with pain or to monitor treatment response in clinical practice, the PAL-I and PAL-S are designed for use in clinical trials to treat cLBP. Although not specifically developed for use in clinical practice, both of these measures may useful to assess a patient's status associated with their cLBP and providing helpful information to clinicians considering the next steps of care.

A limitation of this study was the small sample sizes for CE, cognitive interviews, and preliminary validation of the PAL-I. Saturation of concept, which was achieved with the second round of CE, suggested that a sufficient sample size was used to elicit key concepts for the PAL-I. Large sample sizes were used for quantitative administrations, but cLBP diagnosis was self-reported; however, patients with physician-confirmed cLBP participated in preliminary validation analyses. A strength of the study was the recruitment of patients across the spectrum of pain severity and representing both neuropathic and non-neuropathic pain.

Further studies are planned for the PAL-I and PAL-S. Although preliminary measurement properties were derived from the initial quantitative data collection, a formal validation study including sensitivity to treatment-related change has not yet been conducted. Further investigation of the influence of clinical history and potential differences between conditions (a noted limitation of this study) should also be explored. In addition, studies to develop the instruments for other cultures and other languages are planned. The PAL-I and PAL-S have the potential to provide between-study comparisons in clinical trials of cLBP treatments.

Conflict of interest statement

D.M. Bushnell and M.L. Martin are employed by Health Research Associates, which received funding from Forest Research Institute and Grünenthal GmbH for this study. S.I. Blum was an employee of Forest Research Institute and a stock shareholder of Forest Laboratories (now part of Allergan) at the time this research was conducted. S.I. Blum also reports employment and stock ownership in GlaxoSmithKline and Bristol-Myers Squibb and honoraria from the Patient-Centered Outcomes Research Institute (PCORI), which are unrelated to this work. H. Liedgens reports employment by Grünenthal GmbH. R. Freynhagen reports personal fees from Astellas Pharma US, Inc, Grünenthal GmbH, Eli Lilly & Co, Pfizer Inc, Merck & Co, Inc, Develco Pharma Schweiz AG, Mitsubishi Tanabe Pharma, and Galapagos NV outside the submitted work. M. Eerdekens reports income from Grünenthal GmbH outside the submitted work. M. Kok reports income from Grünenthal GmbH during the conduct of the study and outside the submitted work. The remaining authors have no conflicts of interest to disclose.

This study was supported by research funds from both Forest Research Institute and Grünenthal GmbH.