Reliability and validity of the International Spinal Cord Injury Basic Pain Data Set items as self-report measures.

OBJECTIVE: To evaluate the psychometric properties of a subset of International Spinal Cord Injury Basic Pain Data Set (ISCIBPDS) items that could be used as self-report measures in surveys, longitudinal studies and clinical trials.
SETTING: Community.
METHODS: A subset of the ISCIBPDS items and measures of two validity criteria were administered in a postal survey to 184 individuals with spinal cord injury (SCI) and pain. The responses of the participants were evaluated to determine: (1) item response rates (as an estimate of ease of item completion); (2) internal consistency (as an estimate of the reliability of the multiple-item measures); and (3) concurrent validity.
RESULTS: The results support the utility and validity of the ISCIBPDS items and scales that measure pain interference, intensity, site(s), frequency, duration and timing (time of day of worst pain) in individuals with SCI and chronic pain. The results also provide psychometric information that can be used to select from among the ISCIBPDS items in settings that require even fewer items than are in the basic data set.

Introduction

Pain is a significant problem in many individuals with spinal cord injury (SCI).1–2 To better understand the nature of pain in individuals with SCI and the efficacy of treatment, valid and reliable pain measures are needed. There is also a need to standardize pain assessment in SCI research to allow for more direct comparisons between studies.

The International Spinal Cord Injury Basic Pain Data Set (ISCIBPDS) was developed to provide a standardized way of collecting a minimal yet clinically relevant set of measures of up to three pain problems in individuals with SCI across settings and countries.3 The ISCIBPDS is intended to be used by healthcare professionals with expertise in SCI and should be used in conjunction with the International SCI Core Data Set 4 which includes demographic information and neurological status.

Although designed to be administered by a health care professional, self-report versions of the ISCIBDIPS items may be useful in clinical practice, surveys and longitudinal studies, or in treatment outcome studies. In order to support their use, however, their psychometric properties need to be evaluated. The purpose of this study was therefore to evaluate the utility, reliability, and validity of a subset of the ISCIBPDS items that can be used when (1) self-report measures are needed and (2) the clinician or researcher wishes to use standardized measures that can be compared with published findings.

Materials and Methods

Participants

Participants were 184 adults with SCI and pain who completed a postal survey. Participants were paid $25 for returning a completed questionnaire. Study procedures were approved by the University of Washington Institutional Review Board, and informed consent was obtained from each participant. Potential participants came from a pool of individuals from previous studies who had expressed a willingness to be contacted for additional studies. Of 308 consent forms and surveys that were mailed, 203 (66%) were completed and returned. Of these, 184 (91%) were from participants who reported that they experienced persistent pain in the past three months.

The 184 participants were primarily white (90.8%) men (74.5%) who had an average SCI duration of 19.2 years (range, 2–63 years). Average age was 54.4 years (range, 21–87 years) and most reported completing at least a high school education (95.2%). Only 25.6% reported that they were working full or part time. Thirty-eight percent reported complete SCI, and the most common levels of injury were at C5–C7 and T10-L1 (35.9% each).

Measures

ISCIBPDS items

Some of the ISCIBPDS items (e.g., classification of neuropathic versus nociceptive pain) are not appropriate as self-report items. However, the majority of the ISCIBPDS items can be used as self-report measures. Those that required no modification included an item about number of pain problems, a 0–10 Numerical Rating Scale (NRS) of pain intensity, and 2 items about pain frequency and, pain duration for each of up to three pain problems.

Two of the ISCIBPDS items did require modification to make them easier for patients providing information without the supervision of a health care provider. First, in order to simplify the assessment, each of the six pain interference items was asked only once, about pain in general (as opposed to about each of up to three pain problems). Second, the response options for the item asking about pain location(s) was reduced from 50 to eight options (see Appendix).

Validity criteria

Two validity criterion measures were included in the survey – one assessing psychological functioning and one sleep problems. Psychological functioning was assessed using the 5-item Mental Health scale of the SF-36 (SF-36 MH; 5) which has demonstrated validity and high levels of reliability in numerous samples of healthy and chronically ill populations.5 Sleep problems were assessed using the 6-item Medical Outcomes Study Sleep Problem Index (SPI-I), which has demonstrated validity and good reliability in a large normative sample.6

Data analysis

Four sets of analyses were performed to evaluate the psychometric properties of the ISCIBPDS items and scales. First, we computed the frequency of missing responses to each item. Second, we computed internal consistency coefficients using Cronbach's alpha for the six ISCIBPDS interference items, and for two different subsets of these items (the alphas for the item subsets were computed to explore the possibility that reliable scales may be created by fewer items). Coefficients ranging from 0.61–0.80 may be considered to be “moderate”, and 0.81–1.0 “substantial”).7 We considered coefficients in the moderate to substantial range to be acceptable.

Next, we computed Pearson correlation coefficients among the ISCIBPDS interference items and scales and pain intensity ratings, and between all of the ISCIBPDS items and scales and the two validity criterion measures. We hypothesized that if the ISCIBPDS items and scales were valid: (1) the interference measures would be at least moderately (r = .30 or greater) and significantly associated with pain intensity; (2) items assessing pain frequency and pain duration would be significantly associated with both pain interference and average pain intensity; and (3) ISCIBPDS measures of pain interference, pain intensity, pain duration, and pain frequency would have negative associations with psychological functioning and positive associations with the severity of sleep problems. An alpha level of .01 was set for these analyses to control for the risk of Type I errors due to multiple comparisons.

Finally, we evaluated the extent to which information about pain intensity from the worst, second, and third worst pain problem would predict pain interference and the validity criterion measures using three regression analyses. In these analyses, we entered the average pain intensity ratings for the worst, second, and third worst pain problem in steps 1, 2, and 3, respectively. We determined a priori that the importance of obtaining intensity ratings for two pain problems would be supported if the second worst pain intensity rating made a significant contribution to the prediction of the criterion variables when the worst pain intensity rating was controlled, and that the importance of obtaining ratings for three pain problems would be supported if the third worst pain intensity rating made a significant contribution to the prediction of the criterion variables when the worst and second worst pain problem intensity ratings were controlled. P ≤ 0.05 was considered statistically significant for the regression analyses.

Results

Descriptive Statistics

Means and standard deviations of the continuous ISCIBPDS self-report items and scales are presented in Table 1, and the rates of responding to each response category for the ISCIBPDS items are listed in Table 2. Table 2 also indicates how the responses to each item were coded. As can be seen, the means of most of the interference items and scales tended to be near 2.5 (range, 1.97 – 2.60) on the 0 to 6 scales, indicating a moderate amount of average pain interference for the sample. Consistent with previous research, most (79.3%) of the participants reported that they had more than one pain problem. The frequency of missing responses was less than 2% for any item, and the response category with the highest response rate was the “unpredictable” response associated with the question about the time of day that pain is most severe (56.0–66.2%).

Table 1

Descriptive statistics for the non-categorical modified ISCIBPDS self-report items

Item (possible range)	Mean (SD)
Pain interference items (0 – 6)
Limit activities	2.60 (1.82)
Social/recreational: Changed ability	2.19 (1.93)
Family-related: Changed satisfaction	1.97 (1.85)
Activity interference	2.51 (1.82)
Mood interference	2.48 (1.85)
Sleep interference	2.41 (1.83)
Pain interference scales (0 – 6)
Total Interference	2.36 (1.62)
LSF Interference	2.25 (1.72)
AMS Interference	2.46 (1.65)
Questions about the worst pain problem
Average intensity (0 – 10)	5.72 (2.47)
Number of days experienced pain (0–7)	4.85 (2.26)
Questions about the second worst pain problem
Average intensity (0 – 10)	4.53 (2.17)
Number of days experienced pain (0–7)	5.42 (2.06)
Questions about the third worst pain problem
Average intensity (0 – 10)	4.00 (1.97)
Number of days experienced pain (0–7)	5.19 (2.14)

Note: ISCIBPDS = International Spinal Cord Injury Basic Pain Data Set; see Appendix for specific wording of each item. Three pain interference scales can be computed from the six interference items: Total Interference (average of all six items), LSF Interference (average of the Limit activities, Social/recreational changes, and Family-related changes interference items), and AMS Interference (average of the Activity, Mood, and Sleep interference items).

Table 2

Item response level response rates for the ISCIBPDS self-report items

Item	Response Rates (percent)
Items with five response categories
	1	2	3	4	5
Number of pain problems	20.7%	27.2%	27.7%	12.0%	12.5%
	AM	PM	Eve	Night	Unpredictable
When is worst pain most intense*	7.1%	9.2%	25.5%	6.5%	56.0%
When is 2nd worst pain most intense*	7.7%	4.9%	14.1%	7.0%	66.2%
When is 3rd worst pain most intense*	14.3%	11.0%	14.3%	4.4%	60.4%

Participants could select more than one response in this category, so total is > 100%

Note: ISCIBPDS = International Spinal Cord Injury Basic Pain Data Set; see Appendix for specific wording of each item of the self-report version.

Internal consistency

We computed three pain interference scales from the six ISCIBPDS interference items: (1) a 6-item total inference scale (Total Interference); (2) a 3-item composite made up by a subset of the MPI-SCI items8 asking about limits in general activity, changes in recreational and social activity, and changes in satisfaction with family activities (LSF Interference; for Limits in activity and changes in Social and recreational activity and Family related activity); and (3) a 3-item composite made up of items (original to the ISCIBPDS) asking about interference with day-to-day activities, mood, and sleep (AMS Interference; for interference with Activities, Mood, and Sleep). The internal consistency coefficients associated with these three scales were .94, .91, and .89, respectively.

Concurrent Validity

Associations between interference and intensity ratings

The Pearson correlation coefficients between the three 0–10 NRS pain intensity ratings; and the three interference scales and six interference items are presented in Table 3. As can be seen, all of the coefficients were positive, all but one were statistically significant at p < .01, and all but one were greater than .30.

Table 3

Pearson correlation coefficients between the three ISCIBPDS 0–10 NRS pain intensity ratings and the pain interference items and scales.

Pain interference items and scales	0–10 NRS pain intensity rating for…
	Worst pain	2nd worst pain	3rd worst pain
Interference items
Limit activities	.53**	.53**	.38**
Social/recreational:
Changed ability	.49**	.40**	.31*
Family related:
Changed satisfaction	.45**	.47**	.25
Day-to-day activity interference	.56**	.53**	.46**
Mood interference	.54**	.52**	.44**
Sleep interference	.42**	.37**	.35*
Interference scales*
Total Interference	.57**	.53**	.43**
LSF Interference	.53**	.50**	.35*
AMS Interference	.56**	.52**	.47**

p <.01

p <.001.

Note: ISCIBPDS = International Spinal Cord Injury Basic Pain Data Set; Three pain interference scales can be computed from the six interference items: Total Interference (average of all six items), LSF Interference (average of the Limit activities, Social/recreational changes, and Family-related changes interference items), and AMS Interference (average of the Activity, Mood, and Sleep interference items).

Associations between reported pain frequency and duration and interference

The correlation coefficients between (1) the frequency of the worst and second worst pain problem and the duration of the worst and second worst pain problem and (2) the Total Interference score were all positive. However, these coefficients were weak (rs range = .13 to .24), and only two of the four were statistically significant (p < .01). The correlation coefficients between (1) reported pain frequency and pain duration of the third worst pain problem and (2) the Total Interference score were very weak and negative (rs = −.07 and −.01, respectively).

Associations between interference, intensity, frequency, duration, and the validity criteria measures

The correlation coefficients between (1) the ISCIBPDS interference scales, pain intensity ratings, pain frequency items, and pain duration items and (2) the two validity criterion measures are presented in Table 4. All three interference scales demonstrated strong associations with both validity criteria in the expected directions, although the 3-item AMS Interference scale was more strongly associated with the criterion measures than was the 3-item LSF Interference scale. Responses to the intensity, days of pain, and pain duration items for the worst and second worst pain problems showed the hypothesized patterns of associations with the validity criterion; all four coefficients associated with these intensity items and two of the four coefficients associated with the days of pain items were statistically significant (p < .01). However, although the pattern of associations for third worst pain problem and the criteria variables were in the expected directions, only one was statistically significant (p < .01). Moreover, neither the days of pain nor duration of pain items for the third worst pain showed the expected pattern of associations with the criteria variables.

Table 4

Pearson correlation coefficients between the two validity criteria (SF-36 MH scale and SPI-I) and the ISCIBPDS scales and ratings.

	Validity criteria
ISCIBPDS items and scales	Psychological functioning (SF-36 MH)	Sleep problems (SPI-I)
Interference items
Limit activities	−.39**	.45**
Changed ability	−.40**	.40**
Changed satisfaction	−.45**	.48**
Day-to-day activity interference	−.46**	.51**
Mood interference	−.61**	.59**
Sleep interference	−.57**	.74**
Interference scales*
Total Interference	−.55**	.60**
LSF Interference	−.45**	.48**
AMS Interference	−.60**	.68**
Worst pain
Intensity	−.23*	.38**
Days of pain	−.19*	.23*
Duration of pain	−.09	.06
2nd worst pain
Intensity	−.25*	.35**
Days of pain	−.18	.15
Duration of pain	−.09	.07
3rd worst pain
Intensity	−.25	.33*
Days of pain	.14	−.16
Duration of pain	.15	−.06

p <.01

p <.001.

Note: SF-36 MH= SF-36 Mental Health scale; SPI-I = MOS Sleep Problem Index-I; ISCIBPDS = International Spinal Cord Injury Basic Pain Data Set; Three pain interference scales can be computed from the six interference items: Total Interference (average of all six items), LSF Interference (average of the Limit activities, Social/recreational changes, and Family-related changes interference items), and AMS Interference (average of the Activity, Mood, and Sleep interference items).

Regression analyses predicting interference and criterion variables from worst, second, and third worst intensity ratings

The results of the three regression analyses predicting the Total Interference score and the two validity criterion measures are presented in Table 5. As a group, the pain intensity ratings of the worst, second, and third worst pain problems were significantly associated with pain interference (F(3,85) = 11.35, p < .001), and the worst and second worst pain intensity ratings both made significant contributions to this criterion. The three pain intensity ratings were also significantly associated with sleep problems as a group (F(3,85) = 4.02, p = .01), with the worst and second worst intensity ratings showing a significant association with sleep problems. As a group, the three pain intensity ratings were not significantly associated with psychological functioning. Interestingly, however, the intensity rating of the third worst pain problem was a significant, and unique, predictor of psychological functioning after the worst and second worst pain intensity ratings were entered into the equation.

Table 5

Regression analyses model predicting pain interfernce (ISCIBPDS Total interference score), psychological functioning (SF-36 Mental Health scale) and sleep problems (SPI-I) from the worst, second worst, and third worst pain intensity ratings (N = 88)

Step and predictor variables	Total R2	R2 Δ	F (R2 Δ)	Beta to enter
Pain interference (ISCIBPDS Total Interference)
1: Worst pain intensity	.21	.21	23.52***	.46***
2: 2nd worst pain intensity	.27	.05	6.12*	.28*
3: 3rd worst pain intensity	.29	.02	2.49	.19
Sleep problems (SPI-I)
1: Worst pain intensity	.05	.05	4.56*	.22*
2: 2nd worst pain intensity	.09	.04	4.08*	.26*
3: 3rd worst pain intensity	.12	.03	3.04†	.23†
Psychological functioning (SF-36 Mental Health)
1: Worst pain intensity	.00	.00	0.37	−.07
2: 2nd worst pain intensity	.02	.02	1.70	−.17
3: 3rd worst pain intensity	.07	.05	4.13*	−.38*

p < .10

p < .05

p < .01

p < .001

Note: ISCIBPDS Total interference = Composite score of the six interference items from the International Spinal Cord Injury Basic Pain Data Set Items; SF-36 = SF-36 Mental Health scale, higher scores indicates better psychological functioning; SPI-I = MOS Sleep Problem Index-I, a 6-item short-form version of the MOS Sleep Scale.

Discussion

The current findings support the concurrent validity of the self-report items and scales of the ISCIBPDS. There were very few missing responses to any of the items, suggesting that the items were understandable enough to elicit a response. Thus, the ISCIBPDS items and scales may prove very useful as self-report measures of key pain-related domains in clinical practice and research.

The Total Interference scale demonstrated excellent internal consistency (Cronbach's α = .94) suggesting the possibility that all six items may not be required to assess the pain interference domain. Moreover, the validity of the interference items and scales was supported by their strong associations with the pain intensity ratings and with the validity criterion. Some differences in the validity coefficients were found, however, with the AMS Interference scale showing stronger associations with the validity criteria than the LSF Interference scale. This finding may be related to the wording of the items, given that the AMS items ask specifically about interference with mood and sleep, while the LSF items do not specifically ask about these domains. These results suggest that, if a shorter interference scale is ultimately adopted for standard use, the 3-item AMS Interference scale may prove most useful. However, more research in additional samples would be useful before any revisions of the ISCIBPDS items are performed.

The validity of the worst, second, and third worst pain intensity ratings was strongly supported, with moderate to strong associations found between these items and the pain interference items and scales and validity criteria. This finding is consistent with the strong support for the validity of numerical scales of pain intensity in many studies and populations.9 Strong support was also found for asking patients to report on the intensity of at least their worst and second worst pain problems, as each of these made statistically significant and independent contributions to the prediction of the validity criteria. Limited support was also found for assessing the intensity of the third pain problem given the trend for this rating to contribute to the prediction of sleep problems, and its significant and independent contribution to the prediction of psychological functioning.

There was somewhat less support for the validity of the two pain duration items (days of pain and duration of pain incidents). Given our clinical observation that the nature and amount of pain duration differs between musculoskeletal and neuropathic pain, it is possible that the associations between the duration and criterion variables may have been attenuated because that our sample contained individuals with both neuropathic and musculoskeletal pain. Nevertheless, the findings suggest that in survey studies of heterogeneous patients, the duration items may not be as important (at least as predictors of patient functioning) as the intensity items, although they clearly are necessary for diagnostic purposes.

The limitations of this study include the fact that it is based on a single sample that was self-selected (i.e., were willing to complete surveys). Also, some of the original ISCIBPDS items were modified (simplified), and the sequence of items was altered (the interference items were administered first in this self-report version). Although we do not anticipate that these changes altered the psychometric properties of the items to a large extent, their effect on the reliability and validity of the items is not known. For these reasons, replication of the findings in additional samples is needed to determine their generalizability. Additional limitations include the lack of diagnostic information regarding type of pain (e.g., neuropathic vs nociceptive) problems in the sample, as well as the fact that the validity criteria used were limited to measures of psychological functioning and sleep problems. It would be useful to determine the extent to which the ISCIBPDS items are associated with other validity criteria, and also to determine the moderating effects, if any, that pain type has on the validity of the ISCIBPDS items and scales.

Despite the study's limitations, the findings support the utility and validity of the self-report version of the ISCIBPDS items for assessing pain in individuals with SCI in clinical practice and research settings. The items assess domains that are important to assess in clinical practice, including pain interference, location, intensity, frequency, and duration. Interference and intensity must be measured if clinicians wish to monitor the effects of treatments over time. Pain location, frequency, and duration information is necessary for diagnosis. Although the self-report version of ISCIBPDS is quite brief, clinicians could make it even more practical by assessing the treatment outcome domains (interference and intensity) at each assessment, and the other domains useful for diagnosis less often.

The brevity of the self-report version of the ISCIPBPDS also indicates that it may be particularly useful in research settings where assessment burden is an issue. This includes survey studies that might include additional questionnaires or measures, or longitudinal studies where a minimal number of items are needed to help minimize subject attrition. Researchers may choose to administer the location, frequency, and duration domains only once (for descriptive purposes), but administer the interference and intensity items over time in clinical trials and longitudinal survey studies. Of particular importance, the adoption and use of the ISCIPBDS items by researchers would also enhance the ability to compare findings across studies and populations.