ODI Cannot Account for All Variation in PROMIS Scores in Patients With Thoracolumbar Disorders.

STUDY
DESIGN: Retrospective review of single institution.
OBJECTIVE: To assess the relationship between Patient-Reported Outcomes Measurement Information System (PROMIS) and Oswestry Disability Index (ODI) scores in thoracolumbar patients.
METHODS: Included: Patients ≥18 years with a thoracolumbar spine condition (spinal stenosis, disc herniation, low back pain, disc degeneration, spondylolysis). Bivariate correlations assessed the linear relationships between ODI and PROMIS (Physical Function, Pain Intensity, and Pain Interference). Correlation cutoffs assessed patients with high and low correlation between ODI and PROMIS. Linear regression predicted the relationship of ODI to PROMIS.
RESULTS: A total of 206 patients (age 53.7 ± 16.6 years, 49.5% female) were included. ODI correlated with PROMIS Physical Function (r = -0.763, P < .001), Pain Interference (r = 0.800, P < .001), and Pain Intensity (r = 0.706, P < .001). ODI strongly predicted PROMIS for Physical Function (R 2 = 0.58, P < .001), Pain Intensity (R 2 = 0.50, P < .001), and Pain Interference (R 2 = 0.64, P < .001); however, there is variability in PROMIS that ODI cannot account for. ODI questions about sitting and sleeping were weakly correlated across the 3 PROMIS domains. Linear regression showed overall ODI score as accounting for 58.3% (R 2 = 0.583) of the variance in PROMIS Physical Function, 63.9% (R 2 = 0.639) of the variance in Pain Interference score, and 49.9% (R 2 = 0.499) of the variance in Pain Intensity score.
CONCLUSIONS: There is a large amount of variability with PROMIS that cannot be accounted for with ODI. ODI questions regarding walking, social life, and lifting ability correlate strongly with PROMIS while sitting, standing, and sleeping do not. These results reinforce the utility of PROMIS as a valid assessment for low back disability, while indicating the need for further evaluation of the factors responsible for variation between PROMIS and ODI.

Introduction

Quality health care, especially as it relates to the treatment of spinal disorders, requires an accurate evaluation of patient pain and disability. Critical to the assessment of clinical outcomes following spine surgery are patient-reported outcome measures, typically in the form of questionnaires that evaluate symptoms and functionality via a numeric scoring system. For patients undergoing thoracolumbar spine surgery, one of the most commonly used patient-reported outcome measures is the Oswestry Disability Index (ODI), a validated, 10-section questionnaire that quantifies disability secondary to low back pain on a scale from 0 to 100.[1-3] The ODI has been cited as the “gold standard” outcome assessment tool for surgical thoracolumbar patients; however, prior studies have identified a number of key shortcomings, including lack of unidimensionality for component ODI sections and poor stratification of high-functioning patients.[4-7]

In an effort to address the shortcomings of legacy patient-reported outcome measures like the ODI, the National Institutes of Health developed the Patient-Reported Outcomes Measurement Information System (PROMIS) instruments of Physical Function, Pain Intensity, and Pain Interference.[8-10] These PROMIS instruments can be administered as computer adaptive tests, with each successive question algorithmically selected based on answers to previous items.[8] As a result, PROMIS has the potential to provide patients and health professionals a precise assessment of patient-reported health-related quality of life (HRQL) with lower administrative burden than legacy outcome measures.[11]

In surgical cervical spine populations, the PROMIS domains of pain and physical function have proven to be valid and responsive measures of both symptoms and quality of life, showing significant correlations with legacy outcome measures like the Neck Disability Index and the Short-Form Health Survey.[12] Recently, the PROMIS Physical Function domain was also shown to provide superior coverage and dimensionality as compared to the ODI in a population of spine patients presenting with low back pain, although it remains to be seen whether this relationship persists in patient populations with specific thoracolumbar disorders.[7] In this context, we sought to assess the relationship between ODI scores and the PROMIS instruments of Physical Function, Pain Intensity, and Pain Interference in a population of patients with thoracolumbar conditions.

Materials and Methods

Data Source and Inclusion Criteria

This study was a retrospective review of consecutive, patient-reported HRQL outcomes collected at a single urban tertiary hospital from December 2016 to July 2017. Patients included in the present analysis had thoracolumbar spine diagnoses, ODI scores recorded at baseline, and were >18 years of age. Institutional review board approval was obtained prior to study initiation.

Data Collection and Outcome Assessments

Demographic data, including age and sex, was collected for all patients at the time of initial presentation. Outcome assessments were collected at baseline and included the following HRQL questionnaires: the PROMIS instruments of Pain Intensity, Pain Interference, and Physical Function and the ODI. All HRQL assessment tools were administered to patients via tablet.

The ODI questionnaire is one of the most commonly used condition-specific outcome measures for spine patients, assessing low back pain and disability through a series of 60 questions.[13] ODI scores range from 0 to 100, with higher scores corresponding to greater disability.

The PROMIS instruments of Pain Intensity, Pain Interference, and Physical Function assess patient-reported capability utilizing a computer adaptive algorithm. This computer adaptive testing selects subsequent questionnaire items based on previous item answers; for spine patients, previous research shows an average of 4.15 question items administered per PROMIS domain.[7] PROMIS scores similarly range from 0 to 100. Higher Physical Function scores indicate superior function, while higher Pain Intensity and Pain Interference scores indicate inferior outcomes.

Statistical Analysis

Linear relationships between baseline PROMIS scores (Pain Intensity, Pain Interference, Physical Function) and baseline ODI scores were assessed using Pearson bivariate correlation. Linear regression was used to assess the relationship between PROMIS outcomes and ODI score, effectively determining the extent of variation in PROMIS score explained by the ODI score. Forward stepwise multivariable linear regression was used to assess component ODI items most strongly associated with PROMIS outcomes. All analyses were conducted using SPSS software for Windows (v.23, IBM, Armonk, NY), and statistical significance was set to P < .05.

Results

Cohort Overview

Overall, 206 patients met inclusion criteria and were included in the analysis. The cohort was comprised of 49.5% females, and mean age was 53.7 ± 16.6 years. Among the most common diagnoses were low back pain (65.3%), thoracolumbar degenerative disc disease (49.5%), radiculopathy (46.7%), degenerative spondylolisthesis (36.7%), and scoliosis (18.1%). Patients presented with a mean ODI score of 42.7 ± 20.7, PROMIS Pain Intensity score of 54.7 ± 7.3, Pain Interference score of 64.7 ± 7.4, and Physical Function score of 36.8 ± 8.5.

Correlating ODI and PROMIS Domains

Overall ODI score showed strong correlations with the PROMIS measures of Pain Intensity (r = 0.706, P < .001), Pain Interference (r = 0.800, P < .001), and Physical Function (r = −0.763, P < .001). All component domains of the ODI questionnaire also correlated with PROMIS Pain Intensity, Pain Interference, and Physical Function; however, there was substantial variability in the strength of these relationships across component ODI domains (Table 1). Specifically, the component ODI scores correlating best with PROMIS Physical Function score were walking and social life; while the weakest were sitting and sleeping (Figure 1). The ODI domains of pain intensity and social life were correlated strongest with PROMIS Pain Intensity, while sex life and lifting ability correlated the weakest (Figure 2). PROMIS Pain Interference correlated the strongest with social life and traveling and was weakly correlated with sex life and sleeping (Figure 3). Figure 4 shows a case example of a 70-year-old patient with degenerative spondylolisthesis and severe lumbar stenosis. The patient presented with a Physical Function score of 24.7, Pain Intensity score of 68.3, Pain Interference score of 71.6, and overall ODI score of 78.

Table 1.

Pearson Bivariate Correlation Results Describing the Relationship Between PROMIS Domains and Individual Items of the ODI.a

Patient-Reported Outcome Domains		PROMIS
		Physical Function r	Pain Intensity r	Pain Interference r
ODI	Overall	−0.763	0.706	0.800
	Pain Intensity	−0.515	0.672	0.613
	Personal Care	−0.587	0.470	0.619
	Lifting Ability	−0.614	0.434	0.517
	Walking	−0.712	0.557	0.650
	Sitting	−0.369	0.473	0.520
	Standing	−0.577	0.482	0.535
	Sleeping	−0.439	0.537	0.508
	Sex Life	−0.464	0.383	0.472
	Social Life	−0.673	0.636	0.751
	Traveling	−0.612	0.592	0.681

Abbreviations: ODI, Oswestry Disability Index; PROMIS, Patient-Reported Outcomes Measurement Information System.

aAll correlations were significant at P < .001.

Figure 1.

Regression scatterplots showing component ODI items that correlate best (left; ODI Walking domain) and worst (right; ODI Sitting domain) with PROMIS Physical Function outcomes. ODI, Oswestry Disability Index; PROMIS, Patient-Reported Outcomes Measurement Information System.

Figure 2.

Regression scatterplots showing component ODI items that correlate best (left; ODI Pain Intensity domain) and worst (right; ODI Sex Life domain) with PROMIS Pain Intensity outcomes. ODI, Oswestry Disability Index; PROMIS, Patient-Reported Outcomes Measurement Information System.

Figure 3.

Regression scatterplots showing component ODI items that correlate best (left; ODI Social Life domain) and worst (right; ODI Sex Life domain) with PROMIS Pain Interference outcomes. ODI, Oswestry Disability Index; PROMIS, Patient-Reported Outcomes Measurement Information System.

Figure 4.

Baseline radiograph case example of a 70-year-old patient with degenerative spondylolisthesis and severe lumbar stenosis. The patient presented with a Physical Function score of 24.7, Pain Intensity score of 68.3, Pain Interference score of 71.6, and overall ODI score of 78. ODI, Oswestry Disability Index; PROMIS, Patient-Reported Outcomes Measurement Information System.

Regression Analysis

ODI was found to be a significant predictor for PROMIS Pain Interference, Pain Intensity, and Physical Function scores (Figure 5, all P < .001). Overall ODI score accounted for 63.9% of the variance in Pain Interference score, 49.9% of the variance in Pain Intensity score, and 58.3% of the variance in Physical Function score.

Figure 5.

Probability plots showing normality in the linear regressions assessing the relationship between ODI score and (A) PROMIS Physical Function score, (B), PROMIS Pain Interference score, and (C) PROMIS Pain Intensity score. ODI, Oswestry Disability Index; PROMIS, Patient-Reported Outcomes Measurement Information System.

Forward stepwise regression analysis showed the component ODI factors of pain intensity, social life, sleeping, and walking to be responsible for 57.9% of variability in PROMIS Pain Intensity score. Social life, walking, pain intensity, and personal care ODI domains were responsible for 66.1% of variability in PROMIS Pain Intensity score; walking, social life, and lifting ability were responsible for 64.5% of variability in PROMIS Physical Function score.

Discussion

The PROMIS instruments of Physical Function, Pain Intensity, and Pain Interference are increasingly used in clinical practice as responsive HRQL assessment tools for patients undergoing spine surgery.[10,14] Specifically, in populations of patients with thoracolumbar disorders, PROMIS scores have shown moderate to strong correlations with ODI scores, suggesting that PROMIS assessments offer adequate concurrent validity in assessing low back pain and disability.[15,16] Despite these correlations between overall ODI and PROMIS outcomes, few studies have investigated the relationship between PROMIS scores and component items of the ODI. As such, this study aimed to assess the relationship between overall ODI score, individual domains of the ODI, and PROMIS outcome instruments in a population of patients with thoracolumbar conditions.

The present analysis shows strong correlations between overall ODI score and the PROMIS instruments of Physical Function (r = −0.763), Pain Intensity (r = 0.706), and Pain Interference (r = 0.800). This finding is particularly important, as the bulk of PROMIS validity assessments for spine patients are rooted in comparisons to ODI.[14,16-19] While the literature is still developing in assessing the concurrent validity of the PROMIS domains of Pain Intensity and Pain Interference in populations with thoracolumbar disorders, recent research by Papuga et al[4] similarly shows a strong correlation between PROMIS Physical Function and ODI score (R2 = 0.683) for patients presenting for spine surgical evaluation. In a population of patients undergoing surgical treatment for lumbar spinal stenosis, Patel et al[19] also showed multiple correlations between ODI and the PROMIS domains of Physical Function (r = −0.58), Pain Intensity (r = 0.73), and Pain Behavior (r = 0.60), further demonstrating the convergent validity of PROMIS assessments. Our analysis adds to the growing body of literature describing the relationship between the “gold standard” legacy assessment of low back disability and the “new standard” of PROMIS.

These results are important, especially given the ODI’s limitations as an outcome assessment tool. Although the ODI has been widely used as a reliable assessment of low back disability for patients with thoracolumbar conditions, a recent systematic review by Chiarotto et al[20] demonstrated high-quality evidence suggesting problematic unidimensionality for the ODI. Additional research shows that compared to the PROMIS Physical Function tool, the ODI takes more time to administer, has more questions to answer, and has greater ceiling and floor effects, indicating worse coverage for patients at either end of the low back disability spectrum.[7] Similar floor and ceiling effects were encountered in this work and reinforce the notion that certain components of the ODI lack sufficient discriminative capacity among patients at the extreme ends of the spectrum of symptomatic thoracolumbar disorders. As out results show PROMIS enjoys satisfactory concurrent validity with the ODI, a holistic assessment supports PROMIS’ use as a superior outcome metric for patients with thoracolumbar disorders.

In addition to showing significant correlations between overall ODI and PROMIS scores, our results demonstrate that ODI only accounts for a portion of the variability in PROMIS outcomes. The linear regression analysis demonstrated that the overall ODI score accounted for only 58.3% (R2 = 0.583) of the variance in PROMIS Physical Function, 63.9% (R2 = 0.639) of the variance in Pain Interference score, and 49.9% (R2 = 0.499) of the variance in Pain Intensity. Multivariable regression further showed the component ODI factors of walking and social life to account for a substantial amount of the variation across all 3 PROMIS domains assessed in this study. These results suggest that, although significant correlations exist between ODI and PROMIS outcomes, relatively few component ODI items may be responsible for the observed relationships. These results may also suggest that the ODI is less comprehensive than PROMIS at measuring multiple dimensions of back pain and disability. However, further research is needed to investigate the comparative comprehensiveness of PROMIS and ODI, as well as the specific differences between these outcomes instruments that culminate in the relatively high levels of variation encountered here.

A key limitation of this analysis is the single-institution study design, which likely affects clinical heterogeneity of patients included in our analysis. PROMIS is a relatively new technology administered via tablet, and as such, resource constraints limited the PROMIS and ODI data collection in this analysis to 8 attending surgeons. This certainly introduces selection bias into our study design; however, the multiple diagnoses and surgeons included in our analysis lends our sample sufficient generalizability. As there was no standardized order of administration for the PROMIS and ODI assessments, this analysis could not control for factors like respondent fatigue, which can contribute to increased variability in the relationship between PROMIS and ODI scores. While the goal of this study was to assess the relationship between ODI and PROMIS in a series of patients with thoracolumbar spinal conditions, the diagnoses among included patients were highly heterogeneous and not necessarily representative of the full scope of thoracolumbar spinal disorders. The relationship between PROMIS and ODI warrants further investigation with more specific diagnostic criteria. Despite these shortcomings, this study is unique in that it is among the first to investigate the relationship between component ODI items and PROMIS measures, thus providing further insight into the relationship between 2 widely used HRQL utilities.

Conclusion

Despite moderate levels of correlation between overall ODI score and the PROMIS assessments of Physical Function, Pain Intensity, and Pain Interference, ODI score was only found to account for a portion of the observed variability in PROMIS outcomes among patients with thoracolumbar conditions. While a portion of this variability may be attributable to heterogeneity in thoracolumbar diagnoses, these results suggest that PROMIS, administered as a computer-adaptive test, may be more comprehensive at measuring multiple dimensions of back pain and disability than the ODI. The results of this study reinforce the utility of PROMIS as a valid outcome assessment for low back disability, while highlighting the need for further evaluation of the factors responsible for the observed variation between PROMIS and ODI scores.