Patient-Reported Outcomes Measurement Information System Test Is Less Responsive Than Legacy Hip-Specific Patient-Reported Outcome Measures in Patients Undergoing Arthroscopy for Femoroacetabular Impingement Syndrome.

PURPOSE: To evaluate and compare the Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function (PF) computerized adaptive test and PROMIS Pain Interference (PI) instruments versus legacy patient-reported outcome measures (PROMs) in patients undergoing hip arthroscopy for femoroacetabular impingement syndrome at 1-year follow-up.
METHODS: Patients undergoing primary hip arthroscopy for the treatment of femoroacetabular impingement syndrome without concomitant procedures performed by a single surgeon between August 2018 and January 2019 were identified. The PROMIS PF score, PROMIS PI score, Hip Outcome Score-Activities of Daily Living Subscale (HOS-ADL), Hip Outcome Score-Sports Subscale (HOS-SS), modified Harris Hip Score (mHHS), International Hip Outcome Tool 12 (iHOT-12), and visual analog scale (VAS) pain score were obtained preoperatively and at 6 months and 1 year postoperatively. Bivariate correlation analyses between PROMIS and legacy PROMs were performed. The floor and ceiling effects, responsiveness, and relative efficiency (RE) of each PROM were calculated.
RESULTS: This study included 124 patients with an average age of 32.7 ± 12.3 years. The PROMIS PF score showed very good correlations with the HOS-ADL (r = 0.73, P < .001) and mHHS (r = 0.70, P < .001) and good correlations with the HOS-SS (r = 0.62, P < .001), iHOT-12 score (r = 0.62, P < .001), and VAS pain score (r = -0.64, P < .001). The PROMIS PI score showed very good inverse correlations with the HOS-ADL (r = -0.72, P < .001) and mHHS (r = -0.79, P < .001) and good correlations with the HOS-SS (r = -0.64, P < .001), iHOT-12 score (r = -0.65, P < .001), and VAS pain score (r = 0.65, P < .001). No floor effect was observed for any measure. Ceiling effects were not observed in the PROMIS PROMs but were detected for the HOS-ADL (16.1%) and mHHS (19.3%). The effect size was large for all outcomes: iHOT-12 score, d = 1.77; HOS-ADL, d = 1.37; HOS-SS, d = 1.45; PROMIS PI score, d = 1.05; and PROMIS PF score, d = 1.01. The iHOT-12 score was more responsive than the PROMIS PI score (RE = 1.69), PROMIS PF score (RE = 1.75), HOS-ADL (RE = 1.29), and HOS-SS (RE = 1.22).
CONCLUSIONS: At 1-year follow-up, PROMIS PROMs displayed very good correlations with legacy PROMs. However, PROMIS PROMs had lower effect sizes and were not as responsive as legacy PROMs. LEVEL OF EVIDENCE: Level II, development of diagnostic criteria on the basis of consecutive patients.

Since its development in 2004 by the National Institutes of Health, the Patient-Reported Outcomes Measurement Information System (PROMIS) tool has been used increasingly in the orthopaedic literature. The PROMIS tool functions with integrated item response theory with computerized adaptive testing (CAT) to provide a single, generalizable and validated patient-reported outcome measure (PROM) while attempting to negate floor or ceiling effects with fewer individual questions when compared with legacy PROMs.1, 2, 3, 4, 5 A number of studies focusing on a variety of knee and shoulder injuries and surgical procedures have compared PROMIS measures with legacy PROMs in the past, showing good to excellent correlations with legacy measures.6, 7, 8, 9, 10 However, there is a relative paucity of literature comparing PROMIS measures with legacy PROMs in patients who have undergone hip arthroscopy for femoroacetabular impingement syndrome (FAIS), with past studies showing good to excellent correlations preoperatively and at 6-month follow-up but decreased responsiveness of the PROMIS Physical Function (PF) instrument at 6-month follow-up.11, 12, 13, 14, 15

Hip arthroscopy is an established procedure for FAIS with demonstrable improvement in PROMs such as the modified Harris Hip Score (mHHS), Hip Outcome Score (HOS), and International Hip Outcome Tool 12 (iHOT-12) score. Although these hip-specific PROMs have been examined to understand their responsiveness and representative clinically significant changes, no studies have compared these legacy measures with PROMIS measures at 1 year postoperatively. Although the PROMIS tool does not have hip-specific questions, patients with hip and other lower-extremity conditions are defaulted to the PROMIS PF instrument. Little is known about whether this delineation is appropriate. The PROMIS PF CAT consists of an item bank of 165 questions, and patients complete between 4 and 12 questions, with the patients’ responses guiding subsequent questions. The PROMIS Pain Interference (PI) instrument measures the consequences of pain and may include the extent to which pain hinders engagement with social, cognitive, emotional, physical, and recreational activities. Similarly to the PROMIS PF CAT, the PROMIS PI CAT consists of an item bank of 40 questions and patients complete between 4 and 12 questions. The PROMIS PI instrument also incorporates items regarding sleep and overall enjoyment in life. The PI form is universal rather than injury specific.

The purpose of the study was to evaluate and compare the PROMIS PF CAT and PROMIS PI instruments versus legacy PROMs in patients undergoing hip arthroscopy for FAIS at 1-year follow-up. We hypothesized that legacy and PROMIS PROMs would have good to excellent correlations and that legacy PROMs would be more responsive with greater relative efficiencies (REs) when compared with PROMIS measures at 1-year follow-up for patients undergoing hip arthroscopy for the treatment of FAIS.

Methods

Patient Selection

After approval from our institutional review board, data were prospectively collected on all patients undergoing hip arthroscopy for the treatment of FAIS performed by a single, fellowship-trained and board-certified orthopaedic surgeon (S.J.N.). Data in a clinical repository were retrospectively analyzed for patients undergoing primary hip arthroscopy for the treatment of FAIS between August 2018 and January 2019. The inclusion criteria consisted of a clinical and radiographic diagnosis of FAIS, failure of conservative management (including physical therapy, activity modification, oral anti-inflammatories, and/or intra-articular cortisone injections), and surgical treatment with hip arthroscopy for FAIS. Only patients completing at least 1 outcome measure preoperatively and at 1 year postoperatively were included. The exclusion criteria consisted of a history of bilateral hip surgery, hip arthroscopy for an indication other than FAIS, signs of osteoarthritis (Tönnis grade > 1), hip dysplasia (lateral center-edge angle < 20°), a history of pediatric hip disorders (slipped capital femoral epiphysis, hip dysplasia, Perthes disease, and so on), concomitant trochanteric bursectomy, and concomitant gluteus medius or minimus repair.

Functional Outcome Evaluation

Preoperatively, demographic data were collected from all patients, including sex, age, operative extremity, body mass index, and comorbidities. Preoperatively and at 6 months and 1 year postoperatively, Patients completed legacy hip-specific PROMs including the HOS–Activities of Daily Living Subscale (HOS-ADL), HOS–Sports Subscale (HOS-SS), mHHS, and iHOT-12 score. Patients also completed the CAT version of the PROMIS PF and PROMIS PI instruments via the Outcome Based Electronic Research Database (OBERD; Universal Research Solutions, Columbia, MO).

Statistical Analysis

Analysis was performed to identify any existing floor and ceiling effects for the legacy hip-specific PROMs, PROMIS PF measure, and PROMIS PI measure. For the legacy PROMs, any percentage of patients greater than or equal to 15% of the study population achieving the maximum or minimum score was deemed a significant ceiling or floor effect. For the PROMIS measures, any percentage greater than or equal to 15% of the study population in the top or bottom fifth percentile was deemed a significant ceiling or floor effect. Pearson coefficient analysis was used to identify correlations between PROMIS scores and the legacy PROMs. Correlation was defined as follows: excellent, greater than 0.80; very good, 0.71 to 0.80; good, 0.61 to 0.70; fair, 0.41 to 0.60; or poor, 0.21 to 0.40.

To directly compare the responsiveness between PROMs, the effect size and RE were calculated for each PROM tool., The effect size, or Cohen d, is a measure of the magnitude of the preoperative-to-postoperative change in relation to the amount of variability in the scores. The Cohen d was calculated by dividing the absolute difference in the mean change score for each PROM by the pooled standard deviation for that PROM tool. The RE was calculated to directly compare responsiveness between PROM tools., The RE is the t score—the ratio between the difference between the 2 groups and the difference within the groups derived from the paired t test of the respective preoperative and 1-year PROM scores—from 1 PROM tool divided by the t score of another PROM tool and then squared. If the RE was greater than 1, the first PROM would be considered “more responsive” than the second PROM tool; if it was less than 1, the first PROM tool would be considered “less responsive” than the second PROM tool. These measures allow one to compare the psychometric properties of the PROMs and determine whether they are suitable for detecting changes in function in the postoperative period.

Descriptive statistics for all continuous variables are reported as means with standard deviations, and frequency statistics are reported for all non-continuous variables. Paired-samples t tests were used to compare preoperative and 1-year postoperative PROMs. Statistical significance for all analyses was set at α ≤ .05. All statistical analyses were performed using SPSS software (version 26; IBM, Armonk, NY).

Results

Demographic Characteristics

A total of 222 patients underwent primary hip arthroscopy between August 2018 and January 2019. Of these patients, 9 underwent hip arthroscopy for an indication other than FAIS, 18 underwent prior ipsilateral or contralateral hip arthroscopy, 3 had a history of pediatric hip disorders or dysplasia, and 38 were missing preoperative PROMIS PROMs and were excluded. A total of 124 patients completed at least 1 preoperative PROM and at least 1 PROM at a minimum of 1-year follow-up and were included in the analysis; the average follow-up period was 13.7 ± 2.1 months. Female patients (73.4%) comprised the majority of this cohort; the average age and body mass index were 32.7 ± 12.3 years and 26.2 ± 6.9 kg/m2, respectively.

Clinical Outcomes Analysis

There were statistically significant improvements in all PROMs at 1-year follow-up: PROMIS PF (40.4 ± 6.5 vs 51.2 ± 10.9, P < .001), HOS-ADL (62.4 ± 17.0 vs 88.4 ± 13.1, P < .0001), HOS-SS (40.0 ± 20.7 vs 79.6 ± 22.9, P < .0001), mHHS (60.2 ± 15.0 vs 85.0 ± 15.8, P < .0001), and iHOT-12 score (31.9 ± 18.0 vs 62.3 ± 33.5, P < .0001). The PROMIS PI score (61.6 ± 6.2 vs 50.6 ± 9.2, P < .001) and VAS pain score (55.0 ± 18.7 vs 17.8 ± 19.6, P < .001) showed statistically significant improvements at 1 year after surgery.

Floor and ceiling effects at 1 year postoperatively for all PROMs are summarized in Table 1. Ceiling effects were observed for the HOS-ADL and mHHS, with 16.1% and 19.3% of patients, respectively, achieving the maximum scores. No floor effect was observed for any measure.

Table 1

Floor and Ceiling Effects of Postoperative Outcome Scores

	Floor, %	Ceiling, %
HOS-ADL	0	16.1∗
HOS-SS	0	12.1
iHOT-12 score	0	3.3
mHHS	0	19.3∗
PROMIS Pain Interference score	3.3	7.4
PROMIS Physical Function score	4.8	3.2

HOS-ADL, Hip Outcome Score–Activities of Daily Living Subscale; HOS-SS, Hip Outcome Score–Sports Subscale; iHOT-12, International Hip Outcome Tool 12; mHHS, modified Harris Hip Score; PROMIS, Patient Reported Outcomes Measurement Information System.

For ceiling or floor effects, at least 15% of patient outcomes had to achieve the maximum or minimum score.

Correlation Analysis

The results of Pearson coefficient analysis between PROMIS outcome and legacy PROM scores are summarized in Table 2. Preoperatively, the PROMIS PF score showed a very good correlation with the iHOT-12 score and good correlations with the HOS-ADL, HOS-SS, and mHHS. The PROMIS PI score showed a fair correlation with the HOS-SS and good correlations with the HOS-ADL, iHOT-12 score, and mHHS. At 1 year postoperatively, the PROMIS PF score showed a very good correlation with the HOS-ADL and good correlations with the HOS-SS, mHHS, and iHOT-12 score. The PROMIS PI score showed very good correlations with the HOS-ADL and mHHS and good correlations with the HOS-SS and iHOT-12 score.

Table 2

Correlation Analysis of Preoperative and 1-Year Postoperative Patient-Reported Outcomes

	PROMIS Physical Function Score		PROMIS Pain Interference Score
	r	P Value	r	P Value
Preoperative
HOS-ADL	0.680	<.0001∗	–0.688	<.0001∗
HOS-SS	0.616	<.0001∗	–0.574	<.0001∗
iHOT-12 score	0.710	<.0001∗	–0.647	<.0001∗
mHHS	0.675	<.0001∗	–0.690	<.0001∗
VAS pain score	–0.471	<.0001∗	0.616	<.0001∗
Postoperative at 1 yr
HOS-ADL	0.729	<.0001∗	–0.721	<.0001∗
HOS-SS	0.621	<.0001∗	–0.635	<.0001∗
iHOT-12 score	0.622	<.0001∗	–0.654	<.0001∗
mHHS	0.701	<.0001∗	–0.788	<.0001∗
VAS pain score	–0.642	<.0001∗	0.658	<.0001∗

HOS-ADL, Hip Outcome Score–Activities of Daily Living Subscale; HOS-SS, Hip Outcome Score–Sports Subscale; iHOT-12, International Hip Outcome Tool 12; mHHS, modified Harris Hip Score; PROMIS, Patient Reported Outcomes Measurement Information System; VAS, visual analog scale.

Statistically significant.

Responsiveness Analysis

The pooled effect sizes for each PROM can be found in Table 3. The PROMIS outcomes had smaller mean differences and smaller pooled standard deviations compared with the legacy PROMs. The pooled effect size was large for all PROMs.

Table 3

Preoperative Versus Postoperative Outcome Score Difference and Pooled Effect Size

	Mean Score Difference	Pooled SD	Pooled Effect Size (d)
PROMIS Pain Interference score	–11.04	10.47	1.05
PROMIS Physical Function score	10.78	10.62	1.01
HOS-ADL	26.03	19.00	1.37
HOS-SS	39.57	27.27	1.45
mHHS	23.63	17.92	1.32
iHOT-12 score	43.54	24.61	1.77

HOS-ADL, Hip Outcome Score–Activities of Daily Living Subscale; HOS-SS, Hip Outcome Score–Sports Subscale; iHOT-12, International Hip Outcome Tool 12; mHHS, modified Harris Hip Score; PROMIS, Patient Reported Outcomes Measurement Information System; SD, standard deviation.

The REs between outcomes are summarized in Table 4. A value greater than 1 indicates that the PROM tool in the left column is more responsive than the PROM tool in the top row. The iHOT-12 score was the most responsive PROM, being more responsive than the PROMIS PI score, PROMIS PF score, HOS-ADL, and HOS-SS. The PROMIS PF score was overall the least responsive across all PROMs.

Table 4

Relative Efficiency Between Preoperative and 1-Year Postoperative Outcome Measures

	PROMIS Pain Interference Score	PROMIS Physical Function Score	HOS-ADL	HOS-SS	iHOT-12 Score
PROMIS Pain Interference score	—	1.04	0.77	0.72	0.59
PROMIS Physical Function score	0.96	—	0.74	0.70	0.57
HOS-ADL	1.30	1.36	—	0.94	0.77
HOS-SS	1.38	1.44	1.06	—	0.82
mHHS	1.08	1.12	0.93	0.74	0.80
iHOT-12 score	1.69	1.75	1.29	1.22	—

NOTE. A value greater than 1 indicates that the patient-reported outcome measure in the left column is more responsive than the patient-reported outcome measure in the top row.

HOS-ADL, Hip Outcome Score–Activities of Daily Living Subscale; HOS-SS, Hip Outcome Score–Sports Subscale; iHOT-12, International Hip Outcome Tool 12; mHHS, modified Harris Hip Score; PROMIS, Patient Reported Outcomes Measurement Information System.

Discussion

In this study, we found that both the PROMIS PF and PROMIS PI instruments show good to very good correlations with legacy hip-specific PROMs. We also found that at minimum 1-year follow-up, there were no floor effects for any PROM; however, there were ceiling effects for the HOS-ADL and mHHS, with 16.1% and 19.3% of patients, respectively, achieving the highest possible scores. When comparing responsiveness, we found that the effect size was large for all outcomes (Cohen d > 1) but was largest for the iHOT-12 score. The PROMIS PF score was overall the least responsive (with the PROMIS PI score being marginally more responsive), and the iHOT-12 score was more responsive than the HOS-SS, HOS-ADL, PROMIS PI score, and PROMIS PF score. These findings confirmed our hypothesis.

There is a relative paucity of literature regarding the psychometric properties of PROMIS scores in patients after hip arthroscopy for FAIS. In 2017, Sheean et al. first showed that PROMIS scores were significantly different between FAIS patients and controls, establishing the PROMIS instrument as a measure to potentially capture the disability associated with the syndrome. Nwachukwu et al. evaluated the preoperative performance of the PROMIS PF score in 197 nonoperative patients with FAIS. They found good to excellent Pearson correlations between the PROMIS score and the iHOT-12 score, HOS-ADL, HOS-SS, and mHHS. Each PROM was measured preoperatively in a cohort in which 100% of patients received a diagnosis of FAIS and were indicated for hip arthroscopic surgery. The study was limited, however, by a lack of postoperative follow-up and analysis. After this work, Nwachukwu et al. evaluated the performance of the PROMIS PF and PROMIS PI scores in patients with a minimum 6-month follow-up after hip arthroscopy for FAIS. In contrast to the high performance of the PROMIS PF score preoperatively, the authors found that the PROMIS PF score had a lower effect size compared with legacy PROMs and that there was a worse correlation with the iHOT-12 score as compared with the HOS-SS. Overall, the iHOT-12 score was the most responsive and had the largest effect size. Given that there is continued improvement in function in the postoperative period and PROMIS measures have been shown to be less responsive than legacy PROMs, it is necessary to compare how well PROMIS measurements are able to detect this improvement at multiple time points in the postoperative period. When compared with our study, the correlations at 6-month follow-up between the PROMIS PROMs and legacy PROMs were slightly stronger.

In this study, we found that the PROMIS PF and PROMIS PI measures correlated well with legacy hip-specific PROMs, with the best correlation shown for the HOS-ADL and mHHS. However, it should be noted that none of these correlations were excellent. At 1 year postoperatively, the best correlation between the PROMIS CAT and legacy PROMs was with the HOS-ADL and mHHS and the worst was with the HOS-SS and iHOT-12 score. This finding is important because both the HOS-SS and iHOT-12 score showed the highest responsiveness and the iHOT-12 score showed the highest RE over other PROMs. In this context, it should also be considered that neither the HOS-SS nor the iHOT-12 score had substantial ceiling effects. Simultaneously, the PROMIS PF and PROMIS PI scores showed the lowest responsiveness and lowest RE, which indicates that these measures may not be suitable to fully detect changes in function at 1-year follow-up in patients undergoing hip arthroscopy for FAIS. These data underscore the caution that we should take prior to replacing legacy PROMs with the PROMIS CAT. Given these findings, the PROMIS CAT may not completely gather all dimensions of hip-specific disability in FAIS compared with other hip-specific legacy PROMs.

Limitations

This study is not without limitations. The major limitation in this study is the limited follow-up period of 1 year postoperatively. Although we can support the use of the PROMIS tool and how it correlates with legacy PROMs over a 1-year period, we are unable to predict further outcomes. Additionally, our study population consisted of patients who underwent hip arthroscopy with routine capsular closure, and thus, we are unable to evaluate the correlation between PROMIS measures and legacy PROMs for patients not undergoing surgical intervention. Next, this is a single-surgeon series, with all surgical procedures being performed by a high-volume hip arthroscopist, and therefore, there may be limited generalizability. Finally, this study may be limited by its sample size, as well as its nonrandomized retrospective study design.

Conclusions

At 1-year follow-up, PROMIS PROMs displayed very good correlations with legacy PROMs. However, PROMIS PROMs had lower effect sizes and were not as responsive as legacy PROMs.