Performance of the PROMIS in Patients After Anterior Cruciate Ligament Reconstruction.

BACKGROUND: The Patient-Reported Outcomes Measurement Information System (PROMIS) is designed to advance patient-reported outcome (PRO) instruments by utilizing question banks for major health domains.
PURPOSE: To compare the responsiveness and construct validity of the PROMIS physical function computer adaptive test (PF CAT) with current PRO instruments for patients before and up to 2 years after anterior cruciate ligament (ACL) reconstruction. STUDY
DESIGN: Cohort study (diagnosis); Level of evidence, 2.
METHODS: Initially, 157 patients completed the PROMIS PF CAT, Short Form-36 Health Survey (SF-36 physical function [PF] and general health [GH]), Marx Activity Rating Scale (MARS), Knee injury and Osteoarthritis Outcome Score (KOOS activities of daily living [ADL], sport, and quality of life [QOL]), and EuroQol-5 dimensions questionnaire (EQ-5D) at 6 weeks, 6 months, and 2 years after ACL reconstruction. Correlations between instruments, ceiling and floor effects, effect sizes (Cohen d), and standardized response means to describe responsiveness were evaluated. Subgroup analyses compared participants with and without additional arthroscopic procedures using linear mixed models.
RESULTS: At baseline, 6 weeks, and 6 months, the PROMIS PF CAT showed excellent or excellent-good correlations with the SF-36 PF (r = 0.75-0.80, P < .01), KOOS-ADL (r = 0.63-0.70, P < .01), and KOOS-sport (r = 0.32-0.69, P < .01); excellent-good correlation with the EQ-5D (r = 0.60-0.71, P < .01); and good correlation with the KOOS-QOL (r = 0.52-0.58, P < .01). As expected, there were poor correlations with the MARS (r = 0.00-0.24, P < .01) and SF-36 GH (r = 0.16-0.34, P < .01 ). At 2 years, the PROMIS PF CAT showed good to excellent correlations with all PRO instruments (r = 0.42-0.72, P < .01), including the MARS (r = 0.42, P < .01), indicating frequent return to preinjury function. The PROMIS PF CAT had the fewest ceiling or floor effects of all instruments tested, and patients answered, on average, 4 questions. There was no significant difference in baseline physical function scores between subgroups; at follow-up, all groups showed improvements in scores that were not statistically different.
CONCLUSION: The PROMIS PF CAT is a valid tool to assess outcomes after ACL reconstruction up to 2 years after surgery, demonstrating the highest responsiveness to change with the fewest ceiling and floor effects and a low time burden among all instruments tested. The PROMIS PF CAT is a beneficial alternative for assessing physical function in adults before and after ACL reconstruction.

Anterior cruciate ligament (ACL) reconstruction is a commonly performed orthopaedic procedure, with an estimated 200,000 ACL ruptures occurring yearly in the United States (US).[28] The vast majority (94%) of ACL reconstructions occur within 1 year of the initial injury.[9] With the influence of a growing body of literature suggesting possible associations between subtle instability and secondary joint damage leading to early arthritis, this number could continue to grow as more nonathletes consider surgical repair after an injury.[17,27]

Patient-reported outcome (PRO) instruments are an essential component in evaluating health care value and the effectiveness of orthopaedic procedures.[3,5,34] They also provide patients with a quantitative means of interpreting their own health and postoperative progress. An ideal test is quick and nonstrenuous for a patient to complete; simple for the practitioner to interpret; and highly reproducible, reliable, and accurate.[10] The Patient-Reported Outcomes Measurement Information System (PROMIS) was developed by the National Institutes of Health in an effort to clarify and improve PRO instruments by creating question banks for major health domains and computerized adaptive testing tools.[6] Adaptive testing incorporates real-time comparison of responses to individual questions and relationships between domains, allowing questions to be quickly selected and narrowed and reducing the test burden for the patient while maintaining high reliability and precision.[24] The PROMIS physical function computer adaptive test (PF CAT) is a broad instrument evaluating physical function in both upper and lower extremities; by responding to the test taker’s selections in real time, it is able to reduce the number of questions needed to accurately ascertain a patient’s functional status. A patient may answer, on average, between 4 and 12 questions from a bank of 121 possible questions, and usually, the test is completed in less than 1 minute.

PROMIS measures are scored on a t-score metric, with 50 representing the mean value in a referent population (for physical function, the general US adult population), and a patient’s SD from the mean score representing his or her final metric.[6,7] This specific test has been evaluated in multiple patient populations and disease processes, most recently by Hancock et al[12] for healthy patients undergoing meniscal surgery.

The present study expands this line of work to a larger subset of patients, those undergoing ACL reconstruction and rehabilitation. As these patients represent a sizable and often active population with high athletic demands, reliable adaptive testing with minimal ceiling effects is particularly desirable. To this end, we investigated the physical function domain of the PROMIS in patients scheduled to undergo arthroscopic ACL reconstruction. Our hypotheses were that the PROMIS PF CAT (1) would have a high correlation with other commonly used PRO measures assessing physical function and a low correlation with PRO measures evaluating other health domains or preinjury status in the rehabilitation period, (2) would have fewer ceiling effects, and (3) would have a low test burden compared with other currently employed PRO instruments.

Methods

This study was approved by an institutional review board and deemed HIPAA (Health Insurance Portability and Accountability Act) compliant. A total of 157 consecutive patients aged 14 to 53 years indicated for and scheduled to undergo primary or revision ACL reconstruction by 1 of 5 sports medicine surgeons at our institution were enrolled by research staff at the time of the surgical indication. Inclusion criteria included age ≥14 years, diagnosis of a partial or complete ACL tear, and written English language proficiency. Patients undergoing bilateral reconstructions were excluded, as laterality was not clearly specified on all survey questions.

Enrolled patients were prospectively asked to complete the PROMIS PF CAT, the Short Form–36 Health Survey (SF-36), the Marx Activity Rating Scale (MARS), the EuroQol–5 dimensions questionnaire (EQ-5D), and the Knee injury and Osteoarthritis Outcome Score (KOOS) using a computer at their preoperative office visit. These surveys were repeated at their 6-week and 6-month postoperative visits as well as at a final visit 2 years after surgery. Participant demographic data, including age, body mass index, sex, history of ipsilateral ACL reconstruction, and operative side, were obtained from chart reviews. Procedural data, including intraoperative diagnoses, additional procedures performed, and graft type, were recorded. Additional technique details were not specifically assessed in this study, although anatomic single-bundle reconstruction and fixation with a bioabsorbable tibial interference screw with extracortical femoral fixation was the method commonly utilized by all of the contributing surgeons.

Construct validity was tested by assessing the correlation of the PROMIS PF CAT with other PRO instruments that measured physical function (SF-36 physical function [PF], KOOS–activities of daily living [ADL]) and with instruments measuring other health domains (MARS, SF-36 general health [GH], and EQ-5D). The relationships between PRO instruments were described using Pearson or Spearman correlation coefficients. Correlation was defined as excellent (>0.7), excellent-good (0.61-0.7), good (0.4-0.6), and poor (0.2-0.3).[26] The effect size (Cohen d) and standardized response mean were used to describe the responsiveness of each PRO measure at the 6-week and 6-month follow-up visits and were defined as small (0.2), medium (0.5), and large (0.8). Ceiling and floor effects were defined as present if ≥15% of participants scored the highest or lowest score on a PRO instrument, respectively.[29] Subgroup analyses comparing participants with and without additional arthroscopic procedures (such as meniscectomy and/or repair, microfracture, or osteochondral allograft transplantation surgery) were performed using linear mixed models. Statistical software (SAS version 9.4; SAS Institute) was utilized for analyses, and P < .05 was considered statistically significant.

Results

Of the 157 patients enrolled, 139 were included in the final analysis; 111 of these patients returned for 2-year follow-up (Figure 1). The mean age was 24.0 ± 9.3 years, and 48.2% were women; the mean body mass index was 26.3 ± 4.7 kg/m2. Isolated ACL reconstruction was performed in 48 patients; 91 patients had undergone at least 1 additional arthroscopic or open procedure. Graft types included hamstring autografts, bone–patellar tendon–bone autografts, and tibialis anterior allografts (Table 1).

Figure 1.

Flowchart of patient selection and analysis. Of the original 157 patients enrolled, 139 patients with a 6-month follow-up were able to be included; of these, 111 completed 2-year testing. PROMIS, patient-reported outcomes measurement information system.

TABLE 1

Arthroscopic Procedures Performed

ACL reconstruction by graft type
Hamstring autograft	82
Bone–patellar tendon–bone autograft	49
Tibialis anterior allograft	8
Additional arthroscopic procedures
Partial meniscectomy	63
Meniscal repair	33
Microfracture	7
Osteochondral allograft transplantation	2
Loose body removal	1
Additional open procedures
Hardware removal	5
Peroneal neurolysis	3
High tibial osteotomy	2
Fulkerson osteotomy	1
Medial collateral ligament reconstruction	1
Lateral collateral ligament reconstruction	1
Biceps femoris repair	1
Revision ACL reconstruction	23
Total ACL reconstructions	139

Values presented as n. ACL, anterior cruciate ligament.

There were 116 primary ACL procedures and 23 revision reconstructions. On subgroup analysis, there was no statistical difference in PROMIS scores between patients who underwent isolated ACL reconstruction versus those who underwent additional concomitant arthroscopic procedures at any time point (mean PROMIS t-scores of 40.9, 0.4, 52.4, and 52.8 vs 41.2, 1.5, 51.3, and 52.9 [P = .67, .72, .45, and .85] at baseline, 6 weeks, 6 months, and 2 years after surgery, respectively). Overall, the PROMIS PF CAT demonstrated a strong correlation with other commonly used physical function PRO instruments (Table 2).

TABLE 2

Spearman Correlation Coefficients Between PROMIS PF CAT and Traditional Knee and General Health Instruments

Instrument	Baseline	6 Weeks	6 Months	2 Years
SF-36 PF	0.75 (P < .01)	0.80 (P < .01)	0.77 (P < .01)	0.72 (P < .01)
SF-36 GH	0.16 (P = .05)	0.34 (P < .01)	0.32 (P < .01)	0.43 (P < .01)
KOOS-ADL	0.70 (P < .01)	0.63 (P < .01)	0.64 (P < .01)	0.57 (P < .01)
KOOS-sport	0.69 (P < .01)	0.32 (P < .01)	0.57 (P < .01)	0.67 (P < .01)
KOOS-QOL	0.52 (P < .01)	0.58 (P < .01)	0.58 (P < .01)	0.61 (P < .01)
MARS	0.01 (P = .88)	0.00 (P = .42)	0.24 (P < .01)	0.42 (P < .01)
EQ-5D	0.71 (P < .01)	0.68 (P < .01)	0.60 (P < .01)	0.60 (P < .01)

ADL, activities of daily living; EQ-5D, EuroQol–5 dimensions questionnaire; GH, general health; KOOS, Knee injury and Osteoarthritis Outcome Score; MARS, Marx Activity Rating Scale; PF, physical function; PROMIS PF CAT, Patient-Reported Outcomes Measurement Information System physical function computer adaptive test; QOL, quality of life; SF-36, Short Form–36 Health Survey.

At baseline, 6 weeks, and 6 months, there were excellent or excellent-good correlations of the PROMIS PF CAT with the SF-36 PF (r = 0.75-0.80, P < .01), KOOS-ADL (r = 0.63-0.70, P < .01), and KOOS-Sport (r = 0.32-0.69, P < .01); excellent-good correlation with the EQ-5D (r = 0.60-0.71, P < .01); and good correlation with the KOOS-QOL (r = 0.52-0.58, P < .01). At 2 years, there was good to excellent correlation of the PROMIS PF CAT with all PRO instruments (r = 0.42-0.72, P < .01) indicating frequent return to preinjury function. As expected, for the MARS evaluation of preinjury activity, there was no correlation with the PROMIS PF CAT at baseline and 6 weeks and poor correlation with the PROMIS PF CAT at 6 months (r = 0.24, P < .01). This improved to a good correlation (r = 0.42, P < .01) at 2 years. Similarly, for the SF-36 GH, there was a poor to good correlation at all time points (r = 0.16, 0.34, 0.32, and 0.43, respectively; P = .05, P < .01, P < .01, and P < .01, respectively).

Overall, the PROMIS PF CAT demonstrated superior floor and ceiling effects compared with other PRO instruments, with a minimal ceiling effect in only 2 patients (1.4%) at 6 months and 10 patients (9.0%) at 2 years (Table 3). In comparison, at 6 months, there was a ceiling effect in 14 patients (10.1%) on the SF-36 PF and in 18 patients (13.0%) with the KOOS-sport. At 2 years, there was a significant ceiling effect (>15.0%) in all PRO measures except the PROMIS PF CAT. Additionally, the PROMIS PF CAT was the only PRO instrument without ceiling or floor effects during preoperative testing.

TABLE 3

Ceiling and Floor Effects

	Baseline		6 Weeks		6 Months		2 Years
	Ceiling	Floor	Ceiling	Floor	Ceiling	Floor	Ceiling	Floor
EQ-5D	4 (2.9)	0 (0.0)	13 (9.4)	0 (0.0)	41 (29.5)	0 (0.0)	50 (45.1)	0 (0.0)
MARS	65 (46.8)	7 (5.0)	61 (43.9)	11 (7.9)	42 (30.2)	17 (12.2)	18 (16.2)	10 (9.0)
KOOS-ADL	6 (4.3)	0 (0.0)	5 (3.6)	1 (0.7)	53 (38.1)	0 (0.0)	50 (45.1)	0 (0.0)
KOOS-sport	3 (2.2)	16 (11.5)	14 (10.1)	10 (7.2)	18 (13.0)	0 (0.0)	30 (27.0)	1 (0.9)
KOOS-QOL	3 (2.2)	13 (9.4)	0 (0.0)	2 (1.4)	3 (2.2)	1 (0.7)	18 (16.2)	0 (0.0)
SF-36 PF	13 (9.4)	1 (0.7)	0 (0.0)	1 (0.7)	14 (10.1)	0 (0.0)	46 (41.4)	0 (0.0)
SF-36 pain	5 (3.6)	0 (0.0)	5 (3.6)	0 (0.0)	35 (25.2)	0 (0.0)	46 (41.4)	0 (0.0)
PROMIS PF CAT	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)	2 (1.4)	0 (0.0)	10 (9.0)	0 (0.0)

Data are shown as n (%). ADL, activities of daily living; EQ-5D, EuroQol–5 dimensions questionnaire; KOOS, Knee injury and Osteoarthritis Outcome Score; MARS, Marx Activity Rating Scale; PF, physical function; PROMIS PF CAT, Patient-Reported Outcomes Measurement Information System physical function computer adaptive test; QOL, quality of life; SF-36, Short Form–36 Health Survey.

Effect size estimates for all physical function PRO measures increased from small or none to large changes from the 6-week to 6-month visits (Table 4), demonstrating similar ability for the PROMIS PF CAT to track change as compared with other PRO instruments.[8]

TABLE 4

Effect Size of PRO Instruments Between Study Time Periods

	Baseline to 6 Weeks		6 Weeks to 6 Months
Instrument	Effect Size (95% CI)	SRM (95% CI)	Effect Size (95% CI)	SRM (95% CI)
EQ-5D	0.36b (0.19 to 0.53)	0.357 (0.18 to 0.53)	1.15d (0.92 to 1.38)	1.009 (0.78 to 1.23)
KOOS-ADL	0.50c (0.28 to 0.70)	0.405 (0.19 to 0.61)	1.51d (1.24 to 1.78)	1.201 (0.95 to 1.48)
KOOS-sport	0.34b (0.13 to 0.55)	0.274 (0.06 to 0.48)	1.53d (1.25 to 1.80)	1.204 (0.95 to 1.45)
KOOS-QOL	0.41b (0.24 to 0.58)	0.402 (0.23 to 0.57)	1.22d (0.99 to 1.46)	1.098 (0.87 to 1.32)
SF-36 PF	0.07b (–0.12 to 0.25)	0.057 (–0.13 to 0.24)	1.37d (1.10 to 1.64)	1.050 (0.80 to 1.29)
PROMIS PF CAT	0.12b (–0.07 to 0.31)	0.100 (–0.09 to 0.29)	1.43d (1.17 to 1.69)	1.197 (0.95 to 1.44)

ADL, activities of daily living; EQ-5D, EuroQol–5 dimensions questionnaire; KOOS, Knee injury and Osteoarthritis Outcome Score; MARS, Marx Activity Rating Scale; PF, physical function; PRO, patient-reported outcome; PROMIS PF CAT, Patient-Reported Outcomes Measurement Information System physical function computer adaptive test; QOL, quality of life; SF-36, Short Form–36 Health Survey; SRM, standardized response mean.

Small effect size.

Medium effect size.

Large effect size.

Patients answered, on average, 4 questions at each visit utilizing the PROMIS PF CAT (SD, 1.0, 0.2, 1.9, and 3.2 at each time point, respectively), indicating a low time burden throughout the study for the test taker. The greatest variance occurred at 2 years, with patients answering between 4 and 12 questions at this visit (median, 4.00; mean ± SD, 6.59 ± 3.27). In comparison, participants answered 6 questions for the EQ-5D, 42 questions for the KOOS (including 17 ADL questions, 4 QOL questions, and 5 sport questions), and 36 questions for the SF-36 (including 12 PF questions).

Last, subgroup analysis between patients who underwent isolated ACL reconstruction and those with additional arthroscopic or open procedures showed no significant differences in physical function PRO scores, and at the 6-week and 6-month follow-up, both groups showed significant improvement in physical function PRO scores that were not statistically different (P > .05 for all comparisons).

Discussion

This study evaluated the performance of the PROMIS PF CAT in 139 patients after ACL reconstruction: it demonstrated good to excellent correlation with the KOOS-ADL, KOOS-sport, and SF-36 PF; exhibited similar effect size changes; and had a low test-taking burden as well as the fewest floor or ceiling effects of all instruments tested. It performed consistently both in the postoperative rehabilitation period and after return to sports at 2 years, as well as in both patients who had undergone isolated ACL reconstruction and those who had undergone secondary arthroscopic procedures at the time of reconstruction.

Comparisons between the many PRO instruments are complex, given differences in both scoring systems and the overall domains or categories that these tests assess. General indices of health and well-being, including the EQ-5D and SF-36, are well-established PRO measures with broad coverage of a patient’s overall status.[22,23,33] Their ability to capture the nuances of change in physical function in high-demand activities is limited, however, with ceiling effects observed in prior studies.[21]

Region- or disease-specific PRO instruments for the knee that are relevant to ACL reconstruction include the KOOS, the Cincinnati Knee Rating System (CKRS), the Tegner Lysholm Knee Scoring Scale, and the International Knee Documentation Committee Evaluation Form (IKDC), all of which demonstrate good reliability and high intertest correlation in athletic patients.[19] The Lysholm scale, CKRS, and KOOS have been particularly well validated for ACL-deficient patients.[2,18,25] Other PRO measures relevant in this population include the MARS, which assesses baseline (preinjury) activity in an effort to better evaluate postrehabilitation progress in particularly active patients compared with those returning to a baseline sedentary status.[20]

The PROMIS PF CAT has already been shown in multiple patient populations to have high reliability, precision, and accuracy, with a low test burden.[1,4,11,14] This instrument was recently validated in healthy patients undergoing meniscal surgery and in patients with ACL injuries indicated for reconstruction; it is an effective, quick tool for the busy practitioner and patient.[12] It also remains unique in that, unlike traditional instruments that require all questions to be completed in their entirety for composite scoring, PROMIS questionnaires can still be scored even if only partially completed. There has been particular interest of the PROMIS for use in healthy, highly active patient populations such as those undergoing ACL reconstruction; the progress made by these patients during postoperative rehabilitation can be lost because of ceiling effects with some instruments.[11] An ideal test for this population retains its utility through rehabilitation and return to sports.

Accordingly, the PROMIS PF CAT demonstrated excellent to excellent-good correlation with the alternative PRO instruments evaluated in this study. As we could not feasibly evaluate every alternative PRO measure, we selected a representative few that are well validated in ACL reconstruction; these scoring systems have also demonstrated high correlation with other instruments, including the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC), CKRS, IKDC, and Lysholm scale.[16,30,31,32]

In our study, the PROMIS PF CAT was equally as responsive to change as other instruments, indicating excellent utility throughout the postoperative course in ACL reconstruction (Table 4). We chose to re-evaluate the patient at 6 weeks after surgery, given that this is often a transition time in ACL rehabilitation toward beginning functional activities. By 6 months, we expect return to sports or close to it; those who have not yet returned by 6 months would be captured at the final time point at 2 years, at which time we are most likely to potentially observe ceiling effects in athletic patients.[12,15,19]

The PROMIS PF CAT had the lowest ceiling effect of all the instruments that we tested, at 1.4% at 6 months and 9.0% at 2 years; it was the only PRO measure at 2 years that remained below the significance cutoff of 15.0% for ceiling effects. Comparatively, the KOOS-sport had a ceiling effect at 6 months and 2 years of 13.0% and 27.0%, respectively; there was an even more pronounced ceiling effect for the SF-36 PF at 10.1% and 41.4%, respectively. It is also worth noting that several PRO instruments, including the EQ-5D, KOOS-ADL, SF-36 PF, and SF-36, pain all demonstrated ceiling effects above 40.0% at the 2-year time point, indicating an inability to accurately assess the fully rehabilitated patient. We believe that the PROMIS PF CAT is therefore a better alternative to these tests for its improved ability to capture patients at the end range of physical function testing throughout and after the rehabilitation process while maintaining high correlation with these more traditional PRO instruments. With its ability to capture nuances in high-level performance, there may be a role for the PROMIS in determining readiness for return to sports, which could be explored in future studies.

On subgroup analysis, PROMIS scores in patients who underwent secondary arthroscopic or open procedures demonstrated no significant difference in responsiveness to change, floor and ceiling effects, or correlation with other instruments in comparison with those who underwent isolated ACL reconstruction. Some of these secondary procedures, including microfracture or high tibial osteotomy, involve substantially different postoperative protocols from isolated ligamentous reconstruction, and they may indicate chronic changes within the joint; thus, with larger numbers, this subanalysis could reveal differences in the effect size or final PROMIS score. However, the consistency of data between groups in our study potentially indicates the ability of the PROMIS to evaluate a large variety of arthroscopic knee procedures beyond ACL reconstruction.

As other PROMIS validation studies have noted, the efficiency of the PROMIS PF CAT with a mean test burden of 4 questions has potential to improve and expedite patient care in the clinic setting, reducing test burnout in patients and the likelihood of incomplete data collection.[6,11,13] As our patients completed multiple PRO instruments in the same setting, we did not assess time for the completion of the PROMIS PF CAT or patient satisfaction with the test in comparison with other instruments. An evaluation of practitioner and patient perception of this type of computerized adaptive testing may bear formal analysis.

Limitations to our study include challenges encountered in other PRO instrument studies, including barriers in language and patient understanding. We did not encounter any patients who were unable to complete testing on the computer, although computer proficiency could be a factor in some populations. An additional limitation was the lower return rate for 2-year follow-up (70.7%); it is possible that some patients who did not return were doing particularly well or poorly and may have demonstrated ceiling or floor effects on the PROMIS. Last, it should be noted that PRO instruments are not designed to be a replacement for functional testing; complete return to sports without perceived functional limitation by the patient or provider remains a goal endpoint in ACL reconstruction.[15,28]

Conclusion

The PROMIS PF CAT appears to be an ideal test in patients undergoing ACL reconstruction. It demonstrated a low test-taking burden; had a high correlation with multiple other measures of physical function, including the SF-36 PF, KOOS-ADL, and KOOS-sport; had the fewest floor or ceiling effects of all instruments evaluated; and remained equally efficacious in patients undergoing secondary arthroscopic and open procedures in addition to ACL reconstruction. It performed consistently well throughout the postoperative period and after return to sports, with similar responsiveness to change as other instruments. The PROMIS PF CAT is an ideal choice for the evaluation of physical function both preoperatively and postoperatively in patients being considered for ACL reconstruction.