Performance of the PROMIS in Patients Undergoing 3 Common Elbow Procedures.

BACKGROUND: Ulnar collateral ligament (UCL) reconstruction, distal biceps tendon repair, and elbow arthroscopic surgery are common elbow procedures performed in active patients. HYPOTHESIS: We hypothesized (1) good to excellent correlation between Patient-Reported Outcomes Measurement Information System (PROMIS) instruments and traditional orthopaedic upper extremity patient-reported outcome (PRO) measures; (2) that PROMIS instruments would demonstrate ceiling effects; and (3) that the PROMIS physical function computer adaptive test (PF CAT) would demonstrate a low question burden compared with other PRO instruments. STUDY
DESIGN: Cohort study (diagnosis); Level of evidence, 2.
METHODS: A total of 76 patients undergoing UCL repair/reconstruction, distal biceps tendon repair, or elbow arthroscopic surgery filled out the Short Form-36 Health Survey (SF-36) Physical Function subscale, EuroQol-5 Dimensions (EQ-5D) questionnaire, PROMIS PF CAT, and PROMIS upper extremity item bank (UE). Excellent correlation between PROs was defined as ≥.70.
RESULTS: The PROMIS PF CAT had excellent correlation with the SF-36 (r = 0.74; P < .0001), Disabilities of the Arm, Shoulder and Hand (DASH) survey (r = -0.76; P < .0001), and PROMIS UE (r = 0.73; P < .0001). The PROMIS UE demonstrated excellent correlation with the SF-36 (r = 0.73; P < .0001) and DASH survey (r = -0.81; P < .0001). The PROMIS UE had ceiling effects in 33% of patients. The SF-36 showed ceiling effects in 20% of patients. On average, patients answered 5.1 ± 2.2 questions on the PROMIS PF CAT.
CONCLUSION: The PROMIS PF CAT and PROMIS UE are valid in patients undergoing distal biceps tendon repair, elbow arthroscopic surgery, and UCL repair. The PROMIS UE demonstrated high ceiling effects in younger, higher functioning patients and should be used with caution in this group. A further evaluation and modification of the PROMIS UE in younger, high-functioning patients are warranted. Finally, the PROMIS PF CAT exhibited a low question burden relative to traditional PRO instruments without the loss of reliability.

Three common elbow procedures in an active patient population include ulnar collateral ligament (UCL) repair/reconstruction, distal biceps tendon repair, and elbow arthroscopic surgery. Patients who undergo UCL procedures have been able to return to an elite level of play and to experience improved performance after their procedure.[9,14] Patients undergoing distal biceps tendon repair have achieved restoration of normal function as measured by patient-reported outcome (PRO) measures, such as the Disabilities of the Arm, Shoulder and Hand (DASH) survey, with minimal morbidity and a low complication rate.[15] Previous work has found that the 3 most common indications for elbow arthroscopic surgery were osteochondritis dissecans, lateral epicondylitis, and release and debridement, with surgical indications increasing over time.[8] Patients who undergo elbow arthroscopic surgery for osteochondritis dissecans have good clinical outcomes, with a reported 62% rate of return to sports.[2]

PRO instruments provide patients with an opportunity to actively evaluate their health care and report on their physical function in a standardized manner and are increasingly critical in evaluating the effectiveness of orthopaedic interventions.[3,6,7,12,13] Further, the utility of PRO instruments continues to expand, with these tools now playing a role in assessing how patient care is valued.[17] The Patient-Reported Outcomes Measurement Information System (PROMIS) was developed by the United States National Institutes of Health in an effort to advance PRO measurements by developing question banks for major health domains. The PROMIS can be used to assess specific PROs, and it provides the option for computer adaptive tests (CATs), which employ a dynamic sequence of questions based on patient responses and end when all criteria for the given questionnaire are met.[13,16] Item response theory, which analyzes the response to a question and how it relates to other questions within the same field, was used to create item banks for CATs.[13,16] By utilizing item response theory to create CATs, individual question responses and their relationship with other questions in the domain can be analyzed to deliver a questionnaire that evaluates all necessary patient attributes but does so with the fewest number of questions.[13,16] The PROMIS upper extremity item bank (UE) includes 16 questions focused on upper extremity musculoskeletal abnormalities; the PROMIS physical function CAT version 1.2 (PF CAT) is broader than the PROMIS UE and includes 121 possible questions assessing both upper and lower extremities.

Prior work has identified ceiling effects when utilizing the PROMIS UE in a young, healthy patient population.[1] Ceiling effects occur when greater than 15% of the observed population achieves the highest possible score on a PRO questionnaire and can be observed when high-functioning patients are asked about their ability to complete relatively simple physical tasks.[18] The presence of ceiling effects may indicate inadequate content validity for a PRO measure, as the questions available may not be broad enough to cover the target population completing the questionnaire.[18] Further studies are needed to evaluate the PROMIS in specific patient populations and disease processes, including common elbow orthopaedic abnormalities.[1,10,12]

The PROMIS has not previously been assessed in patients undergoing UCL repair/reconstruction, distal biceps tendon repair, and elbow arthroscopic surgery. We hypothesized that (1) there would be good to excellent correlation between PROMIS instruments and other traditional orthopaedic upper extremity PRO measures; (2) PROMIS instruments would demonstrate ceiling effects in a younger, more active patient population; and (3) the PROMIS PF CAT would demonstrate a relatively low question burden compared to other PRO instruments.

Methods

Institutional review board approval for this study was waived. A total of 76 consecutive patients indicated and scheduled for an operative intervention for common elbow abnormalities were prospectively enrolled in a sports medicine clinic at their preoperative visit. Demographic data on each patient were collected from an electronic medical record review. Participants completed standard PRO instruments, including the Short Form–36 Health Survey (SF-36) with subscales (Physical Function, Emotional Well-being, Vitality, Social Function, Pain, General Health), EuroQol–5 Dimensions (EQ-5D) questionnaire, DASH survey, PROMIS PF CAT, and PROMIS UE, on a computer kiosk during the preoperative office visit.

We tested construct validity by evaluating the correlation between the PROMIS instruments (UE and PF CAT) and other PRO instruments that measured physical function (convergent validity: DASH, SF-36 Physical Function subscale) and with instruments measuring other health domains (divergent validity: other SF-36 subscales, EQ-5D). We utilized the Shapiro-Wilk test to assess the normality of variables. We subsequently utilized these results and described the relationships between various PRO instruments using Pearson or Spearman correlation coefficients. Correlation was defined as excellent (>0.70), excellent-good (0.61-0.70), good (0.31-0.60), and poor (0.20-0.30).[11] We determined floor and ceiling effects to be present if more than 15% of participants scored the lowest or highest possible total score, respectively, on a PRO instrument.[18]

A prospective sample size was estimated. We determined that a minimum sample size of 46 would allow us to detect a correlation of 0.40 (moderate) between PROs (80% power; alpha = .05). Statistical software (SAS version 9.4; SAS Institute) was utilized for analyses, and P < .05 was considered statistically significant.

Results

Of the 76 study patients, 28 were indicated for elbow arthroscopic surgery, 26 patients were indicated for distal biceps tendon repair, and 22 patients were indicated for UCL repair/reconstruction. The mean age was 35 years (range, 17-70 years), 93% were male, and the mean body mass index was 29.1 kg/m2 (range, 20-49 kg/m2).

The scores for each PRO instrument are shown in Table 1. Differences between PROs among those who underwent UCL repair/reconstruction versus those who underwent distal biceps tendon repair or elbow arthroscopic surgery can be viewed in Table 2. Correlations among the PROs are presented in Table 3.

TABLE 1

Scores for PRO Instruments

Instrument	Median (Interquartile Range) [Minimum-Maximum]
SF-36
Physical Function	85 (25) [15-100]
Emotional Well-Being	18.6 (7.8) [2.8-40.0]
Vitality	65 (30) [15-100]
Social Function	100 (25) [12.5-100]
Pain	65 (42.5) [0-100]
General Health	80 (27.5) [40-100]
DASH	19.6 (24.6) [0.0-71.1]
EQ-5D	0.817 (0.157) [0.436-1.000]
PROMIS PF CAT	49.7 (12.0) [33.2-73.3]b
PROMIS UE	41.5 (19.3) [19.2-56.0]

DASH, Disabilities of the Arm, Shoulder and Hand; EQ-5D, EuroQol–5 Dimensions; PRO, patient-reported outcome; PROMIS PF CAT, Patient-Reported Outcomes Measurement Information System physical function computer adaptive test; PROMIS UE, Patient-Reported Outcomes Measurement Information System upper extremity item bank; SF-36, Short Form–36 Health Survey.

The PROMIS PF CAT is normally distributed (49.9 ± 8.6).

TABLE 2

Differences in PRO Scores

Surgery Type	SF-36						DASH	EQ-5D	PROMIS PF CATb	PROMIS UE
	Physical Function	Emotional Well-Being	Vitality	Social Function	Pain	General Health
UCL repair/reconstruction	95 (20)[60-100]c,d	18.7 (3.8)[7.6-22.8]	67.5 (15)[40-85]	100 (25)[25-100]	72.5 (45)[10-100]c	87.5 (15)[55-100]	12.9 (10.8)[0.0-46.7]c,d	0.830 (0.145)[0.446-1.000]	54.1 ± 8.7; 53.0 (13.4)[40.4-73.3]d	56.0 (14.6)[37.7-56.0]c,d
Arthroscopic surgery	85 (15)[15-100]c	18.4 (8.1)[2.8-40.0]	60 (25)[15-100]	57.5 (40)[10-80]	57.5 (40)[10-80]c	80 (20)[40-100]	23.8 (22.1)[6.7-58.3]c,d	0.778 (0.183)[0.463-1.000]	49.2 ± 8.4; 49.1 (10.0)[35.6-73.3]d	39.1 (10.0)[25.1-56.0]c,d
Biceps tendon repair	80 (30)[50-100]d	17.1 (8.2)[3.0-23.0]	67.5 (30)[15-100]	67.5 (45)[0-100]	67.5 (45)[0-100]	80 (30)[45-100]	30.8 (34.1)[4.2-7.2]	0.806 (0.154)[0.861-0.436]	47.2 ± 7.6; 48.1 (8.9)[33.2-60.7]	39.6 (23.6)[19.2-56.0]

Data are presented as median (interquartile range) [minimum-maximum] unless otherwise specified. DASH, Disabilities of the Arm, Shoulder and Hand; EQ-5D, EuroQol–5 Dimensions; PRO, patient-reported outcome; PROMIS PF CAT, Patient-Reported Outcomes Measurement Information System physical function computer adaptive test; PROMIS UE, Patient-Reported Outcomes Measurement Information System upper extremity item bank; SF-36, Short Form–36 Health Survey; UCL, ulnar collateral ligament.

Data are presented as mean ± SD; median (interquartile range) [minimum-maximum].

Significant difference between UCL repair/reconstruction and arthroscopic surgery.

Significant difference between UCL repair/reconstruction and biceps tendon repair.

TABLE 3

Correlations (SCCs) Among PROs

Instrument	SF-36					EQ-5D
	Emotional Well-Being	Vitality	Social Function	Pain	General Health
SF-36 Physical Function	0.18	0.44b,c	0.41b,c	0.64b,d	0.40b,c	0.62b,d
DASH	–0.31b,c	–0.46b,c	–0.37b,c	–0.70b,d	–0.41b,c	–0.79b,e
PROMIS PF CAT	0.19	0.36b,c	0.36b,c	0.62b,d	0.24b,f	0.74b,e
PROMIS UE	0.15	0.32b,c	0.20f	0.53b,c	0.34b,c	0.62b,d

DASH, Disabilities of the Arm, Shoulder and Hand; EQ-5D, EuroQol–5 Dimensions; PRO, patient-reported outcome; PROMIS PF CAT, Patient-Reported Outcomes Measurement Information System physical function computer adaptive test; PROMIS UE, Patient-Reported Outcomes Measurement Information System upper extremity item bank; SCC, Spearman correlation coefficient; SF-36, Short Form–36 Health Survey.

< .05.

Good correlation (0.31 ≤ SCC ≤ 0.60).

Excellent-good correlation (0.61 ≤ SCC ≤ 0.70).

Excellent correlation (SCC > 0.70).

Poor correlation (0.20 ≤ SCC ≤ 0.30).

In 76 patients, ceiling effects were prevalent in 33% of the patients who responded to the PROMIS UE. Ceiling effects were also present in 20% of the patients filling out the SF-36 Physical Function subscale. The majority of ceiling effects when administering the PROMIS UE were present in patients undergoing distal biceps tendon repair (31%) and UCL repair/reconstruction (18%). Of the patients who underwent UCL repair/reconstruction, 41% experienced ceiling effects when administered the SF-36 Physical Function subscale. No ceiling effects were experienced by patients who underwent elbow arthroscopic surgery. There was a relatively low question burden when administering the PROMIS PF CAT compared with other PRO measures (Table 4). Patients averaged 5.1 ± 2.2 questions on the PROMIS PF CAT.

TABLE 4

Question Burden, Person Reliability, and Item Reliability of PRO Instruments

	PROMIS PF CAT	PROMIS UE	DASH	SF-36 Physical Function
No. of questions	5.1 ± 2.2b	15 (13-15)c	30d	10d
Item reliability (r)	0.87	0.94	0.98	0.98
Person reliability	0.91	0.75	0.93	0.85

DASH, Disabilities of the Arm, Shoulder and Hand; PRO, patient-reported outcome; PROMIS PF CAT, Patient-Reported Outcomes Measurement Information System physical function computer adaptive test; PROMIS UE, Patient-Reported Outcomes Measurement Information System upper extremity item bank; SF-36, Short Form–36 Health Survey.

Data are presented as mean ± SD.

Data are presented as mean (range).

Data are presented as mean.

Discussion

PRO instruments are indispensable tools for orthopaedic clinical research and for evaluating the efficacy of surgical interventions by assessing a patient’s report of his or her symptoms. The utility of a PRO instrument is based on it being reproducible, having internal consistency, and having validity.[6] Further, tools such as the PROMIS are especially useful because they enable an individualized assessment without the loss of precision or internal validity.[4] However, the external or criterion validity of new PRO instruments still needs to be assessed by relating them to gold-standard measures—in this case, validated PRO measures—in a similar population.[18] Our study findings show that in patients undergoing 3 target orthopaedic elbow procedures, there was excellent correlation between the PROMIS PF CAT and traditional PRO instruments, including the SF-36 Physical Function subscale (r = 0.74; P < .0001) and DASH survey (r = –0.76; P < .0001). Similarly, the PROMIS UE had excellent correlation with the SF-36 Physical Function subscale (r = 0.73; P < .0001) and DASH survey (r = –0.81; P < .0001). Alternatively, poor to good correlation was noted between the PROMIS PF CAT and other SF-36 subscales. In all, our results suggest that the PROMIS UE and PROMIS PF CAT could be effective substitutes or supplements to established PRO instruments in patients undergoing UCL reconstruction, distal biceps tendon repair, and elbow arthroscopic surgery.

Healthy, able-bodied patients may easily achieve high scores and have been identified as a population in which PROMIS instruments might be susceptible to ceiling effects because of insufficient discriminatory power.[10] Ceiling effects were found to be present when using the PROMIS in a previous investigation that considered a young, healthy patient population with shoulder instability.[1] The present study found notable ceiling effects present in several PRO instruments across the entire group and on subgroup analysis. Of the significant ceiling effects noted with the PROMIS UE (33%) in our total cohort, all instances were in those undergoing UCL repair/reconstruction (n = 5; 23%) and biceps tendon repair (n = 8; 31%). Likewise, the significant ceiling effects seen with the SF-36 Physical Function subscale were constrained to patients who underwent UCL repair/reconstruction (n = 9; 41%). We found that patients in the UCL reconstruction group were considerably younger (20.3 ± 3.1 years) than those in the arthroscopic surgery (36.5 ± 25.1 years) and biceps tendon repair groups (48.0 ± 11.0 years). This variation in age among groups supports the notion of decreased discerning power with the PROMIS instruments, specifically the PROMIS UE, in higher functioning and younger patients.

Additionally, we found high ceiling effects in those undergoing distal biceps tendon repair. In our experience, patients who undergo distal biceps tendon repair tend to be high-functioning, middle-aged male patients who are often involved in fitness/weight-lifting activities or have labor-intensive jobs. Thus, this group could be more likely to experience ceiling effects when utilizing the PROMIS UE. We advise using the PROMIS UE with caution in patients undergoing UCL repair/reconstruction and distal biceps tendon repair. Modifications to the PROMIS UE to create questions inquiring about a patient’s capability to complete higher level physical activity and that increase the specificity for the type of activities performed by higher functioning patients should be considered to decrease potential ceiling effects.

Increasing the precision of PRO instruments such as those presented in our study has classically required increasing the question burden.[5] However, the application of CATs, as with the PROMIS PF CAT, is an effective method of achieving high precision without increasing the number of questions and while still maintaining internal reliability.[4,5,12] Unlike conventional PRO instruments with set numbers of questions, these new instruments implementing CATs allow for an individualized patient assessment with fewer questions and higher precision, which ultimately results in smaller sample sizes needed for clinical research.[5,10,13] Additionally, a lower question burden typically results in a greater overall response by study participants, thus improving research efficiency.[10,13] We found a low question burden for the PROMIS PF CAT compared with other PRO instruments in our investigation (Table 4). The PROMIS PF CAT had the lowest question burden (5.1 ± 2.2 questions) in comparison to the DASH survey (30 questions) and SF-36 Physical Function subscale (10 questions). Moreover, the PROMIS PF CAT maintained excellent item and person reliability, similar to the DASH survey and SF-36 Physical Function subscale (Table 4). In patients with operative orthopaedic elbow abnormalities, our findings suggest that the PROMIS PF CAT may be superior to established PRO instruments in terms of decreased question burden, which may lead to increased questionnaire completion as has been previously reported.

Several limitations are present in our study. First, based on our design, patients were required to fill out several PRO instruments in a short period of time. This high volume of questions could have induced question fatigue among participants, which may have altered our findings. However, the order of the PRO instruments was randomized, so this did not affect one PRO more than any other, and using fewer PRO instruments would have limited our ability for an assessment of the PROMIS instruments. Additionally, we likely did not compare the PROMIS instruments to all PRO measures being utilized by orthopaedic surgeons for the elbow interventions that we considered. Finally, our study purposefully constrained its evaluation to 3 common orthopaedic elbow procedures, meaning that our results may not generalize to other elbow abnormalities not treated through these methods or differing patient populations.

Conclusion

The PROMIS PF CAT and PROMIS UE revealed good to excellent correlation with established PRO instruments in patients with common orthopaedic elbow conditions. The PROMIS UE demonstrated high ceiling effects in younger, higher functioning patients and should be used with caution in this group. A further evaluation and modification of the PROMIS UE in younger, high-functioning patients are warranted. Finally, the PROMIS PF CAT exhibited a low question burden relative to traditional PRO instruments, without a loss of reliability.