How Comprehensive and Efficient Are Patient-Reported Outcomes for Rotator Cuff Tears?

BACKGROUND: Increasing emphasis is placed on patient-reported outcomes (PROs) after common orthopaedic procedures as a measure of quality. When considering PRO utilization in patients with rotator cuff tears, several different PROs exist with varying levels of accuracy and utilization. HYPOTHESIS/
PURPOSE: Understanding which disease-specific PRO may be most efficiently administered in patients after rotator cuff repair may assist in promoting increased patient and physician adoption of these useful scores. Using a novel assessment criterion, this study assessed all commonly used rotator cuff PROs. We hypothesize that surveys with fewer numbers of questions may remain comparable (with regard to comprehensiveness) to longer surveys. STUDY
DESIGN: Systematic review.
METHODS: Commonly utilized rotator cuff PROs were analyzed with regard to number of survey components, comprehensiveness, and efficiency. Comprehensiveness (maximum score, 11) was scored as the total number of pain (at rest/baseline, night/sleep, activities of daily living [ADLs], sport, and work) and functional (strength, motion/stiffness, and ability to perform ADLs, sport, and work) metrics included, along with inclusion of quality of life/satisfaction metrics. Efficiency was calculated as comprehensiveness divided by the number of survey components.
RESULTS: Sixteen different PROs were studied. Number of components ranged from 5 (University of California at Los Angeles score [UCLA]) to 36 (Short Form-36 [SF-36], Japanese Orthopaedic Association score [JOA]). The Quality of Life Outcome Measure for Rotator Cuff Disease (RC-QoL) included all 5 pain components, while 7 PROs contained all 5 functional components. Ten PROs included a quality of life/satisfaction component. The most comprehensive scores were the RC-QoL (score, 11) and Penn (score, 10), and the least comprehensive score was the Marx (score, 3). The most efficient PROs were the UCLA, the Quick Disabilities of the Arm, Shoulder, and Hand score (QuickDASH), and Constant scores. The least efficient scores were the JOA and SF-36 scores.
CONCLUSION: Many commonly utilized PROs for rotator cuff tears are lacking in comprehensiveness and efficiency. Continued critical assessment of PRO quality may help practitioners identify the most comprehensive and efficient PRO to incorporate into daily clinical practice.

Patient-reported outcomes (PROs) play a critical role in clinical and research aspects of orthopaedic surgery. Because many ailments in orthopaedic surgery affect functional outcomes, PROs provide a means to detect and track these outcomes as patients progress through the disease course and treatment process. Moreover, these scores have been targeted for inclusion in many new health care policies for incorporation into the measure of value affecting all orthopaedic practitioners and patients.[1,2]

Unfortunately, practical implementation of PROs on a widespread scale presents several limitations. In the setting of rotator cuff disease, which is one of the most commonly encountered musculoskeletal conditions of the upper extremity,[11] a multitude of outcome scores have been introduced. However, these tools can be cumbersome to administer in a busy clinic setting, especially without ample administrative or research support staff.[5,16] On the patient side, administration of multiple outcome measures can be repetitive, time consuming, and lead to poor patient compliance. Finally, the validation of several of these metrics has been called into question, as has the rationale regarding scoring and final total score calculation.[19]

The purpose of this study was to critically assess the most commonly used PROs in the setting of rotator cuff disease. The primary goal was to determine to what degree these metrics were both repetitive and complementary. We hypothesize that those PROs with a lesser number of components (rather than those with greater inclusion of metrics) will be the most efficient.

Methods

Identification of commonly utilized PROs was determined by a previous study of high-impact rotator cuff literature after a review of high-impact rotator cuff studies[10] (Table 1). Full versions of these tools were then identified through a web-based search and included for review. All data retrieval and analysis were performed individually by 2 members of the research team (E.C.M., J.T.H.), and any disputes were reconciled through mutual agreement. The PRO identified consisted of shoulder-specific (American Shoulder and Elbow Surgeons [ASES], Constant, University of California at Los Angeles score [UCLA], simple shoulder test [SST], Japanese Orthopaedic Association [JOA], Penn, Shoulder Pain and Disability Index [SPADI], L’Insalata Shoulder Rating Questionnaire [SRQ], Korean Shoulder Scoring System [KSS], and Marx Shoulder Activity Scale), disease-specific (Western Ontario Rotator Cuff Index [WORC], Quality of Life Outcome Measure for Rotator Cuff Disease [RC-QoL]), general health (Short Form–12 [SF-12], Short Form–36 [SF-36]), and upper extremity-specific (Disabilities of the Arm, Shoulder, and Hand score [DASH]; Quick-DASH).

TABLE 1

Source Files for Patient-Reported Outcome Measures

Instrument	Source File
ASES	www.cebp.nl/vault_public/filesystem/?ID=1234
Constant	http://www.orthopaedicscore.com/scorepages/constant_shoulder_score.html
UCLA	http://www.orthopaedicscore.com/scorepages/ucla_shoulder_score.html
SST	http://depts.washington.edu/orthodev/drupal/sites/default/files/Portals/21/www/Patient%20Care/Our%20Services/Shoulder%20&%20Elbow/Articles/PDFs/Simple%20Shoulder%20Test.pdf
SF-12	https://www.hss.edu/physician-files/huang/SF12-RCH.pdf
SF-36	http://www.shcdenver.com/Portals/902/web-content/files/JamesGenuario/JG-health%20questionnaire.pdf
WORC	http://www.shouldercommunity.com/wp-content/uploads/2013/09/Western-Ontario-Rotator-Cuff-Index-WORC.pdf
DASH	http://dash.iwh.on.ca/system/files/dash_questionnaire_2010.pdf
QuickDASH	http://dash.iwh.on.ca/system/files/quickdash_questionnaire_2010.pdf
JOA	https://www.joa.or.jp/english/evaluation/pdf/Shoulder36v1_3engsheet.pdf
Penn	http://www.eliterehabsolutions.com/pdfs/PENN%20SHOULDER%20SCORE.pdf
RC-QoL	http://www.sciencedirect.com.proxy.cc.uic.edu/science/article/pii/S1058274600168545
SPADI	http://rehabilitation.uchc.edu/docs/form_spadi.pdf
SRQ	https://www.hss.edu/secure/files/WSMC-linsalata.pdf
KSS	http://www.sciencedirect.com/science/article/pii/S105827460900038X
Marx	http://www.shoulderkneeandelbowsurgery.com/pdf/roa-marx-shoulder-activity-scale.pdf

ASES, American Shoulder and Elbow Surgeons shoulder outcome score; Constant, Constant Murley score; DASH, Disabilities of the Arm, Shoulder, and Hand score; JOA, Japanese Orthopaedic Association score; KSS, Korean Shoulder Scoring System; Marx, Marx Shoulder Activity Scale; Penn, Penn shoulder score; QuickDASH, Quick Disabilities of the Arm, Shoulder, and Hand score; RC-QoL, Quality of Life Outcome Measure for Rotator Cuff Disease; SF-12, Short Form–12; SF-36, Short Form–36; SPADI, Shoulder Pain and Disability Index; SRQ, L’Insalata Shoulder Rating Questionnaire; SST, simple shoulder test; UCLA, University of California at Los Angeles shoulder score; WORC, Western Ontario Rotator Cuff Index.

Each PRO was assessed according to several parameters. This included the number of questions to be answered by the patient (excluding patient intake information), along with information related to the content of the questions in the survey (Table 2). This criterion was created by the study team according to assessment of common symptoms encountered in patients with rotator cuff disease. Because of a lack of data to suggest otherwise, each component was weighted equally.

TABLE 2

Assessment Criteria Utilized in this Study

Pain/Discomfort	Function	QoL/Satisfaction
Rest/baseline	Strength
Night/sleep	ROM/stiffness
ADL/light activity	ADL/light activity
Sport/strenuous	Sport/strenuous
Work	Work

ADL, activity of daily living; QoL, quality of life; ROM, range of motion. For each patient-reported outcome measure (PRO), the above were collected. One point for each of these 11 categories was assigned during PRO scoring.

Comprehensiveness was assessed according to the presence or absence of survey questions related to pain, function, and quality of life/satisfaction. There were 5 different pain components included (at rest/baseline, during night/sleep, during activities of daily living [ADLs]/light activities, during sport/strenuous activity, and during work). There were also 5 different functional categories included, such as those questions related to shoulder strength, range of motion/stiffness, along with ability to return to ADLs/light activity, sport/strenuous activity, and work. Each PRO was scored according to inclusion of these 10 categories as well as an additional point for including any question related to quality of life/satisfaction. Therefore, the maximum possible score was 11.

Efficiency was calculated by dividing the comprehensiveness score by the total number of questions (to be answered by the patient). Scores with higher ratios were deemed more efficient than those with lower ratios. To maintain simplicity of the score, all questions were weighted equally in the efficiency calculation, regardless of number of possible answers to that individual question. Moreover, to adequately highlight comprehensiveness, this score was presented independently of (and in addition to) efficiency.

Results

Sixteen PROs were included in this study for review (Table 1). These measures were readily accessible with an online web-based search. The Single Assessment Numeric Evaluation (SANE) was excluded as it consists of only a single number from 0 to 100. The number of patient-answered items in these scores ranged from 5 (UCLA) to 36 (SF-36, JOA).

With regard to comprehensiveness, each PRO was awarded 1 point for each of the 5 possible pain components, 1 point for each of the 5 possible functional components, and 1 point for inclusion of quality of life/satisfaction questions. With regard to pain components, only the RC-QoL score included all 5 pain categories (Table 3). The Penn and SPADI scores incorporate 4 different pain components. The Marx score did not assess pain, while the JOA and SF-12 only included 1 pain component.

TABLE 3

Pain Assessment of Patient-Reported Outcome Measures

Instrument	Rest/Baseline	Night/Sleep	ADL/Light Activity	Sport/Strenuous	Work
ASES	×	×
Constant	×	×
UCLA	×		×	×
SST	×	×
SF-12					×
SF-36	×				×
WORC	×	×
DASH	×	×
QuickDASH	×	×
JOA		×
Penn	×	×	×	×
RC-QoL	×	×	×	×	×
SPADI	×	×	×	×
SRQ	×	×
KSS	×	×	×
Marx

ADL, activity of daily living; ASES, American Shoulder and Elbow Surgeons shoulder outcome score; Contant, Constant Murley score; DASH, Disabilities of the Arm, Shoulder, and Hand score; JOA, Japanese Orthopaedic Association score; KSS, Korean Shoulder Scoring System; Marx, Marx Shoulder Activity Scale; Penn, Penn shoulder score; QuickDASH, Quick Disabilities of the Arm, Shoulder, and Hand score; RC-QoL, Quality of Life Outcome Measure for Rotator Cuff Disease; SF-12, Short Form–12; SF-36, Short Form–36; SPADI, Shoulder Pain and Disability Index; SRQ, L’Insalata Shoulder Rating Questionnaire; SST, simple shoulder test; UCLA, University of California at Los Angeles shoulder score; WORC, Western Ontario Rotator Cuff Index.

Outcomes were generally more comprehensive with regard to functional components (Table 4). Seven different scores included all 5 functional components (ASES, WORC, QuickDASH, DASH, Penn, RC-QoL, and SRQ). The Marx and SPADI scores only assessed 3 components each. Finally, 10 of 16 scores included a question related to quality of life or patient satisfaction.

TABLE 4

Functional Assessment of Patient-Reported Outcome Measures

Instrument	Strength	ROM/Stiffness	ADL/Light Activity	Sport/Strenuous	Work
ASES	×	×	×	×	×
Constant	×	×		×	×
UCLA	×	×	×		×
SST	×	×		×	×
SF-12	×		×	×	×
SF-36	×		×	×	×
WORC	×	×	×	×	×
DASH	×	×	×	×	×
QuickDASH	×	×	×	×	×
JOA	×	×	×	×
Penn	×	×	×	×	×
RC-QoL	×	×	×	×	×
SPADI	×	×	×
SRQ	×	×	×	×	×
KSS	×	×	×		×
Marx	×	×		×

ADL, activity of daily living; ASES, American Shoulder and Elbow Surgeons shoulder outcome score; Contant, Constant Murley score; DASH, Disabilities of the Arm, Shoulder, and Hand score; JOA, Japanese Orthopaedic Association score; KSS, Korean Shoulder Scoring System; Marx, Marx Shoulder Activity Scale; Penn, Penn shoulder score; QuickDASH, Quick Disabilities of the Arm, Shoulder, and Hand score; RC-QoL, Quality of Life Outcome Measure for Rotator Cuff Disease; ROM, range of motion; SF-12, Short Form–12; SF-36, Short Form–36; SPADI, Shoulder Pain and Disability Index; SRQ, L’Insalata Shoulder Rating Questionnaire; SST, simple shoulder test; UCLA, University of California at Los Angeles shoulder score; WORC, Western Ontario Rotator Cuff Index.

Only 1 PRO scored a perfect score of 11 (RC-QoL; Table 5). The Penn score was the second most comprehensive score with 10 points. The least comprehensive scores were the Constant, SST, SF-12, JOA (6 points each), and the Marx (3 points).

TABLE 5

Comprehensiveness Assessment of Patient-Reported Outcome Measures

Instrument	Pain/Discomfort (max = 5)	Function (max = 5)	QoL/Satisfaction (max = 1)	Score (max = 11)	Questions, n	Efficiency Score
ASES	2	5	0	7	18	0.39
Constant	2	4	0	6	10	0.6
UCLA	3	4	1	8	5	1.6
SST	2	4	0	6	12	0.5
SF-12	1	4	1	6	12	0.5
SF-36	2	4	1	7	36	0.19
WORC	2	5	1	8	21	0.38
DASH	2	5	1	8	30	0.27
QuickDASH	2	5	0	7	11	0.64
JOA	1	4	1	6	36	0.17
Penn	4	5	1	10	25	0.4
RC-QoL	5	5	1	11	34	0.32
SPADI	4	3	0	7	13	0.54
SRQ	2	5	1	8	21	0.38
KSS	3	4	1	8	17	0.47
Marx	0	3	0	3	7	0.43
Mean	2.31	4.31	0.63	7.25	19.25	0.49

ASES, American Shoulder and Elbow Surgeons shoulder outcome score; Contant, Constant Murley score; DASH, Disabilities of the Arm, Shoulder, and Hand score; JOA, Japanese Orthopaedic Association score; KSS, Korean Shoulder Scoring System; Marx, Marx Shoulder Activity Scale; Penn, Penn shoulder score; QoL, quality of life; QuickDASH, Quick Disabilities of the Arm, Shoulder, and Hand score; RC-QoL, Quality of Life Outcome Measure for Rotator Cuff Disease; SF-12, Short Form–12; SF-36, Short Form–36; SPADI, Shoulder Pain and Disability Index; SRQ, L’Insalata Shoulder Rating Questionnaire; SST, simple shoulder test; UCLA, University of California at Los Angeles shoulder score; WORC, Western Ontario Rotator Cuff Index.

When considering efficiency (Figure 1), which was defined as the comprehensiveness score divided by the number of patient-answered components, the most efficient score was the UCLA PRO, with a score of 1.60. The UCLA was the only efficiency score of >1.00 because the questions in the survey addressed multiple domains. The next most efficient scores were the QuickDASH (0.64) and Constant (0.60) scores. The ASES was ranked 10th out of 16 scores.

Figure 1.

Bar graph of comprehensiveness scores (score) with overlying line graph of efficiency scores. ASES, American Shoulder and Elbow Surgeons shoulder outcome score; Constant, Constant-Murley score; DASH, Disabilities of the Arm, Shoulder, and Hand score; JOA, Japanese Orthopedic Association score; KSS, Korean Shoulder Scoring System; Marx, Marx Shoulder Activity Scale; Penn, Penn Shoulder score; QuickDASH, Quick Disabilities of the Arm, Shoulder, and Hand score; RC-QoL, Quality of Life Outcome Measure for Rotator Cuff Disease; SF-12, Short Form–12; SF-36, Short From–36; SPADI, Shoulder Pain and Disability Index; SRQ, L’Insalata Shoulder Rating Questionnaire; SST, simple shoulder test core; UCLA, University of California at Los Angeles shoulder score; WORC, Western Ontario Rotator Cuff Index. “Score” was defined as the total number of categories included from Table 2, and “Efficiency” was defined as the total score divided by the number of questions in the patient-reported outcome.

Discussion

The results of this study indicate that the UCLA, QuickDASH, and Constant scores represent the most efficient outcome measures in patients with rotator cuff tears. These scores were derived by a novel, prospectively applied criterion that factored in comprehensiveness and number of survey components for each PRO. While this methodology is novel and has not been validated, it is the first attempt of its kind to quantitatively assess the relative comprehensiveness and efficiency of PROs for rotator cuff disease.

The importance of PROs has been increasingly signified in the current medical literature. Especially in shoulder surgery, where many procedures are elective and performed to improve function while decreasing pain, PROs are critical in monitoring outcomes after surgery.[3,4] These tools have proven extremely useful when combined with registry data in patients undergoing total joint replacement.[4,13] Within shoulder surgery, they have also been integral in tracking outcomes, as in after treatment for rotator cuff tear, glenohumeral arthritis, and glenohumeral instability.[6,8,14]

Despite their utility, there are significant limitations in gaining widespread utilization of PROs. On a primitive level, there remains debate regarding their true correlation to clinical outcomes (and, in particular, to patient satisfaction).[7,12] For the physician, given the time and administrative resources required for successful implementation, concerns exist whether practitioners can afford to routinely administer these scores.[5,16] On the patient side, completion of multiple scores is often repetitive and time consuming, resulting in poor compliance, particularly at longer follow-up time periods.

When considering PROs in rotator cuff tears, there has been a significant amount of variability and inconsistency reported in high-impact literature.[10] Such variability creates challenges in comparing results from different studies and therefore with truly understanding outcomes after treatment.[17,20] In addition, the lack of standardization will make incorporation of PROs into value-based reimbursement models challenging for both providers and payers. At this time, there is no consensus from clinical experts regarding which PRO is most appropriate to use in patients with rotator cuff tear, as well as whether there is any single PRO that is adequate to use in isolation. Identification of the “ideal” PRO—one that is comprehensive, efficient, disease-specific, and validated—will contribute positively to improving value-based assessments of clinical care in these patients. Therefore, standardizing utilization of these scores and choosing scores that are efficient and comprehensive to administer will become increasingly important. The results of our study indicate that the most commonly utilized scores are variable in the consistency of the domains that they assess. However, significantly more variability exists when considering the number of questions in each survey. In our study, we found that the UCLA outcome tool was significantly more efficient than other tools, and this was primarily because of the fact that it only contains 5 questions. It is important to note, however, that multiple domains were included in individual questions, therefore allowing for an efficiency score of >1. In contrast, the ASES, which is the most commonly utilized PRO for rotator cuff tears,[10] only had mediocre efficiency. Moreover, when compared with the Constant score, the UCLA is more comprehensive and efficient; therefore, this would indicate that the UCLA score should be preferentially utilized over the Constant score.

To our knowledge, there have been no other studies that have quantitatively assessed the composition of commonly utilized rotator cuff PROs. A recent study by Schmidt et al,[15] however, did attempt to quantitatively assess the quality of validated PROs for shoulder conditions. In this comprehensive study, the authors used EMPRO criteria to assess the quality of shoulder PROs, with special emphasis on the development, administration properties, and metric characteristics. The authors reported that the ASES, FLEX-SF (Flexilevel Scale of Shoulder Function), and OSS (Oxford Shoulder Score) scored highest with their assessment. However, this assessment was heavily focused on statistical validation attributes of the PRO as opposed to comprehensiveness and efficiency. Moreover, the PROs featured in that study do not comprise the most commonly utilized tools, as were included in our study. In a separate study by Tashjian et al,[18] the study team investigated factors contributing to patient satisfaction after rotator cuff repair. They identified factors such as improvement in physical function and general health status, as well as status of working and without disability, as being correlated with satisfaction. Therefore, inclusion of patient satisfaction and quality of life, as assessed in our study, is integral in assessing the efficacy of shoulder PROs. It is our recommendation that future studies focus on the strength of correlation of individual PROs according to clinical metrics, as well as in comparison with other PROs. In this way, identification of (ideally) a single PRO for rotator cuff tear that is comprehensive may allow for improved efficiency through elimination of multiple form administration in any given clinical or research setting.

In another recent study by Huang et al,[9] the authors performed a systematic review of the psychometric properties of PROs used in rotator cuff disease. They reported that the 3 most commonly studied outcomes were the Constant, DASH, and SPADI scores. The authors reported that, despite common utilization in clinical studies,[10] both the ASES and the Constant scores had poor background data regarding several important validity characteristics of the tools. Moreover, the authors reported that, of the 16 tools assessed that were utilized in the literature for rotator cuff disease, only the WORC and the quality of life (RC-QoL) measures were actually designed for use in patients with this diagnosis. Finally, the authors also reported that the DASH, SPADI, and SST all had “strong or moderate” positive evidence for utilization. Therefore, it is interesting to note that, despite its frequent utilization, the ASES score may not be the best suited for patients with rotator cuff tear. While the study by Huang et al[9] focused on methodological properties related to these commonly utilized PROs, our study detailed question-related attributes of these outcome scores. Moreover, we focused on efficiency ratings of each score in an attempt to define which scores may be most able to be utilized in a busy clinical practice without losing comprehensiveness or content of questions. We also found the ASES to have only mediocre ratings for utility.

There are several limitations to our study. As a de novo criterion was utilized, the study methodology has not been proven in other disease state testing, nor has utilization of the comprehensiveness or efficiency calculations. Moreover, each component was weighted equally, which may not represent true weighting of clinical concern in symptomatic patients. However, study criteria were objectively applied across all PROs tested. Additionally, and unlike previous studies, PRO efficiency was calculated by incorporating the number of survey questions into the assessment. A second limitation is that there was no assessment made of the statistical and reliability merits of the PROs included in this study. While such an assessment would be beneficial, it was beyond the scope of the study and has been previously studied as well. However, one significant strength of this study is that it only focuses on commonly utilized PROs in high-impact literature. Therefore, the impact of such an assessment is likely to be significant. Another limitation is that not all PROs were included in this study. This was intentionally done to exclude those PROs that are no longer commonly reported in the literature, as any impact of assessing these scores would be minimal. Finally, and contributing to overall lack of quality due to poor standardization of administration, we encountered several different versions of the same PROs that had been utilized in different studies. Therefore, the forms we used for investigation may have subtle differences in questions compared with forms that may be used in different clinical practices and studies. To resolve this, we have provided the web URLs for each form that we used (see Table 1). Lastly, we did not include objective data in our calculation of comprehensiveness (eg, strength and range of motion). However, as value-based health care relies on patient benefit for calculation of value, it is likely more relevant to determine the patient perception of outcome and function rather than an objective measure. In addition, there is likely an indirect correlation in that patients with moderate to severe functional weakness would be unlikely to report normal function.

Conclusion

Significant redundancy exists within commonly utilized rotator cuff PROs. From an efficiency perspective, the UCLA, QuickDASH, and Constant scores may provide practitioners with reliable opportunities for seamless integration into clinical practice. Specifically, the UCLA score is the most comprehensive and efficient. However, prior studies have validated utilization of other scores, such as the ASES, based on its methodological strength. Therefore, a focused and international consensus must be reached with regard to selection of a comprehensive PRO that can be used in isolation. Continued scrutiny of these commonly utilized PROs may help practitioners continue to refine models for continued outcome reporting and standardization of scores utilized to report outcomes for a given disease state.