Are Patient-Reported Outcome Scores a Reasonable Substitute for Clinical Follow-up After Surgically Managed Acromioclavicular Joint Injuries?

Background: Various clinical outcome scores have been described to evaluate postoperative shoulder function after operatively treated acromioclavicular joint (ACJ) instability. Clinical outcome scores can be divided between patient-reported outcome measures (PROMs) and examiner-dependent outcome measures (EDOMs) after a clinical examination by a physician. The correlation between PROMs and EDOMs, and thus their interchangeability with regard to operatively treated ACJ instability, has not yet been evaluated. Purpose: To investigate whether PROMs are a reasonable substitute for EDOMs. Correlations between global shoulder (GS) and ACJ-specific outcome measures were also investigated. Study Design: Cohort study (diagnosis); Level of evidence, 3.
Methods: Included in this study were 131 consecutive patients with operatively treated ACJ instability between 2011 and 2017. Postoperative shoulder function was measured using PROMs, including the Subjective Shoulder Value (SSV), Subjective Shoulder Test, and Nottingham Clavicle Score (NCS), and EDOMs, including the Constant-Murley score (CMS), Taft score, ACJ instability (ACJI) score, and SICK Scapula Score (SSS). Associations between PROM and EDOM scores were calculated using the Pearson and Spearman correlation coefficients for linear and nonlinear variables, respectively, and were interpreted using the Cohen classification. The scores were further stratified into GS versus ACJ-specific measures.
Results: A strong correlation was observed between several PROMs and EDOMs (CMS vs SSV [r = 0.59; P = .02] and CMS vs NCS [r = 0.79; P ≤ .001]) and between several GS and ACJ-specific scores (CMS vs NCS; CMS vs ACJI [rs = 0.69; P < .001]; and CMS vs SSS [r = -0.68; P < .001]).
Conclusion: Based on the results of this study, PROMs such as the SSV (a GS measure) and the NCS (an ACJ-specific measure) can substitute for EDOMs. Clinical Relevance: PROMs that can be substituted for EDOMs can enable the conduct of clinical studies in circumstances in which in-person clinical follow-up of the patient by a physician is not possible.

Various scores have been described in the literature to measure shoulder function after operatively treated acromioclavicular joint (ACJ) injuries, leading ultimately to ACJ instability. These scores can be distinguished as either patient-reported outcome measures (PROMs) or examiner-dependent outcome measures (EDOMs). Further, while some clinical outcome scores frequently used to assess operatively treated ACJ evaluate the global shoulder (GS) function —such as the Constant-Murley score (CMS), an EDOM, or the Subjective Shoulder Value (SSV) and the Subjective Shoulder Test (SST), which are PROMs—others are specifically designed to evaluate operatively treated ACJ instability; for example, the Nottingham Clavicle Score (NCS), the Taft score, the ACJ instability (ACJI) score, and the SICK (Scapular malpositioning, Inferior medial border prominence, Coracoid pain and malposition, and dysKinesis of scapular movement) Scapula Score (SSS).

At this time, few recommendations about which of these various scores are appropriate to use to evaluate shoulder—and especially ACJ—pathologies exist; thus, their inclusion in the current literature is limited. However, the European Shoulder and Elbow Society has recommended since 1992 that researchers include the CMS in all shoulder-specific peer-reviewed articles, and as a result, the CMS has been most widely used in the literature, with an observer reliability of 3% (range, 0%-8%).

In 2017, however, Charles et al raised questions regarding the applicability of the global CMS to ACJ-specific interventions. Further, although different combinations of these outcome measures are used in various ACJ studies, the interscore correlations between them have not been reported. Clinical studies are challenging to perform in various circumstances, such as in a pandemic environment or in small, dispersed patient cohorts. Hence, questions have arisen about how strong the correlations between EDOMs and PROMs are, in what capacity or capacities PROMs can replace EDOMs, and in which cases an EDOM is indispensable in measuring the clinical outcomes after treatment for ACJ instability.

The purpose of this study was to investigate whether PROMs are a reasonable substitute for EDOMs when evaluating outcomes after operatively treated ACJ instability. Further, we sought to reveal correlations between GS and ACJ-specific postoperative outcome measures. We hypothesized that PROMs have a relatively strong correlation with EDOMs and can be used independently and remotely in difficult-to-reach patients.

Methods

This study was carried out after receiving ethics committee approval and in accordance with national legal requirements. All procedures performed complied with the ethical standards of the institutional and/or national research committee and with the 1975 Declaration of Helsinki. Informed consent was obtained from all patients before their inclusion in the study. In order to evaluate the correlations between the individual scores, data on 131 consecutive patients who underwent operative treatment for ACJ instability between January 1, 2011, and December 31, 2017, at our institution were analyzed. During this period, all patients were invited to meet once with a board-certified physician to collect scores for the PROMs and EDOMs.

Patients were eligible for inclusion if they received surgical treatment for high-grade ACJ instability (Rockwood types 3-5) confirmed radiologically, were ≥18 years of age, and were willing and able to give informed consent to participate in the study. Any patient with a disease process that could preclude accurate evaluation (eg, neuromuscular, rheumatic, significant psychiatric or metabolic disorders, or amputation) was excluded.

All surgeries were performed by the arthroscopic department of the institution, which contained 5 certified shoulder and elbow surgeons. Patients were treated using either an “open” operative approach with the clavicle hook plate or an arthroscopic-assisted approach with a cortical fixation double-button system.

All outcome scores were collected postoperatively (as this was a retrospective observational study). All measures for the PROMs were provided by the patient, and all EDOMs were collected by a certified orthopaedic surgeon during the follow-up examination (R.-O.D.H., G.J.). GS scores that we evaluated were the CMS (an EDOM), SSV (a PROM), SST (a PROM), and visual analog scale (VAS) for pain (a PROM); and ACJ-specific scores were the Taft score (an EDOM), NCS (a PROM), ACJI score (an EDOM), and SSS (an EDOM). The NCS, SSS, ACJI score, and SSV were added to the follow-up examination in 2013 and were thus evaluated from this time point.

Statistical Analysis

Statistical analysis was performed using SPSS for Windows Version 26 (IBM Corporation). We presented the descriptive measures, such as mean, median, minimum and maximum, and standard deviation values, collected for every single score, to provide an overview of the data. To evaluate the associations between PROM and EDOM scores, the Pearson correlation coefficient (r) was calculated if a linear correlation between 2 scores could be assumed; if a nonlinear correlation was assumed, the Spearman rank correlation coefficient (rs) was used. According to the Cohen classification of effect size, measures were considered equivalent if the correlation coefficient was >0.9. Correlations <0.3 were considered weak, those from 0.3 to 0.49 were considered moderate, and those from 0.5 to 0.9 were considered strong. Statistical significance was set at P < .05.

Results

From the chosen database, 131 patients who had operatively treated ACJ instability between 2011 and 2017 were included in the study. Of this group, 101 patients were treated with an arthroscopic-assisted procedure, and 30 patients were treated with an open procedure. The mean follow-up time was 33.3 ± 19.2 months. Table 1 shows the characteristics of the patient collective. Table 2 shows all the postoperative outcome scores.

Table 1

Epidemiology of the Patient Collective (N = 131 Patients)

Variable	Value
Age, y, mean (range)	40 (18-73)
Sex, male/female, n	119/12
Follow-up, mo, mean ± SD	33.3 ± 19.2
ACJ instability type, n
Rockwood 3	55
Rockwood 5	76
Acute	90
Chronic	41
Surgery type, n
Arthroscopic	101
Open	30

ACJ, acromioclavicular joint.

Table 2

Postoperative Outcomes According to the Different Scoring Systems

Outcome Measure (Type)	Valid Scores, n b	Mean ± SD	Median	Range
GS measures
CMS (EDOM)	98	94.21 ± 7.81	95.79	66.32-100
SSV (PROM)	58	87.29 ± 12.58	90	25-100
SST (PROM)	106	10.5 ± 2.28	11	0-12
VAS pain (PROM)	131	1.9 ± 2.24	1	0-9.2
ACJ-specific measures
Taft score (EDOM)	122	9.5 ± 1.98	10	3-12
ACJI score (EDOM)	54	39.67 ± 32.72	20	0-100
SSS (EDOM)	55	3.2 ± 2.85	2	0-12
NCS (PROM)	58	82.38 ± 11.65	84	52-100

ACJ, acromioclavicular joint; ACJI, acromioclavicular joint instability; CMS, Constant-Murley score; GS, global shoulder; EDOM, examiner-dependent outcome measure; NCS, Nottingham Clavicle Score; PROM, patient-reported outcome measure; SSS, SICK Scapula Score; SST, Subjective Shoulder Test; SSV, Subjective Shoulder Value; VAS, visual analog scale.

The NCS, SSS, ACJI score, and SSV were added to the follow-up examination in 2013 and were thus evaluated from this time point.

Correlation Between GS Measures

Table 3 illustrates the correlation between PROMs and EDOMs for GS measures. There was a moderately negative and statistically significant correlation found between the VAS and CMS, and there was a strong correlation between the CMS and SSV (r = 0.59; P = .02) (Figure 1).

Table 3

Correlations Between PROMs and EDOMs for GS Scores

Comparison	Correlation	P Value
CMS (EDOM) vs SSV (PROM)	r = 0.59 (strong)	.02
CMS (EDOM) vs SST (PROM)	r = 0.25 (weak)	≤.03
CMS (EDOM) vs VAS (PROM)	r = −0.3 (moderate)	.03

CMS, Constant-Murley score; EDOM, examiner-dependent outcome measure; GS, global shoulder; PROM, patient-reported outcome measure; SST, Subjective Shoulder Test; SSV, Subjective Shoulder Value; VAS, visual analog scale.

Figure 1.

The Constant-Murley score showed a strong correlation with the Subjective Shoulder Value (SSV) (r = 0.59; P = .02).

Correlation Between GS and ACJ-Specific Measures

Table 4 illustrates the correlations between the CMS, a GS EDOM outcome measure, and the 4 ACJ-specific clinical outcome measures. A strong positive correlation was observed between the CMS and NCS (r = 0.79; P ≤ .001) as well as ACJI (rs = 0.69; P < .001) (Figure 2), and a strong negative correlation was seen between the CMS and SSS (r = –0.68; P < .001).

Table 4

Correlation Between the CMS and the ACJ-Specific Measures

Comparison to CMS	Correlation	P Value
NCS (PROM)	r = 0.79 (strong)	≤.001
ACJI score (EDOM)	rs = 0.69 (strong)	<.001
Taft score (EDOM)	r = 0.30 (moderate)	≤.003
SSS (EDOM)	r = −0.680 (strong)	<.001

ACJ, acromioclavicular joint; ACJI, acromioclavicular joint instability; CMS, Constant-Murley score; EDOM, examiner-dependent outcome measure; NCS, Nottingham Clavicle Score; PROM, patient-reported outcome measure; SSS, SICK Scapula Score.

Figure 2.

There was a strong correlation between the Constant score and the (A) Nottingham Clavicle Score (NCS) (r = 0.79; P ≤ .001) and (B) acromioclavicular joint instability (ACJI) score (rs = 0.69; P < .001).

Correlation Between the ACJI Score and ACJ-Specific Measures

Table 5 presents the correlations of the ACJI score with the Taft score, SSS, and NCS. Moderate correlations were seen between the ACJI and the SSS and between the ACJI and the NCS, and a strong correlation was seen between the ACJI and the Taft score (rs = 0.63; P ≤ .001).

Table 5

Correlation Between the ACJ-Specific Scores

Comparison	Correlation	P Value
ACJI score (EDOM) vs SSS (EDOM)	rs = 0.44 (moderate)	.014
ACJI score (EDOM) vs NCS (PROM)	rs = 0.33 (moderate)	≤.001
ACJI score (EDOM) vs Taft (EDOM)	rs = 0.63 (strong)	≤.001

ACJ, acromioclavicular joint; ACJI, acromioclavicular joint instability; EDOM, examiner-dependent outcome measure; NCS, Nottingham Clavicle Score; PROM, patient-reported outcome measure; SSS, SICK Scapula Score.

Discussion

The most important findings of this study are, first, the relatively strong correlation between the EDOM CMS (GS outcome measure) and the PROM SSV (GS outcome measure) and NCS (ACJ-specific outcome measure) in surgically managed ACJ injuries.

Second, there was a strong correlation between the CMS and the shoulder girdle–specific NCS (r = 0.79; P ≤ .001) and the ACJI (rs = 0.69; P <.001), and, notably, a strong negative correlation with the SSS (r = –0.68; P < .001) in patients with ACJ instability. Notably, however, the findings of this study contradict the 1992 stated paradigm of the need to include the CMS in all peer-reviewed shoulder-related studies. The presented data of this study showed a strong correlation between the global examiner-specific objective CMS and the validated ACJ-specific and strictly subjective PROM NCS.

These novel findings may provide researchers with more opportunities when choosing clinical outcome measurements according to the prevailing circumstances. Especially in rural areas, difficult-to-reach patient populations, or pandemic-related circumstances, there may now be a remote way to continue clinical studies of patients with ACJ instability and to fulfill the criteria of peer-reviewed studies. For example, a large patient population initially treated nonoperatively for ACJ instability could undergo remote follow-up evaluation by the treating physician and be scheduled for surgery if needed.

Charles et al validated the contents of the NCS, including a total of 10 questions, resulting in a sum of 100 points. The study group showed strong internal consistency, and the NCS was retested for its reliability in ACJ dislocation, sternoclavicular joint (SCJ) dislocation, ACJ arthritis, SCJ arthritis, and all fractures of the shoulder girdle.

A key difference between our study and that of Charles et al is that our study focused exclusively on ACJ instability. With respect to the observation made by Gilbart and Gerber that correlations are disease specific, our data seem to be more specific in all conditions. Additionally, the findings of this study suggest a strong correlation between the SSV and CMS. This finding also adds support to the conclusion of Gilbart and Gerber, who evaluated the association of the SSV with the CMS in cases of rotator cuff tears, arthroplasty, and shoulder instability. Interestingly, the correlation varied depending on the specific injury. Although the correlation was strong in the rotator cuff group, it was lower in the osteoarthritis and instability groups, and Gilbart and Gerber concluded there was a need for disease-specific outcome measures because of so-called deficiencies in the frequently used outcome scores.

Based on these results, it seems that the rather simple SSV, which is defined as the patients’ subjective percentage estimation of the shoulder function compared with their healthy shoulder, is equally usable as the CMS in the setting of ACJ instability. This finding may be used to develop a scoring profile that incorporates both the NCS and the SSV for difficult-to-reach patient populations with ACJ instability, thus adopting 2 PROMs rather than EDOMs and dividing them into 1 GS score and ACJ-specific shoulder score.

It should be noted that the second most important finding of this study contrasts with the hypothesis of Kirkley et al that the sheer impact of the CMS resulted in excellent values in instability cases despite poor results, questioning the impact of the CMS regarding ACJ instability. Thus, on the basis of the results of this study, the CMS is applicable for ACJ instability.

An additional finding of this study is the correlation of EDOMs as either GS scores or shoulder girdle–specific scores. We noted relatively strong correlations between the CMS and the SSS and the ACJI score, a moderate correlation between the CMS and the Taft score (r = 0.30; P ≤ .003), and a strong correlation between the ACJI score and the Taft score (rs = 0.63; P ≤ .001). Thus, one could assume that if the examiner wants to use only 1 EDOM for his or her clinical study, then the ACJI score could be a viable option.

Moderate correlations between the ACJI score and the NCS as well as the SSS prohibit us from offering a recommendation based on our data; however, PROMs such as the NCS and SSV are able to be substituted for EDOMs in patients with operatively treated ACJ instability, representing a valid alternative for difficult-to-reach patient cohorts in which demographics as well as specific circumstances prohibit examiner-specific assessments. Further prospective and more specific studies are necessary to prove these preliminary findings.

Limitations

This study has several limitations. First, it was conducted at a level 1 trauma center with a focus on shoulder arthroscopy. Consequently, the majority of patients were treated using arthroscopy by 5 different board-certified surgeons of the shoulder and elbow unit. With regard to the retrospective study design, not all clinical scores were obtained from every included patient. Further, the NCS, SSS, ACJI score, and SSV were added to the postoperative shoulder evaluation at a later point, in 2013. Because of the main goal of this study to illustrate the correlation between the scores, all available data were included. Last, because of the preliminary characteristics of this study, only patients with operatively treated ACJ instability were evaluated, with data on nonoperatively managed cases missing. With the knowledge gained from this study, further prospective and more specific multicenter studies are needed to prove these data.

Conclusion

Based on the results of this study, PROMs like the SSV (a GS score) and the NCS (an ACJ-specific score) may be substituted for EDOMs.