Inter-rater agreement among shoulder surgeons on treatment options for proximal humeral fractures among shoulder surgeons.

BACKGROUND: The treatment approach for proximal humeral fractures is determined by various factors, including patient age, sex, dominant arm, fracture pattern, presence of osteoporosis, preexisting arthritis, rotator cuff status, and medical comorbidities. However, there is a lack of consensus in the literature regarding the optimal treatment for displaced proximal humeral fractures. This study aimed to assess and quantify the decision-making process for either conservative or surgical treatment and the choice of surgical method among shoulder surgeons when treating proximal humeral fractures.
METHODS: Forty sets of true anteroposterior view, scapular Y projection view, and three-dimensional computed tomography of proximal humeral fractures were provided to 12 shoulder surgeons along with clinical information. Surveys regarding Neer classification, decisions between conservative and surgical treatments, and chosen methods were conducted twice with an interval of 2 months. The factors affecting the treatment plans were also assessed.
RESULTS: The inter-rater agreement was fair for Neer classification (kappa=0.395), moderate for the decision between conservative and surgical treatments (kappa=0.528), and substantial for the chosen method of surgical treatment (kappa=0.740). The percentage of agreement was 71.1% for Neer classification, 84.6% for the decision between conservative and surgical treatment, and 96.4% for the chosen method of surgical treatment. The fracture pattern was the most crucial factor in deciding between conservative and surgical treatments, followed by age and physical activity.
CONCLUSIONS: The decision between conservative and surgical treatment for proximal humeral fractures showed good agreement, while the chosen method between osteosynthesis and arthroplasty showed substantial agreement among shoulder surgeons.

INTRODUCTION

Proximal humeral fractures are the third most common osteoporotic fractures in elderly patients after spine and wrist fractures [1]. Although most fractures can be treated conservatively with good clinical results [2,3], complex fractures may require surgical treatment to achieve anatomic alignment, early mobilization, and functional recovery. For surgical stabilization of these fractures, the use of a locking plate is gaining attention. Although clinical and radiological results are promising with locking plate fixation [4], the relatively high complication rate with this method can make the surgical stabilization of proximal humeral fractures in an osteoporotic bone challenging [5].

The treatment approach for proximal humeral fractures is determined by various factors, including patient age, sex, dominant arm, fracture pattern, presence of osteoporosis, preexisting arthritis, rotator cuff status, and medical comorbidities [6]. The severity of the fracture itself is the single most important determinant of the choice of intervention. However, even Neer classification of proximal humeral fractures appears to show low inter-rater reliability in most studies [7-10]. Moreover, similar fracture patterns can be treated differently according to the preference or experience of the treating surgeon [11-13]. Even for displaced proximal humeral fractures in elderly patients, there currently exists a wide variation in the method of surgical treatment, such as arthroplasty or osteosynthesis. As the incidence of osteoporotic fractures is increasing in aged populations, there is poor consensus on the optimal treatment for displaced proximal humeral fractures in the literature. Therefore, a study of current practice patterns is essential because these are important public health issues, and the results of such an investigation would be very helpful to surgeons for future treatment planning.

Therefore, the purpose of this study was to assess and quantify the decision-making process for either conservative or surgical treatment and the choice of surgical methods among shoulder surgeons when treating proximal humeral fractures. The researchers hypothesized that there would be variability of more than 10% regarding the decision to use surgical or conservative treatment among surgeons.

METHODS

The study was approved by the Institutional Review Board of Inje University Ilsan Paik Hospital (IRB No. 2017-05-002). An agreement of the patient was exempted as it was a retrospective study. The data of patients who visited the emergency room and outpatient clinic of a single institution from March 1, 2014, to July 31, 2016, and were diagnosed with a proximal humeral fracture were consecutively collected and retrospectively reviewed. Among them, 118 patients aged 18 years and above were independently evaluated by an assessor who was not involved in their diagnosis or treatment.

Sample size calculations revealed 40 cases showing 10% inter-rater variability in terms of surgical and conservative treatment choices among raters, assuming a 5% significance level and 80% power. Forty sets of true anteroposterior (AP) view, scapular Y projection view, and three-dimensional (3D) computed tomography (CTs), which could indicate the detailed fracture patterns, were selected via general consensus of the authors to represent the full spectrum of proximal humeral fracture patterns from the 118 eligible patients. In cases of fracture dislocation, both initial and post-reduction radiographs were provided. For the selected 40 sets, medical records were reviewed for factors that could affect treatment decisions, including age, sex, injury mechanism, rheumatic disease, dementia, and medical comorbidities. All patient information other than clinical data was removed.

Twelve fellowship-trained shoulder surgeons who practiced in different university hospitals and had at least 5 years of experience were invited to participate in the study. Multiple-choice questions related to diagnosis and treatment for all 40 study subjects were surveyed, with information regarding clinical parameters including age, sex, injury mechanism, and medical comorbidities. A copy of Neer classification of proximal humeral fractures was also provided as a reference to all of the participating surgeons. The principal investigator for this study did not participate in the questionnaire survey.

The true AP view, scapular Y view, and 3D CTs were saved as DICOM files in the PACS system (M-view 5.4; Marotech, Seoul, Korea) (Fig. 1). For the cases with a fracture dislocation, the post-reduction true AP view and scapular Y view CTs were added as well. The raters were allowed to observe the radiographic data using RadiAnt DICOM viewer 4.0.2 (32 bit; Medixant, Poznan, Poland). The surveys were repeated 2 months later using a different ordering of the patients to assess for any differences with the same rater. All 12 participating surgeons completed the first survey, and nine completed both surveys.

Fig. 1.

Typical plain radiographs showing significant variations among the participating surgeons in terms of Neer classification. (A) True anteroposterior view. (B) Trans-scapular Y projection view. (C) Three-dimensional computed tomography.

Each rater answered the following questions for each of the 40 proximal humeral fractures in the study cohort: (1) What is the Neer classification? (2) Will you perform surgical or conservative treatment? (3) Which of the following factors are the two most important in determining whether to use surgical or conservative treatment? (a) Age and activity level; (b) fracture severity; (c) osteoporosis; (d) medical comorbidity; or (e) arm dominance. For the patients for whom surgical treatment was chosen, the following questions were asked: (1) What specific surgical procedures would you choose? (a) Closed reduction (CR) and pinning; (b) CR and intramedullary (IM) nail fixation; (c) osteosynthesis using plates and screws; (d) hemiarthroplasty; (e) reverse total shoulder arthroplasty; (f) minimally invasive plate osteosynthesis (MIPO); or (g) arthroscopic fixation. (2) How long would you immobilize the patient after surgery? (3) Which immobilization method would you apply after surgery? (4) What are the two most important factors in determining the surgical method? (a) Age and activity level; (b) fracture severity; (c) osteoporosis; (d) medical comorbidity; or (e) cuff status.

If conservative treatment was chosen, the following questions were asked: (1) Which immobilization method would you apply? (2) When would you conduct your next follow-up radiographic examination? (3) For how long would you immobilize the patient? Specific instructions for determining the need for surgery were not provided to the raters, and they were allowed to use the same criteria that they use in their own clinical practice. The demographic information of each rater, including age, sex, and years of practice since fellowship training, was collected and analyzed.

Statistical Analysis

The inter-rater agreement concerning each survey question, including Neer classification, treatment options, surgical methods, and immobilization methods, was evaluated. The inter- and intra-rater reliabilities using Fleiss kappa for each domain were analyzed using R ver. 3.4.0. (The R Foundation for Statistical Computing, Vienna, Austria; https://www.r-project.org/) [14,15]. The generated kappa values were interpreted according to the guidelines of Landis and Koch (i.e., 0.01–0.20, slight agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement; 0.61–0.80, substantial agreement; and 0.81–0.99, almost perfect agreement). For the statistical analysis of surgical methods, the patients who were selected for surgery by more than one rater were chosen, and the surgical methods were roughly categorized as arthroplasty or osteosynthesis. The first radiological follow-up was analyzed after categorization into less than 2 weeks or 2 weeks. Among the variables, such as age, sex, injury mechanism, medical comorbidities (compromised cardiopulmonary system, renal failure, and stroke history), and Neer classification, analysis was performed to determine which factor would affect the treatment option (conservative or surgical treatment) using the Cochran-Mantel-Haenszel test for stratified categorical data.

RESULTS

A total of 12 surgeons participated in this study. All of them were male and had a subspecialty in shoulder surgery and a mean age of 43.4 years (range, 38–48 years). The mean duration of experience after fellowship training was 7.8 years (range, 4–12 years). The 40 proximal humeral fracture cases selected retrospectively for the study survey comprised cases from 27 female and 13 male patients with a mean age of 64.7±17.8 years. Neer classification of the study cohort indicated a one-part fracture in 16.0%, two-part fracture in 52.9%, three-part fracture in 21.3%, and four-part fracture in 9.8% of the patients.

With respect to Neer classification, the percentage of agreement was 71.1% in the first review and 76.9% in the second review. Inter-rater agreement was fair to moderate (kappa=0.395 in the first review and kappa=0.417 in the second review), while the intra-rater agreement was substantial (kappa=0.744) (Table 1). With respect to treatment choices, the mean percentage of agreement was 84.6% in the first review and 86.4% in the second review. The inter-rater agreement was moderate (kappa=0.528 in the first review and kappa=0.549 in the second review), while the intra-rater agreement was almost perfect (kappa=0.879) (Table 2).

Table 1.

Inter-rater agreement and the kappa coefficient and intra-rater agreement for Neer classification

Variable	No. of patients (%)	Inter-rater agreement		Intra-rater agreement[*]
		Mean percentage of agreement (range)	Fleiss’ kappa	Kappa (mean)
First review[†]		71.1 (41.7–100.0)	0.395	0.744
1	77 (16.0)
2	254 (52.9)
3	102 (21.3)
4	47 (9.8)
Second review[‡]		76.9 (44.4–100.0)	0.417	0.744
1	42 (11.7)
2	203 (56.4)
3	75 (20.8)
4	40 (11.1)

Nine raters evaluated the intra-rater agreement;

Twelve surgeons participated;

Nine surgeons participated.

Table 2.

Inter-rater agreement and its kappa coefficient and intra-rater agreement for the decision between conservative and surgical treatment

Variable	No. of patients (%)	Inter-rater agreement		Intra-rater agreement[*]
		Mean percentage of agreement (range)	Fleiss’ kappa	Kappa (mean)
First review		84.6 (50.0–100.0)	0.528	0.879
Conservative treatment	186 (38.8)
Surgical treatment	294 (61.3)
Second review		86.4 (55.6–100.0)	0.549	0.879
Conservative treatment	144 (40.0)
Surgical treatment	216 (60.0)

Nine raters evaluated intra-rater agreement.

Seven of the 12 participating surgeons responded that fracture pattern or severity was the most crucial factor in deciding between conservative and surgical treatment for a proximal humeral fracture: of these seven surgeons, five regarded age and activity level as the second most important factors. The remaining five raters indicated age and activity level as the most crucial factors and the fracture pattern or severity as the second most important factor. One rater indicated that medical comorbidities were the second most important factor, while another rater listed the dominant arm as such.

When surgical treatment was chosen, CR and pinning was the method of choice in 3.1% of the patients, CR and IM nail fixation in 9.9%, open reduction internal fixation in 59.5%, hemiarthroplasty in 3.4%, reverse total shoulder arthroplasty in 15.6%, MIPO in 5.1%, and arthroscopic fixation in 3.4%. When the surgical options were stratified into osteosynthesis and arthroplasty, 56 (19.05%) out of 294 patients were chosen for arthroplasty and 238 (80.95%) for osteosynthesis in the first review. The percentage of agreement was 96.4% in the first review and 96.2% in the second review. The inter-rater agreement was substantial for these reviews, with a mean kappa value of 0.740 and 0.727, respectively.

All surgeons indicated that they would apply immobilization even after surgical treatment, for a mean of 4.2 weeks (median, 4 weeks; range, 2–6 weeks). In most cases, abduction brace application was the indicated method for immobilization (205 of 294 cases), with a mean percentage of agreement of 73.0%. The second radiographic examination after the surgery would have been performed at 2 weeks, with a mean percentage of agreement of 76.5% or 70.4%, although the kappa coefficient showed only a slight agreement (Table 3).

Table 3.

Inter-rater agreement for surgery

Variable	No. of patients (%)	Inter-rater agreement[*]
		Mean percentage of agreement (range)	Fleiss’ kappa
Surgical option
First review		96.4 (58.3–100.0)	0.740
Arthroplasty	56 (19.0)
Osteosynthesis	238 (81.0)
Second review		96.1 (66.7–100.0)	0.727
Arthroplasty	46 (21.3)
Osteosynthesis	170 (78.7)
Immobilization method
First review		73.0 (33.3–100.0)	-[†]
Arm sling	35 (11.9)
Velpeau sling	53 (18.0)
Abduction brace	205 (69.7)
Splint	1 (0.3)
Second review		80.6 (33.3–100.0)	-[†]
Arm sling	7 (3.2)
Velpeau sling	45 (20.8)
Abduction brace	164 (75.9)
Splint	1 (0.5)
Next radiographic evaluation (day)
First review		76.5 (50.0–100.0)	0.004
<14	71 (24.1)
14	223 (75.9)
Second review		70.4 (50.0–100.0)	-0.080
<14	64 (29.6)
14	152 (70.4)
Immobilization period (wk)
First review		51.0 (33.3–100.0)	-[†]
2	55 (18.7)
3	22 (7.5)
4	120 (40.8)
6	97 (33.0)
Second review		49.2 (33.3–100.0)	-[†]
2	17 (7.9)
3	23 (10.6)
4	91 (42.1)
5	9 (4.2)
6	76 (35.2)

The inter-rater agreement was analyzed only in cases where more than one rater selected the surgical treatment option;

The kappa coefficient could not be calculated.

When conservative treatment was chosen, the surgeons indicated that the shoulders would have been immobilized using an arm sling in 5.9%, a Velpeau sling in 21.5%, an abduction brace in 65.6%, and a splint in 7% of the patients. The suggested mean duration of immobilization for conservative treatment was slightly longer than that after surgical treatment, with a mean value of 4.7 weeks (median, 4 weeks; range, 2–8 weeks). The suggested mean time of the next radiographic follow-up was 8.6 days (median, 7 days; range, 3–14 days) (Table 4).

Table 4.

Inter-rater agreement in cases of conservative treatment

Variable	No. of patients (%)	Inter-rater agreement[*]
		Mean percentage of agreement (range)	Fleiss’ kappa
Immobilization method		70.9 (36.4–100.0)	-[†]
Arm sling	11 (5.9)
Velpeau sling	40 (21.5)
Abduction brace	122 (65.6)
Splint	13 (7.0)
Next radiographic evaluation (day)		76.4 (50.0–100.0)	-0.054
<14	136 (73.1)
14	50 (26.9)
Immobilization period (wk)		50.8 (33.3–83.3)	-[†]
2	4 (2.2)
3	34 (18.3)
4	65 (34.9)
5	3 (1.6)
6	77 (41.4)
8	3 (1.6)

The inter-rater agreement was analyzed only in cases where more than one rater selected the surgical treatment option;

The kappa coefficient could not be calculated.

The study also investigated whether age, sex, injury mechanism, medical comorbidities (compromised cardiopulmonary system, renal failure, and stroke history), and fracture classification affected the choice of conservative or surgical treatment but only found a significant association with respect to Neer classification (p<0.001) (Table 5). The medical comorbidities could not be analyzed, and multivariate analysis could not be performed because of the small sample size.

Table 5.

Correlation between Neer classification and treatment options

Neer classification	Conservative treatment	Surgical treatment	p-value[*]
1	71 (38.2)	6 (2.0)	<0.001
2	100 (53.8)	154 (52.4)
3	15 (8.1)	87 (29.6)
4	0	47 (16.0)

Values are presented as number (%).

Statistical significance was tested using the Cochran-Mantel-Haenszel test.

DISCUSSION

The present study showed that the inter-rater agreement regarding treatment decisions and surgical options for proximal humeral fractures was moderate and substantial, respectively. The percentage of agreement for assessing the fracture type and treatment decisions ranged from 71.1% to 86.4%. These findings support the study hypothesis that there would be variability of more than 10% among surgeons for the decision on the fracture type and treatment options.

Previous studies have already shown that the reproducibility or agreement for Neer classification of proximal humeral fractures is low. The kappa values in these studies ranged from somewhere between 0.183 and 0.633 using radiography and two-dimensional or 3D CT scans [10,12,13,16,17]. Thus, several alternative classifications have been introduced [8,18,19]. Nevertheless, Neer classification is still used in clinical practice because it provides convenience in terms of communication and overall guidelines for surgical indication.

The present study provided 12 specialist shoulder surgeons with 3D CTs and clinical information as well as plain radiographs for a retrospective cohort of 40 proximal humeral fracture cases. These clinicians were then asked to determine Neer classification and advise on the treatment plans they would choose. As expected, inter-rater agreement with regard to Neer classification was similar to that reported in previous studies, with a mean kappa value ranging from 0.395 to 0.417. The choice of surgical or conservative treatment showed a similarly low level of agreement as in previous reports. However, when the percentage of agreement for individual questions was evaluated, which is an intuitive way to perform a reliability test, relatively high levels of agreement were observed (i.e., a 71% level of agreement for Neer classification); this indicated that 8.5 out of the 12 (71% of the 12) surgeon raters agreed on Neer classification for the same fracture pattern, although the kappa statistics showed only a slight or fair reliability. Similarly, the percentage of agreement was 84%–86% for the choice of conservative versus surgical treatment, while the kappa statistics demonstrated moderate agreement (0.528–0.549). Furthermore, the inter-rater agreement for the surgical method, arthroplasty versus osteosynthesis, was very high (approximately 96%) and substantial (with kappa values of 0.740 or 0.727). Contrary to what has been proposed in previous studies [7-13], these observations might suggest that there may be some consensus regarding the diagnosis and treatment of osteoporotic proximal humeral fractures. Even if there is a discrepancy in fracture classifications or the selection of conservative and surgical treatments, the choice of the surgical method between arthroplasty and osteosynthesis showed relatively high agreement among the surgeons in this study. Therefore, the treatment plan for these fractures could be similar among surgeons, even though the fracture pattern is assessed differently.

In the study survey, most of the surgeons preferred osteosynthesis over arthroplasty, even in complex cases, such as four-part fractures, although all 12 participating surgeons had a shoulder subspecialty. Reverse arthroplasty has gained more popularity recently, and some reports have indicated that this method can even be superior to osteosynthesis for treating fractures [20,21] . Notably however, it appears that most of the shoulder surgeons in this study weighed the benefits of a well-restored anatomy by osteosynthesis against the higher risk of complications with arthroplasty [22,23].

In contrast to previous reports, this study found that osteoporosis and medical comorbidities were not regarded as significant factors for determining the treatment plans [24]. Okike et al. [24] reported that patients with medical comorbidities and osteoporosis commonly underwent surgical intervention for arthroplasty. However, all of the surgeons surveyed in the present study would have applied immobilization even after surgical treatment. The indicated duration of this immobilization was also similar to that for conservative treatments. This might suggest that even if osteoporosis were not a crucial consideration during treatment decision-making, all of the surgeons surveyed in the present study had concerns about a possible secondary reduction loss caused by osteoporosis. Most of them seemed to give greater weight to protection than early mobilization, even when choosing a surgical treatment.

Previous studies have reported good clinical outcomes with conservative treatment of proximal humeral fractures [1,25] and that even displaced fractures can be treated successfully with a nonoperative approach [3]. All surgical options for proximal humeral fractures, including IM nailing, open osteosynthesis, and arthroplasty, have yielded favorable outcomes [26-30]. However, the currently available evidence from randomized controlled trials is insufficient to guide decision-making between different non-surgical and surgical interventions for these fractures [2]. The current study found substantial variabilities in fracture classification, treatment planning, surgical methods, and immobilization methods and duration. Thus, individually tailored treatment plans should be adopted with consideration for the fracture pattern but also patient-related factors and the surgeon’s experience and preferences.

This study had certain limitations. The principal aim of the study was to determine the trends in decision-making for the treatment of proximal humeral fractures among shoulder surgeons. Thus, the specific treatment that the included patients actually received was not investigated. Furthermore, the small number of patients and involved surgeons limited the power of the statistical analysis regarding the true factors affecting the choice of treatment. However, the researchers provided the participating surgeons with relevant clinical information on the retrospective cases, including age, sex, and medical comorbidities, in addition to radiographs and 3D CTs. This matches the information that would be available in an actual clinical setting.

Although Neer classification of proximal humeral fractures did not demonstrate high agreement among the treating surgeons, the choice of conservative versus surgical treatment showed good agreement. Furthermore, the choice between osteosynthesis and arthroplasty showed a substantial agreement among the shoulder surgeons.