Reliability of the measurement of glenoid bone defect in anterior shoulder instability.

BACKGROUND: The size of the glenoid bone defect is an important index in selecting the appropriate treatment for anterior shoulder instability. However, the reliability of glenoid bone defect measurement is controversial. The purpose of the present study was to investigate the reliabilities of measurements of the glenoid bone defect on computed tomography and to explore the predisposing factors leading to inconsistency of these measurements.
METHODS: The study population comprised 69 consecutive patients who underwent surgery for recurrent anterior shoulder dislocation in Peking University Fourth School of Clinical Medicine from March 2016 to January 2017. The glenoid bone defect was measured by three surgeons on 'self-confirmed' and 'designated' 3-D en-face views, and repeated after an interval of 3 months. Measurements included the ratio of the defect area to the best-fit circle area, and the ratio of the defect width to the diameter of the best-fit circle. The inter- and intra-observer reliabilities of the measurements were evaluated using intraclass correlation coefficients (ICCs). The maximum absolute inter- and intra-observer differences and the cumulative percentages of cases with inter- and intra-observer differences greater than these respective levels were calculated.
RESULTS: Almost all linear defect values were bigger than the areal defect values. The inter-observer ICCs for the areal defect were 0.557 and 0.513 in the 'self-confirmed' group and 0.549 and 0.431 in the 'designated' group. The inter-observer reliabilities for the linear defect were moderate or fair in the 'self-confirmed' group (ICC = 0.446, 0.374) and 'designated' group (ICC = 0.402, 0.327). The ICCs for intra-observer measurements were higher than those for inter-observer measurements. The respective maximum inter- and intra-observer absolute differences were 13.9% and 13.2% in the 'self-confirmed' group, and 15.8% and 9.8% in the 'designated' group.
CONCLUSIONS: The areal measurement of the glenoid bone defect is more reliable than the linear measurement. The reliability of the glenoid defect areal measurement is moderate or worse, suggesting that a more accurate and objective measurement method is needed in both en-face view and best-fit circle determination. Subjective factors affecting the glenoid bone loss measurement should be minimized.

Introduction

The incidence of anterior shoulder dislocation in the general population is reportedly 21.9 per 100,000.[ However, in certain professions the incidence of anterior shoulder dislocation is between 164.4 and 2800 per 100,000.[ With non-operative treatment, 58% to 100% of these patients will experience recurrence.[ Of the shoulders with recurrent anterior shoulder dislocation, 90% have an abnormal glenoid configuration.[ The extent of the glenoid bone defect is an important index that affects the selection of the appropriate surgical procedures. Thus, it is most important to pre-operatively measure the glenoid bone defect accurately with good reliability. However, there is no consensus about the reliability of inter- and intra-observer measurements of the glenoid bone defect. The purpose of the present study was to investigate the reliabilities of measurements of the glenoid bone defect on computed tomography (CT) and to explore the predisposing factors leading to inconsistency of these measurements.

Methods

Ethical approval

The present study was approved by the ethics committee of Peking University Fourth School of Clinical Medicine prior to study commencement. Informed consent for study inclusion and treatment was obtained from all patients.

Patients

The study population comprised 69 consecutive patients who were diagnosed with recurrent anterior shoulder dislocation and underwent surgery in Peking University Fourth School of Clinical Medicine from March 2016 to January 2017. The inclusion criteria were: (1) anterior shoulder instability with recurrent dislocations (≥2), (2) patients’ age ≥14.0 years. The exclusion criteria were: (1) incomplete pre-operative 3-D CT data, (2) obvious shoulder osteoarthritis, (3) bony Bankart lesion, (4) glenoid malformation, and (5) previous surgery on the affected shoulder.

Glenoid bone loss measurement

All patients underwent pre-operative CT imaging using a Toshiba Aquilion 80-slice CT scanner (Toshiba Medical Systems Corp., Tokyo, Japan). The scan parameters were: 120 kVp, 125 mAs, 1 mm slice thickness, 50 cm field of view, and a matrix size of 512 × 512 in spiral and standard reconstructions. Three attending doctors who had completed shoulder fellowship training (henceforth referred to as observers a, b, and c) used Mimics (version 19.0, Materialise NV, Leuven, Belgium) to reconstruct 3-D images with humeral head subtraction. The 3-D scapula was moved until the surface of the glenoid was parallel to the screen of the computer in accordance with each observer's subjective assessment; this view was called the en-face view. A screenshot was imported into ImageJ (version 1.52a, Wayne Rasband, National institutes of Health, USA). Each of the three doctors then individually confirmed the en-face view of each glenoid; these views were called the “self-confirmed” en-face views [Figure 1]. The inferior portion of the healthy glenoid contour reportedly approximates a true circle.[ ImageJ was used to draw an outer best-fit circle based on the inferior portion of the glenoid, and the area of the circle was measured [Figure 1]. “Edit–Clear Outside” was then selected to produce an image in which the area outside of the circle was black [Figure 2]. The colored image was changed into a 32-bit grayscale image, and the upper and lower threshold gray values were adjusted. When the glenoid part inside the circle became red (default color), the area of the red region was measured [Figure 2]. The percentage of the areal glenoid defect was calculated using the formula: (A−B)/A × 100%, where A was the area of the outer best-fit circle based on the inferior portion of the glenoid, and B was the area of the glenoid within the circle.

Figure 1

3-D computed tomography image of the scapula showing the en-face view.

Figure 2

Image obtained using ImageJ in which the red color indicates the area of the glenoid within the best-fit circle.

The percentage of the linear glenoid defect was measured by creating a line along the anteroinferior glenoid rim that intersected the best-fit circle [Figure 3]. The length of the line was measured using the “Measure” tool. The diameter of the circle was then read directly from the toolbar. In accordance with the Pythagorean theorem, the percentage of the linear glenoid defect was calculated as: /d, where d was the diameter of the best-fit circle and l was the length of the line segment along the anterior glenoid rim.

Figure 3

Image showing the line along the anteroinferior glenoid rim that intersects the best-fit circle.

An independent surgeon who was not one of the three observers then examined the CT images and created en-face views for all 69 patients; these views were called the “designated” en-face views. The three observers repeated all measurements on these “designated” en-face views.

All procedures were repeated after an interval of 3 months. All three observers were blinded to the patients’ diagnoses and surgical treatments.

Statistical analysis

The inter- and intra-observer reproducibility of the measurements was assessed by intra-class correlation coefficient (ICC) analyses using a two-way random-effects model and evaluation of absolute agreement. The ICC values ranged from 0 to 1, where 1 indicated perfect reliability. The strength of the reliability was classified as very good (ICC ≥ 0.80), good (0.60 ≤ ICC < 0.80), moderate (0.40 ≤ ICC < 0.60), fair (0.20 ≤ ICC < 0.40), or poor (ICC < 0.20).[ The inter- and intra-observer absolute differences between the “self-confirmed” and “designated” en-face view groups were described and analyzed. Statistical analysis was performed with SPSS (version 23.0; IBM Corp, Chicago, IL, USA). Sample size was established with PASS (version 15.0.5, NCSS, LLC. Kaysville, UT, USA) for reliability studies using ICCs. P < 0.05 was considered statistically significant.

Results

A reliability hypothesis with a 0.05 significance level requires a sample of at least eight cases to test the intra-observer reproducibility of two observations for an estimated sample intra-class correlation of 0.7, and a minimum sample of five cases to test the inter-observer reliability of observations from three observers for an estimated sample intra-class correlation of 0.6. Therefore, 69 cases were adequate to enable definitive conclusions to be drawn from the present study.

The included patients comprised 57 (82.6%) males and 12 (17.4%) females with a mean age of 31.0 ± 9.5 years (range 16.0–57.0 years). The affected shoulder was the left in 26.1% (18 cases) and the right in 73.9% (51 cases). The surgical procedure was Bankart soft tissue repair in 20 cases (29%) and the Latarjet procedure in 49 cases (71%).

Almost all the linear defect values were bigger than the corresponding areal defect values. The inter-observer reliability is shown in Table 1. The inter-observer reliabilities for the areal defect were moderate in the “self-confirmed” en-face view group (ICC = 0.557, 0.513) and “designated” en-face view group (ICC = 0.549, 0.431). The inter-observer reliabilities of the linear defect values were fair to moderate in the “self-confirmed” en-face view group (ICC = 0.446, 0.374) and “designated” en-face view group (ICC = 0.402, 0.327).

Table 1

Inter-observer reliability of measurements among three observers.

The intra-observer reliabilities of the measurements performed by each observer are shown in Table 2. The ICC values of the inter- and intra-observer areal and linear measurements significantly differed between the ‘self-confirmed’ and ‘designated’ groups (P < 0.05).

Table 2

Intra-observer reliability of two measurements by each observer.

The cumulative percentages of cases with inter-observer differences greater than 10%, 7%, and 5% of the areal glenoid defect and the maximum areal glenoid defect differences in the “self-confirmed” and “designated” en-face view groups are shown in Tables 3 and 4. The maximum inter-observer difference was 13.9% in the “self-confirmed” en-face view group and 15.8% in the “designated” en-face view group. Inter-observer differences of greater than 5% occurred in 24.6% to 55.1% and 10.1% to 26.1% of cases in the “designated” en-face view group and the “self-confirmed” en-face view group, respectively.

Table 3

Cumulative percentages of cases with different levels of inter-observer differences and the maximum differences in the ‘self-confirmed’ en-face view group.

Table 4

Cumulative percentages of cases with different levels of inter-observer differences and the maximum differences in the ‘designated’ en-face view group.

The cumulative percentages of cases with intra-observer differences of greater than 10%, 7%, and 5% of the areal glenoid defect and the maximum areal glenoid defect differences in the “self-confirmed” and “designated” en-face view groups are shown in Tables 5 and 6. The maximum intra-observer differences were 13.2% in the “self-confirmed” en-face view group and 9.8% in the “designated” en-face view group. Intra-observer differences of greater than 5% occurred in 2.9% to 18.8% and 2.9% to 23.2% of cases in the “designated” en-face view group and the “self-confirmed” en-face view group, respectively.

Table 5

Cumulative percentages of cases with different levels of intra-observer differences and the maximum differences in the ‘self-confirmed en-face view’ group.

Table 6

Cumulative percentages of cases with different levels of intra-observer differences and the maximum differences in the ‘designated en-face view’ group.

Discussion

Many methods of measuring the glenoid bone defect have been described in the literature.[ These methods are based on imaging modalities such as plain photographs, CT, and magnetic resonance imaging (MRI), and on different computational principles including areal and linear defects. Some authors have used the bare spot[ or arthroscopic center[ as a reference for intraoperative measurements of the glenoid bone defect. Glenoid bone defect measurement is most commonly done using the 3-D CT en-face view with humeral head subtraction, which is considered to produce more accurate and reliable measurements than plain photography and MRI.[

There are two methods used to perform glenoid bone defect measurements on the en-face view image. One method is to calculate the glenoid bone defect as the ratio of the area of the bone fragment to the area of an assumed best-fit circle based on the inferior part of each glenoid[; the other method is to calculate the ratio of the defect width to the diameter of a best-fit circle. Some authors believe that linear measurement is inaccurate[ and might overestimate the size of the defect.[ Our results corroborated these findings. The inter-observer reliabilities showed moderate agreement for areal defect measurements, and fair to moderate agreement for linear defect measurements. Most intra-observer reliabilities showed good agreement for areal defect measurements, and moderate or good agreement for linear defect measurements. The anterior rim of the glenoid with an anterior bone defect is not always straight and the contour can be quite variable. Therefore, measurements of the linear defect in different directions could lead to substantial variability in the evaluation of the size of the glenoid bone defect.

There is currently no consensus about the reliability of areal measurement of glenoid bone loss. One study reported good inter-observer reliability (ICC = 0.667) and intra-observer reliability (ICC = 0.723) for glenoid bone loss measurement based on 3-D CT assessment,[ and two other studies reported very good inter-observer reliability (ICC = 0.90) and intra-observer reproducibility (ICC = 0.94).[ However, another study reported that the glenoid bone defect area and diameter were significantly affected by scapular tilt changes in the en-face view[; furthermore, the overall agreement regarding en-face view selection between three observers was only 30% with a 0.10 K-alpha value, and both the areal and linear defects significantly differed between measurements performed using the conventional and “spoon” techniques to place the best-fit circle.[ Our results also showed only moderate inter-observer reliability (ICC = 0.513, 0.557) and moderate to good intra-observer reliability (ICC = 0.585 to 0.713) for areal glenoid bone defect measurement. To investigate the reliability of the best-fit circle alone, we also measured the glenoid bone defect on the “designated” en-face view determined by another independent observer. The result showed similar moderate inter-observer reliabilities (ICC = 0.549, 0.431) and good intra-observer reproducibility (ICC = 0.611 to 0.783) for areal glenoid bone defect measurements. This suggests that in addition to the bias caused by en-face view determination, the best-fit circle measurement also played an important role in producing measurement variations among observers.

Although the glenoid bone defect measurements used in the present study achieved a moderate to good degree of reliability, the large inter- and intra-observer variation in the measurements of certain patients was substantial enough to affect the selection of surgical procedures. In the present study, the largest absolute difference in glenoid bone defect measurement between observers was 15.8%, and the percentage of patients that had an absolute difference in glenoid bone defect measurement of more than 5% was as high as 55.1%. A 5% difference in glenoid bone loss evaluation could easily change the selected procedure from a soft tissue reconstructive surgery into a bone block procedure.

The present study had several limitations. First, the study assessed only the reliability of the qualification of the glenoid bone defect without considering the location and type of defect, which might affect the measurement reliability. Second, variation in the training and experience of the three observers might be one of the reasons for the inter-observer variations in measurement.

In conclusion, it is more reliable to measure the glenoid defect using areal measurement than linear measurement. However, the reliability of the measurements of areal bone defect is moderate or worse. Thus, there is a need for the development of a more accurate and objective measurement method in the determination of the en-face view and best-fit circle. Subjective factors affecting the measurement of glenoid bone loss should be eradicated as much as possible.

Acknowledgement

We thank the experts from the Department of Radiology and Epidemiology for providing CT data and statistical analysis.

Conflicts of interest

None.