Reliability of the Instability Severity Index Score as a Predictor of Recurrence after Arthroscopic Anterior Capsulolabral Reconstruction: A Multicenter Retrospective Study.

BACKGROUND: Although the instability severity index score (ISIS) is widely used to predict recurrence after arthroscopic anterior instability surgery, its reliability, especially on the weightings and cutoff values, is questionable. The goal of the current retrospective study was to investigate recurrence after arthroscopic capsulolabral reconstruction to evaluate whether each domain of the ISIS has the appropriate predictive power for recurrence by using logistic regression analyses with odds ratios (ORs).
METHODS: This study included 120 consecutive patients who underwent arthroscopic capsulolabral reconstruction between 2004 and 2016. We retrospectively reviewed patients' preoperative history and radiographs, postoperative recurrence or sensation of instability, and risk factors related to the ISIS. The mean postoperative follow-up was 27.6 months (range, 12 to 96 months; median, 21 months). Twenty-six patients with recurrence or positive apprehension were classified as the recurrence group; 94 patients without any symptoms were classified as the non-recurrence group. Logistic regression analyses with ORs were used to verify the utility of each domain of the ISIS for predicting recurrence.
RESULTS: The mean ISIS did not differ significantly between the recurrence and non-recurrence groups (4.3 ± 1.8 vs. 3.4 ± 2.1 points; p = 0.063). Among the domains of ISIS, factors related to bone defects, the presence of a Hill-Sachs lesion and glenoid bone loss had the lowest ORs (0.77 and 0.38, respectively).
CONCLUSIONS: Not all ISIS domains accurately predicted recurrence after arthroscopic capsulolabral reconstruction. The ISIS may not be a proper reference for determining Latarjet procedure in patients with anterior shoulder instability.

Anterior shoulder instability is a common problem, especially in young, active patients. Regardless of the type of surgery (open or arthroscopic repair), the most important and common complication after surgical treatment of anterior shoulder instability is the recurrence of dislocation or instability. There are several scoring systems that combine predictive risk factors to help the surgeon determine the appropriate surgical option and minimize the risk of recurrence.12) In 2007, Balg and Boileau1) introduced the instability severity index score (ISIS) designed to help determine the most suitable repair method by predicting the recurrence rate. They defined a further dislocation or any subjective complaint of subluxation as a recurrence. The ISIS has a maximum score of 10 points and assigns a weight of 1 or 2 to each of the six risk factors. In the original study, the estimated recurrence rate was higher than 70% for patients with over 6 points before surgery; thus, an open Latarjet surgery or open Bankart repair was recommended over an arthroscopic Bankart repair in such patients. Although the ISIS is widely used to predict recurrence after arthroscopic anterior instability surgery, some reports have suggested the need for revision of the risk factors or the cutoff point.34) In particular, the original study did not include the analysis of detailed computed tomography (CT) or magnetic resonance imaging (MRI) findings for percent bone loss that might contribute to recurrence and did not provide the rationale for the weight of each score. On the basis of our clinical experience, we also question whether the ISIS is a reliable scoring system with the appropriate weighting and cutoff values. In this retrospective study, we investigated recurrence after arthroscopic capsulolabral reconstruction to evaluate whether each ISIS domain has the appropriate predictive power for recurrence by using logistic regression analyses with odds ratios. Our hypothesis was that weightings of ISIS would be inappropriate to predict recurrence after arthroscopic anterior instability surgery.

METHODS

Patient Selection

A multicenter retrospective study was conducted at three different university hospitals after obtaining approval from the Institutional Review Board of Seoul National University Bundang Hospital (IRB No. B-1705-395-101). Informed consent was waived. A total of 120 patients were included in the study. All patients underwent arthroscopic capsulolabral reconstruction for recurrent anterior shoulder instability between March 2004 and December 2016. Open Latarjet surgery or open Bankart repair was not performed as a primary operation in any of the patients. The exclusion criteria were a follow-up period shorter than 12 months (n = 21), seizures (n = 2), multidirectional instability (n = 4), severe osteoarthritis (Samilson-Prieto grade II or higher; n = 2), a concomitant rotator cuff lesion (n = 25), revision surgery (n = 11), and an acute/first-time dislocation (n = 5).

Clinical and Radiological Assessment

The medical records of the 120 patients satisfying the inclusion criteria were reviewed immediately postoperatively, at 3, 6, and 12 months postoperatively, and then yearly thereafter until the last follow-up by two shoulder fellowship-trained orthopedic surgeons (HJS, JSP) who were not involved in the treatment. The preoperative medical history, postoperative recurrence or sense of instability, and degree of sports participation, type of sports, and shoulder hyperlaxity (three domains of ISIS) were also reviewed (Table 1). Medical records of hyperlaxity measured preoperatively at each hospital were reviewed by the two shoulder fellowship–trained orthopedic surgeons. As described in Balg and Boileau's study,1) anterior hyperlaxity was defined as an external rotation greater than 85° with the arm at the side, and inferior hyperlaxity was defined as a positive Gagey test. However, it was difficult to define competitive sports according to the degree of sports participation factor as there are many kinds of sports. Thus, we included elite athletes belonging to an athletic club and those who had participated in ball games (e.g., basketball, soccer, baseball, and badminton) or martial arts (e.g., judo, wrestling, and boxing) for more than 2 years (age, 20.5 ± 4.0). The preoperative radiographs (anteroposterior [AP] view with external rotation) were retrospectively reviewed to determine whether there was a glenoid loss of contour or a Hill-Sachs lesion. The presence/absence of a Hill-Sachs lesion was noted on the external rotation view, but no attempt was made to assess its size. Based on these data, the ISIS was calculated. Following the recommendations of Balg and Boileau,1) recurrence rates were compared by using a cutoff of 6 points on the ISIS. The preoperative and 12-month postoperative Rowe scores were also investigated because Rowe score was the only scoring system used commonly among the three hospitals.

Table 1

The Instability Severity Index Score* in the Study Population

Prognostic factor	Point
Age at surgery (yr)
≤ 20	2
> 20	0
Degree of sports participation (preoperative)
Competitive	2
Recreational or none	0
Type of sport (preoperative)
Contact or forced overhead	1
Other	0
Shoulder hyperlaxity
Shoulder hyperlaxity (anterior or inferior)	1
Normal laxity	0
Hill-Sachs on AP radiograph
Visible in external rotation	2
Not visible in external rotation	0
Glenoid loss of contour on AP radiograph
Loss of contour	2
No lesion	0
Total (point)	10

AP: anteroposterior

*As described by Balg and Boileau.1)

Surgical Procedure

All the operations were performed by three surgeons (JHO, SJS, CHC) at their respective institutions with the patient placed in a lateral decubitus position. The capsulolabral reconstruction was carried out in a standard fashion using suture anchors. For Bankart repair or anterior capsular shift, the abnormally attached labrum or anteroinferior glenohumeral ligament was mobilized from the glenoid neck, and then a 2- to 3-mm wide section of subchondral bone was exposed using a motorized burr for the recipient bed. The capsule together with the anterior labrum, if present, was sutured and tacked down by using three to five evenly spaced, single-loaded, knotless suture anchors between the 2- and 6-o'clock positions on the right shoulder. In all cases, arthroscopic remplissage was not performed. The number of suture anchors was evaluated.

Postoperative Rehabilitation

A standardized rehabilitation protocol was applied to all patients. Immobilization in neutral rotation with an abduction pillow was maintained for 6 weeks. At 6 weeks after surgery, range of motion exercises were initiated and use of the brace was discontinued. Muscle strengthening exercises were commenced 3 months after surgery and continued until at least 6 months postoperatively. Return to sports was permitted 6 months after surgery.

Statistical Analysis

Statistical analyses were performed using SPSS ver. 18.0 (SPSS Inc., Chicago, IL, USA). Sample size was calculated for a desired power of 80% and an α-value of 0.05 with estimation of a postoperative failure rate of 10%; a sample size of 100 patients was required. Frequency and descriptive statistics were used to compare the baseline characteristics between the recurrence group and non-recurrence group. Group comparisons were performed using t-tests, chi-square tests, and Mann-Whitney U-tests. Univariate logistic regression analysis was performed to confirm the appropriateness of the weights for each domain of the ISIS. Statistical significance was set at p < 0.05.

RESULTS

The mean age of the 120 patients (110 men and 10 women) was 23.4 ± 7.8 years. Preoperative demographic data and presence of ISIS risk factors are shown in Table 2. The mean ISIS was 3.6 ± 2.1 points, and the most common score was 3 points (27 patients). Ten patients had a score over 7 points and 110 patients had a score below 6 points (Fig. 1). Preoperatively, subluxation symptoms were noted in 18 patients, and dislocation symptoms were found in 102 patients. The median number of suture anchors was 4.

Table 2

Preoperative Demographics and Clinical Data

Population description	Value
Sex
Male	110 (91.7)
Female	10 (8.3)
Affected side
Right	77 (64.2)
Left	43 (35.8)
Affected side
Dominant	71 (59.2)
Nondominant	49 (40.8)
Type of instability
Dislocation	102 (85.0)
Subluxation	18 (15.0)
Preoperative Rowe score	63.0 ± 10.9
Numbers of suture anchors (median)	4
Age (yr)	23.4 ± 7.8
Age ≤ 20 years at surgery
Yes	52 (43.3)
No	68 (56.7)
Level of sport
Competitive	45 (37.5)
Recreational or none	75 (62.5)
Contact or forced overhead activity
Yes	73 (60.8)
No	47 (39.2)
Shoulder hyperlaxity
Yes	42 (35.0)
No	78 (65.0)
Hill-Sachs lesion
Yes	52 (43.3)
No	68 (56.7)
Glenoid bone loss
Yes	10 (8.3)
No	110 (91.7)
Instability severity index score	3.6 ± 2.1

Values are presented as number (%) or mean ± standard deviation unless otherwise indicated.

Fig. 1

Distribution of instability severity index score (ISIS) among the study population.

Twelve of the 120 patients (10%) experienced redislocation at a mean follow-up of 33 months (range, 3 to 68 months), and 14 patients (12%) had instability symptoms (apprehension positive) after surgery at a mean follow-up of 24 months (range, 3 to 69 months); these 26 patients were classified as the recurrence group. The remaining 94 patients had no dislocation or instability symptoms at the last follow-up (mean, 27.6 months; range, 12 to 96 months), and they were classified as the non-recurrence group. The mean preoperative Rowe scores in the recurrence group (64.2 ± 13.2 points) and non-recurrence group (62.7 ± 10.2 points) were not significantly different (p = 0.517). However, the mean postoperative Rowe score in the recurrence group (82.9 ± 15.4 points) was significantly lower than that in the non-recurrence group (94.4 ± 7.4 points, p = 0.001). The mean number of suture anchors and the mean ISIS score were not significantly higher in the recurrence group than that in the non-recurrence group (p = 0.06 and p = 0.063, respectively) (Table 3). Table 4 compares the frequency of risk factors associated with the ISIS between the two groups. Although the recurrence rate was higher (30.0%, 3 / 10) in patients with a score above 6 points than that in patients with a score of 6 points or less (20.9%, 23 / 110), the difference was not statistically significant (p = 0.504).

Table 3

Group Comparisons for ISIS, Rowe Score, and the Number of Suture Anchors

Variable	Recurrence group (n = 26)	Non-recurrence group (n = 94)	p-value
ISIS	4.3 ± 1.8	3.4 ± 2.1	0.063
Preoperative Rowe score	64.2 ± 13.2	62.7 ± 10.2	0.517
Postoperative Rowe score	82.9 ± 15.4	94.4 ± 7.4	0.001*
No. of suture anchors	4.6 ± 1.6	4.00 ± 1.1	0.056

Values are presented as mean ± standard deviation.

ISIS: instability severity index score.

*Statistically significant.

Table 4

Frequency for Each Domain of the ISIS in Recurrence Group and Non-recurrence Group

Variable	Recurrence group (n = 26)	Non-recurrence group (n = 94)	p-value
Patient ≤ 20 years	16	36	0.035
Involved in competitive sports	12	33	0.305
Contact or forced overhead activities	21	52	0.019
Anterior or inferior hyperlaxity	8	34	0.611
Hill-Sachs lesion	10	42	0.573
Loss of the normal inferior glenoid contour	1	9	0.352

ISIS: instability severity index score.

On the logistic regression analyses, the domain with the highest odds ratio (OR) with a meaningful p value was contact or forced overhead activities (OR, 3.39; p = 0.023), followed by patient younger than 20 years (OR, 2.58, p = 0.038) (Table 5). The domain with the lowest OR was the loss of the normal inferior glenoid contour (OR, 0.38), and the second lowest was Hill-Sachs lesion (OR, 0.77); these two domains correspond to the radiographic factors.

Table 5

Weight and Odds Ratio for Each Domain of the ISIS

Variable	ISIS weight (point)	Odds ratio	p-value
Patient ≤ 20 years	2	2.58	0.038
Involved in competitive sports	2	1.58	0.305
Contact or forced overhead activities	1	3.39	0.023
Anterior or inferior hyperlaxity	1	0.78	0.610
Hill-Sachs lesion	2	0.77	0.572
Loss of the normal inferior glenoid contour	2	0.38	0.367
Total	10

ISIS: instability severity index score.

DISCUSSION

predictive power for recurrence after arthroscopic capsulolabral reconstruction; in particular, the two radiology-related domains demonstrated the poorest ORs. Quantitative assessment of bony loss could be revisited by the original ISIS system. Recently, arthroscopic procedures have become widely used in the treatment of anterior shoulder instability with increasingly satisfactory results.56789101112) However, open surgery is traditionally thought to be superior to arthroscopic surgery in terms of recurrence rates,1314) and decisions regarding surgical procedures in patients with a high risk of recurrence are strongly dependent on the surgeon's experience and preference. Balg and Boileau1) used six preoperative risk factors to create a decisive preoperative criterion for arthroscopic or open surgery; based on their results, the cutoff was set at 7 points. However, no clear explanation was provided regarding the weights used for each ISIS domain (1 or 2 points). Rouleau et al.4) found that the interobserver reliability of the ISIS was excellent, with an intraclass correlation coefficient of 0.933. Furthermore, the mean number of dislocations before surgery was significantly higher in those with an ISIS ≥ 6 points, and the mean ISIS was higher in patients undergoing a complex surgery (e.g., Hill-Sachs remplissage or open Latarjet procedure). Thus, the authors concluded that the ISIS is highly reliable and that any modification of the six domains would require a new validation in practice. This study was a multicenter study like our study; however, the results were different from ours. The authors of the study analyzed only the overall interobserver reliability of the ISIS and did not analyze each domain of the ISIS unlike our study. The radiographic domains (loss of the normal inferior glenoid contour and Hill-Sachs lesion) are not usually evaluated on plain radiographs, and quantitative measurements using CT or MRI are important for determining the surgical option based on the severity of the lesion. Charousset et al.15) reported that the Hill-Sachs lesion criterion of ISIS was not a reliable risk factor. The authors also reported that it could be improved by using the P (maximum depth of the notch defect in the internal rotation)/R (the radius of the humeral head in the internal rotation) index, as determined on CT imaging studies. In addition, Bouliane et al.16) conducted a study to evaluate the ISIS as a predictor of failure after arthroscopic Bankart repair and mentioned that while three factors were easy to determine (patient age, level and type of sport, and joint laxity), two components (glenoid and humeral bone loss) were dependent on individual radiological assessments. Determining whether glenohumeral bone loss is present on a single AP radiograph may be difficult because of intra- and inter-rater reliability. The authors also mentioned that using an alternative imaging modality, such as three-dimensional (3D) imaging, may allow a more accurate identification of bone loss. In a study of 83 patients with anterior shoulder instability who underwent arthroscopic Bankart repair using suture anchors, Voos et al.17) reported that the presence of a Hill-Sachs lesion was not a risk factor for recurrence; however, the size of the lesion was a risk factor. Therefore, assigning more points (2 points) to the radiographic domains of ISIS based solely on the presence of bony lesions in plain radiographic examination, rather than on the severity of the bony lesion in CT, appears to undermine the reliability of the ISIS system. Consistent with this, the two radiographic domains of ISIS had the lowest odds ratios in the present study. These results may not indicate that these factors are of low importance; they rather suggest that the presence/absence of the bony lesion determined by simple radiography alone may not reflect the severity of the bony defect in patients with instability, as suggested in previous studies.151617) These radiographic factors should be included as one of the risk factors for recurrence after surgery; however, quantitative measurements using an advanced imaging tool (3D-CT) should be considered.

Several studies have emphasized the influence of hyperlaxity on recurrence,21718) and the ISIS system also assigns 2 points to this. Although Phadnis et al.3) concluded that hyperlaxity was not an independent risk factor for recurrence, the number of patients showing hyperlaxity was low in their study. In our study, we had a sufficient number of patients (42 patients) to evaluate the association of hyperlaxity with recurrence. The OR for this risk factor was 0.78, which was the third lowest in the logistic regression analysis showing no statistical significance. Therefore, we suggest that hyperlaxity should be considered as one of the risk factors, not as an independent risk factor, for recurrence after arthroscopic capsulolabral reconstruction.

On the basis of original data, Balg and Boileau1) suggested 7 points as a cutoff value for determining whether open or arthroscopic surgery should be performed for an anterior shoulder dislocation. Later, Rouleau et al.4) reported that the mean ISIS for simple (arthroscopic Bankart or open Bankart) and complex (Latarjet or associated remplissage) surgery was 3.4 and 4.8, respectively. Thomazeau et al.19) reported the results of arthroscopic Bankart repair for 125 patients with anterior shoulder dislocation with an ISIS of 4 or less and reported that it is appropriate to use ISIS as a consultation tool. The authors also suggested that the cutoff value should be lowered to 4 points. Similarly, Phadnis et al.3) concluded that a score of 4 points was the best threshold value when considering sensitivity and specificity in ROC curve analysis. However, lower cutoff values of the ISIS system will lead to a higher prevalence of complex surgery, such as the Latarjet procedure, as a primary surgical option for patients with “simple” anterior shoulder instability. Given the complications and learning curve of the Latarjet procedure, we may need to guard against overtreatment. Above all, if the ISIS system is to become a proper consultation tool for this complex surgery, each domain will need to be verified properly and revised further, especially the bone defect domains. Therefore, according to the results of the current study, it would be necessary to assign a new weight to each domain and to set a new cutoff value or to modify the radiological assessment domain by using CT scans for an appropriate decision-making process.

The present study has several limitations. Like the original ISIS study, the current study included apprehension-positive patients in the recurrent group. This was because few patients had pure redislocation; however, we believe that postoperative apprehension is also part of recurrence. Another limitation concerns the retrospective and multicenter nature of the present study. Depending on the surgeon's procedure, the outcome of the surgery would be different. However, there were no definite differences in the surgical indications, surgical techniques, or patient position between the centers, and variability across centers was minimized. Moreover, two shoulder fellowship–trained orthopedic surgeons visited each center in person and reviewed the medical records and preoperative radiographs, so the differences in the cohort between hospitals would not have been significant. Nevertheless, a prospective study would offer a better insight into the reliability of the ISIS. It should be taken into consideration that the medical records were not reviewed by the same surgeons who initially measured the instability and hyperlaxity. There might have been some differences in radiographers or radiographic positioning among centers. However, even at the same center, radiographers would vary depending on the working time and the location of radiography room. Therefore, we think there would not be much difference in the imaging of the AP radiographs in internal, neutral, and external rotation views. In addition, the current study only included the Rowe score as a functional outcome; there were not many functional scores used in common across the participating centers. The evolution of arthroscopic surgical techniques over the long period of the cohort collection would have potentially generated omission variable biases for regression analysis. And the two risk factors with the lowest OR were not statistically significant. Lastly, the current study did not suggest a new scoring system for quantitatively measuring bone defects or modify the weighting of risk factors. We believe that a new scoring system using a CT scan is needed to best determine the surgical procedure in a further study.

Not all the ISIS domains showed completely satisfactory predictive power for recurrence after arthroscopic capsulolabral reconstruction. In addition, the ISIS may not be an appropriate reference for determining Latarjet procedure in patients with anterior shoulder instability.