Is the Spinal Instability Neoplastic Score Accurate and Reliable in Predicting Vertebral Compression Fractures for Spinal Metastasis? A Systematic Review and Qualitative Analysis.

Spinal metastases can present with varying degrees of mechanical instability. The Spinal Instability Neoplastic Score (SINS) was developed as a tool to assess spinal neoplastic-related instability while helping to guide referrals among oncology specialists. Some previous papers suggested that the SINS was accurate and reliable, while others disagreed with this opinion. We performed a systematic review regarding the SINS to evaluate its accuracy and precision in predicting vertebral compression fractures (VCFs). The 21 included studies investigated a total of 2118 patients. Thirteen studies dealt with the accuracy of SINS to predict postradiotherapy VCFs, and eight dealt with the precision. Among 13 studies, 11 agreed that the SINS categories showed statistically significant accuracy in predicting VCF. Among eight studies, body collapse was effective for predicting VCFs in six studies, and alignment and bone lesion in two studies. Location has no statistical significance in predicting VCFs in any of the eight studies. The precision of SINS categories was substantial to excellent in six of eight studies. Among the six components of the SINS, the majority of the included studies reported that location showed near perfect agreement; body collapse, alignment, and posterolateral involvement showed moderate agreement; and bone lesion showed fair agreement. Bone lesion showed significant accuracy in predicting VCFs in half of eight studies, but displayed fair reliability in five of seven studies. Although location was indicated as having near perfect reliability, the component showed no accuracy for predicting VCFs in any of the studies and deleting or modifying the item needs to be considered. The SINS system may be accurate and reliable in predicting the occurrence of postradiotherapy VCFs for spinal metastasis. Some components seem to be substantially weak and need to be revised.

INTRODUCTION

Cancer incidences are expected to increase, and overall survival of cancer is expected to decrease due to improved systemic therapy [36]. As a result, the incidence of patients with spinal metastases will increase as will the complexity of management [36]. Moreover, the median survival time of patients with bone metastases has substantially improved over the last decades, mostly because of advances in oncological treatment options [35]. Because the presence of spinal metastases represents advanced cancer, the goal of treatment shifts from long-term survival to preservation of quality of life for the patient’s remaining lifetime by retaining function and relieving symptoms [22,35].

In the past, external beam radiotherapy had been the cornerstone of treatment for painful spinal metastases [35]. Since the study of Patchell et al. [26] in 2005, direct surgical decompression and stabilization in patients with metastatic epidural spinal cord compression has been feasible and demonstrated better outcomes [14,18,25,38]. As surgical outcomes have been enhanced, consultations for surgery have increased to include spinal metastasis [25]. Skeletal-related events such as pathologic fracture after radiotherapy for spine metastases have a substantial negative impact on quality of life and daily functioning [35]. Although advancements in adjunct therapies have offered more treatment options, instability from spinal metastasis remains an important indication for surgical intervention [24,27].

The Tomita and Tokuhashi scores were well-known scoring systems to help determine surgical treatments for spinal metastasis. However, their accuracy was limited, and they were regarded to be difficult for non-surgeons to use [17,24]. For the non-surgeon, difficulty in diagnosing spinal instability can lead to inappropriate referrals of patients without instability or undertreatment of patients with instability [24]. Therefore, the Spinal Instability Neoplastic Score (SINS) was developed by the Spine Oncology Study Group (SOSG) to help physicians assess and categorize spinal instability; facilitate communication and appropriate referrals between oncologists, radiologists, and spine surgeons; and ensure that prompt and appropriate treatment plans can be developed as shown in Table 1 [10,24].

Table 1.

Spinal Instability Neoplastic Score (SINS)

SINS component	Score
Location
Junctional (occiput-C2, C7–T2, T11–L1, L5–S1)	3
Mobile segment (C3–C6, L2–L4)	2
Semirigid (T3–T10)	1
Rigid (S2–S5)	0
Pain
Yes	3
Occasional pain but not mechanical	1
Pain-free lesion	0
Bone lesion
Lytic	2
Mixed (lytic/blastic)	1
Blastic	0
Spinal alignment
Subluxation/translation	4
De novo deformity (kyphosis/scoliosis)	2
Normal alignment	0
Body collapse
>50% collapse	3
<50% collapse	2
No collapse with >50% body involved	1
None of above	0
Posterolateral involvement of spinal elements
Bilateral	3
Unilateral	1
None	0

Three categories (0 to 6, stable; 7 to 12, potentially unstable; and 13 to 18, unstable)

Previous studies have shown that the SINS has a substantial to excellent interobserver and intraobserver reliability [3,4,8,31]. However, the SINS has been evaluated by its authors and still requires independent validation before its wide clinical use [4]. Moreover, some studies reported that the total SINS score was not predictive for new or progressive vertebral compression fractures (VCFs) in patients receiving high-dose stereotactic radiotherapy [6,28,30]. Therefore, the role of SINS has been debatable.

The purpose of this study is to delineate the accuracy and precision of the total and individual components of the SINS system for spinal instability to predict VCF.

MATERIALS AND METHODS

Search strategy and study selection criteria

In accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, we conducted a systematic review of clinical studies that evaluated the SINS for spinal metastasis. A systematic search of PubMed, Embase, Web of Science, and the Cochrane Database was performed on the 30th of October 2019. The search term used was “Spinal Instability Neoplastic Score”. There were no language restrictions on study eligibility, and only the largest study was included for overlapping study populations. In addition, the reference lists of the reviews were screened for qualifying studies. Search results were screened by scanning abstracts for the following exclusion criteria : case reports, letters, comments, reviews, or technical notes, animal studies, duplicate studies, usage of SINS for other disease, and metastatic spinal cord compression. After removing excluded abstracts, full articles were obtained, and studies were thoroughly screened again using the same exclusion criteria. We excluded the following articles : no data of accuracy or precision or no prediction of VCFs. If data were missed in the included studies, we sent an e-mail to the corresponding authors of the papers to request data.

The goals of the search were to find articles that met the following inclusion criteria : comprised a group of spine metastases, included patients evaluated by the SINS after treatments such as radiation therapy or radiosurgery, and included data with accuracy or interobserver reliability of the SINS. Quality assessment was conducted independently in pairs and its consensus was reached by discussion. Study quality was determined for controlled observational cohort studies with the risk of bias assessment tool for nonrandomized studies (RoBANS).

Data were extracted from included studies according to two topics. One is accuracy of SINS. Evaluation tools for this were regression, uni-/multivariate analysis, or sensitivity/specificity. We analyzed accuracy of SINS category divided into two groups (SINS, 0–6 vs. 7–18) or three groups (SINS, 0–6 vs. 7–12 vs. 13–18). We also evaluated accuracy of each component of the SINS (location, pain, bone lesion, radiologic alignment, body collapse, and posterolateral involvement) if data were available. The other topic was precision among evaluators. Interobserver reliability was commonly evaluated using Cohen’s kappa (κ) value. We evaluated the reliability for the total score of the SINS and each component of the SINS. The κ coefficients were interpreted according to the widely accepted Landis and Koch [16] grading system in Table 2.

Table 2.

Levels of agreement for κ statistic levels quoted by Landis and Koch [16]

κ value	Level of agreement
0.00–0.20	Slight
0.21–0.40	Fair
0.41–0.60	Moderate
0.61–0.80	Substantial
>0.80	Excellent

Three categories (0 to 6, stable; 7 to 12, potentially unstable; and 13 to 18, unstable)

RESULTS

Search results for relevant studies

An initial literature search using the subject headings identified 85 studies in PubMed, 86 in Embase, 65 in Web of Science, and four in the Cochrane Central Register of Controlled Trials. Among these 240 studies, 141 were excluded as duplicates. Twenty-three of the 99 remaining papers were case reports, review articles, letters, technical notes, or patents and were excluded from our analysis. After screening the titles and abstracts, 15 studies that did not include metastatic spinal disease such as myeloma and granuloma and 11 that dealt with the metastatic epidural spinal cord compression. The 50 remaining studies were subjected to a full-text review, resulting in the exclusion of another 29 papers. The reasons for the exclusion of these articles consisted of the absence of data regarding accuracy or precision of SINS (n=21) and no prediction of VCF (n=8). Finally, a total of 21 studies were included in the meta-analysis. Detailed results of the selection process are shown in Fig. 1.

Fig. 1.

Flow diagram outlining the process for selecting relevant studies.

Participant characteristics

The characteristics of the included studies and participants are described in Table 3. The 21 included studies investigated a total of 2118 patients and the countries in which the studies were conducted included worldwide (n=3), United States (n=7), Canada (n=4), the Republic of Korea (n=3), Brazil (n=1), Japan (n=1), Netherlands (n=1), and Spain (n=1). Among 13 studies that evaluated accuracy of the SINS, 11 studies analyzed the accuracy with which the SINS predicted VCF development, one study [1] evaluated skeletal-related events such as pathologic fracture, the need for surgery, bone radiation, spinal compression, and hypercalcemia; the other study [5] assessed spinal adverse event (SAE) such as VCF, spinal cord compression, and a reduced quality of life. Ten studies estimated the incidence of VCFs after radiation therapy, two studies did it after stereotactic radiosurgery, and one studies did it after surgery. The aim and design of two studies by Thibault et al. [32,33] were overlap precisely, and differ only as the subjects have been expanded into multicenter. Therefore, we used data from the larger one.

Table 3.

Baseline characteristics of enrolled studies

Study	Study region	Primary cancer (major organs)	Mean age (years)	Study period	Mean follow-up (months)	Number of patients	Incidence of event (%)
Cunha et al. [6] (2012)	Canada	Multiple (kidney, breast, lung)	57.0±18.4	2007 to 2011	7.4±9.4	90 (167)	11.4
Sahgal et al. [28] (2013)	USA	Multiple (kidnry, breast, lung)	57.6±18.4	N/D	11.5±28.8	252 (410)	13.9
Sung and Chang [30] (2014)	Korea	Multiple (breast, lung, sarcoma)	51.0±15.3	N/D	11.0±5.4	72 (72)	36.1
Thibault et al. [32] (2014)	Canada	Renal cell cancer	63.0±12.5	October 2007 to August 2012	12.3±13.8	37 (61)	16.4
Thibault et al. [33] (2015)	USA, Canada	Renal cell cancer	60.2±13.9	N/D	8.0±19.4	116 (187)	18.2
Lam et al. [15] (2015)	USA	Multiple (breast, prostate, lung)	62	January 2008 to Decmeber 2013	N/D	299	17.1
Aiba et al. [1] (2016)	Japan	NSCLC	67	2009 to 2013	10.2±13.7	47	31.9
Lee et al. [20] (2016)	USA	Multiple (kidney, breast, thyroid)	57.2±14.5	March 2004 to March 2011	29.2	79	40.5
Germano et al. [12] (2016)	USA	Multiple (liver, breast, lung)	62.0±11.0	November 2007 to January 2014	16.0±18.0	79 (143)	21.0
Virk et al. [37] (2017)	USA	Multiple (renal, prostate, sarcoma)	60.7±4.6	2005 to 2013	12.6±7.2	323	8.0
Bollen et al. [3] (2017)	Netherland	Multiple (lung, breast, kidney)	60.4±13.8	January 2000 to December 2010	N/D	110	14.5
Shi et al. [29] (2018)	USA	Multiple (breast, lung)	60.0±18.9	2006 to 2013	5.9	203 (250)	20.4
Lee et al. [19] (2018)	Korea	Colorectal cancer	61.0±10.7	January 2007 to December 2014	10.0±24.5	53 (147)	15.0
Chang et al. [5] (2018)	Korea	Multiple (breast, lung, liver)	55.0±13.0	January 2008 to December 2009	24	78	26.9
Fourney et al. [10] (2011)	World	N/D	N/D	N/D	N/D	30	N/D
Teixeira et al. [31] (2013)	Brazil	N/D	N/D	N/D	N/D	40	N/D
Campos et al. [4] (2014)	N/D	Multiple (kidney, breast, lung)	65.0±14.3	January 2004 to March 2011	N/D	30	N/D
Fisher et al. [9] (2014)	World	Multiple (lung, prostate, breast)	N/D	N/D	N/D	30	N/D
Fisher et al. [8] (2014)	World	Multiple (lung, prostate, breast)	N/D	N/D	N/D	30	N/D
Arana et al. [2] (2016)	Spain	Multiple (breast, prostate, lung)	N/D	N/D	N/D	90	N/D
Fox et al. [11] (2017)	Canada	N/D	N/D	N/D	N/D	30	N/D

Values are presented as mean±standard deviation or number (segments). N/D : not described, NSCLC : non-small cell lung cancer

Accuracy of SINS to predict VCF

Qualitative analysis of the accuracy of the SINS for estimating VCFs is demonstrated in Table 4. Among 13 studies, 11 studies agreed that the categories of the SINS described statistical significance in predicting VCFs. Eight studies reported accuracy of each item of the SINS [3,5,6,12,19,28,29,33]. Six studies agreed that body collapse was effective in predicting VCFs. Four studies agreed that alignment and bone lesion were effective, and two supported that pain and posterolateral involve were effective. All eight studies showed that location has no statistical significance in predicting VCFs. A study [3] reported that location was significantly associated with the cumulative incidence of an adverse event in the multivariate analysis. However, location was not significant in the univariate analysis, and the results indicated that a high-risk score indicates less risk of VCFs. Hence, the authors addressed that location might not provide any clinically relevant information when assessing spinal instability [3].

Table 4.

Accuracy of SINS for estimating VCF

Study	Predicting target	Cat. of SINS (number of Cat.)	Location	Pain	Bone lesion (lytic/blastic)	Alignment	Body collapse	Posterolateral involve
Sung and Chang [30] (2014)	Post-SRS VCF	◯ (3, U)
Lam et al. [15] (2015)	Post-RT SAE	◯ (2, U/M)
Aiba et al. [1] (2016)	(Post-RT) SRE	◯[*]
Lee et al. [20] (2016)	Post-RT VCF	◯ (2, U/M)
Virk et al. [37] (2017)	Post-SRS VCF	◯[*]
Cunha et al. [6] (2012)	Post-RT VCF	◯ (2)	X	X	◯ (U/M)	◯ (U/M)	◯ (U)	X
Sahgal et al. [28] (2013)	Post-RT VCF	◯ (2)	X	X	◯ (U/M)	◯ (U/M)	◯ (U/M)	◯ (U)
Shi et al. [29] (2018)	Post-RT VCF	◯ (3, U/M)	X	◯ (U/M)	◯ (U/M)	◯ (U)	◯ (U/M)	◯ (U/M)
Chang et al. [5] (2018)	Post-Op VCF	◯ (3)	X	◯ (U/M)	X	◯ (U/M)	X	X
Lee et al. [19] (2018)	Post-RT VCF	◯ (3)	X	X	◯	X	◯	X
Thibault et al. [33] (2015)	Post-RT VCF	◯ (3, U)	X	X	X	X	◯ (U/M)	X
Germano et al. [12] (2016)	Post-RT VCF	X	X	X	X	X	◯ (U)	X
Bollen et al. [3] (2017)	Post-RT VCF	X	X	X	X	X	X	X

The probability of target events was correlated to total score of SINS.

SINS : spinal instability neoplastic score, VCF : vertebral compression fracture, Cat. : category, SRS : stereotactic radiosurgery, U : results of univariate analysis (not multivariate analysis), RT : radiation therapy, SAE : spinal adverse event such as vertebral compression fracture, spinal cord compression, and reduced the quality of life, U/M : results of both univariate and multivariate analysis, SRE : skeletal-related events such as pathologic fracture, the need for surgery, bone radiation, spinal compression, and hypercalcemia, OP : operation

Precision of SINS to predict VCF

Inter-observer reliability illustrated by the level of agreement and κ-values is demonstrated in Table 5. Eight studies evaluated the precision of the categories of the SINS; four studies reported excellent agreement; one, substantial; two, moderate; one, fair. Although there was a difference in the studies, the overall reliability of the SINS seems to be high. Among the six components of the SINS, the majority of the included studies reported that location showed near perfect agreement; body collapse, alignment, and posterolateral involvement showed moderate agreement; and bone lesion showed fair agreement. Regarding pain, the agreement of studies varied from moderate to excellent.

Table 5.

Precision of SINS for inter-rater reliability (κ-value)

Study	Category of SINS	Location	Pain	Bone lesion	Alignment	Body collapse	Posterolateral involve
Arana et al. [2] (2016)	Moderate (0.546)
Fisher et al. [9] (2014)	Excellent (0.83)	Excellent (0.94)	Substantial (0.73)	Substantial (0.65)	Moderate (0.49)	Substantial (0.61)	Moderate (0.55)
Fisher et al. [8] (2014)	Excellent (0.80-0.85)[*]	Excellent (0.94)	Excellent (0.88)	Moderate (0.55)	Moderate (0.42)	Moderate (0.57)	Moderate (0.43)
Campos et al. [4] (2014)	Substantial (0.790)	Excellent (0.811)	Moderate (0.587)	Fair (0.210)	Moderate (0.453)	Moderate (0.421)	Fair (0.295)
Fox et al. [11] (2017)	Excellent (0.990)	Excellent (0.948)	Substantial (0.739)	Fair (0.382)	Moderate (0.427)	Moderate (0.550)	Moderate (0.435)
Fourney et al. [10] (2011)	Excellent (0.846)	Substantial (0.790)	Excellent (0.841)	Fair (0.244)	Moderate (0.456)	Moderate (0.462)	Moderate (0.492)
Teixeira et al. [31] (2013)	Fair (0.375)	Substantial (0.719)	Moderate (0.419)	Fair (0.220)	Moderate (0.553)	Moderate (0.428)	Moderate (0.424)
Bollen et al. [3] (2017)	Moderate (0.536)	-	-	Fair (0.299)	Fair (0.358)	Moderate (0.453)	Moderate (0.436)

The value was rate 0.85 by spine surgeons, and 0.80 by oncologists.

SINS : spinal instability neoplastic score

DISCUSSION

The SINS was known to show high accuracy and reliability and to help determine surgery to prevent post-radiotherapy VCFs for spinal metastasis. A qualitative assessment of the studies demonstrated certain consistencies regarding the SINS. The categories of the SINS demonstrated substantial accuracy and precision for predicting VCFs. Although a previous study insisted that the total score or each component of SINS had a prognostic value for the occurrence of VCFs [36], our study concluded a different result. In terms of the accuracy of the six components of SINS, more than half of the studies reported that body collapse, bone lesion, and alignment were statistically significant as a predictor of VCF. All studies addressed that location has no statistical significance to predict VCF. In terms of precision, location demonstrated excellent reliability with high consistency of the included studies and bone lesion showed fair reliability. Some showed high accuracy and low precision and others showed low accuracy and high precision.

The SINS score is the sum of individual components of which some quantify the risk of spinal instability (e.g., location, bone lesion, posterolateral involvement) while other factors express the current degree of spinal instability (e.g., body collapse, alignment, pain) [36]. The latter usually showed high accuracy to predict VCF and the former showed a low accuracy. In our review, body collapse was the most important component. Whereas, pain has little meaning. The pain of SINS described aggravated by movement, upright posture, or loading of the spine but relieved by recumbence (mechanical pain). However, this may be difficult to distinguish from non-mechanical back pain. Precision of components that express the current degree of spinal instability were usually relatively low. Body collapse and alignment demonstrated moderate interobserver reliability in six of seven studies. These components seem to be accurate and reliable. The precision of pain was quite heterogenous, and two studies showed moderate reliability among six studies; two showed substantial and two, excellent. The reason of the low accuracy of the pain may be the subjectivity in the assessment. The pain component needs to be defined more clearly. Sahgal et al. [28] reported that baseline body collapse, lytic tumor, and misalignment were predictive, whereas presence of pain, location, and posterolateral involvement of spinal elements were not.

Three components of the possible risk of spinal instability (e.g., location, bone lesion, posterolateral involvement) usually presented low accuracy and high reliability. Bone lesion showed significant accuracy in predicting VCFs in half of the studies, but displayed fair reliability in five of seven studies. Removing this component of the SINS might improve reliability, but would be detrimental to content validity. The SOSG addressed that the decision to include this category was made based on strong biomechanical literature suggesting that the cross-sectional area of a defect combined with bone mineral density are excellent predictors of vertebral body failure and pathologic fracture risk [10]. To improve reliability, SOSG may consider using data with multi-slice computed tomography [10]. As another possible risk, the location of a spine metastasis seemed to affect compression fractures that occur in mobile segments because of mechanical stress. However, location showed no correlation with VCFs in any of the studies. Even if the component displayed near perfect reliability, deleting or modifying the item needs to be considered.

The increasing incidence of spinal metastases has led to an increased need for surgical and radiotherapy interventions to maintain or improve patient-related quality of life. The SOSG recommends consultation with a spine surgeon for all patients with a SINS of 7 or greater [7]. After the application of the SINS in clinical practice, a change in the referral pattern might be expected with a decrease in inappropriate and/or late referrals [35]. This fact is important if one considers that one of the purposes of the SINS is to improve communication and coordination among physician from diverse disciplines dedicated to oncologic patient care, for instance, suggesting the need for surgical consultation [4]. The future role of the SINS has yet to be determined. Ongoing research is required to continually update and validate each component and score to remain contemporary to modern treatments and practices.

The SINS is developed to determine whether to perform surgery or radiation therapy depending on the risk of VCFs. To develop evidence-based treatment strategies for spinal neoplastic-related instability, it is important to use uniform definitions and outcome parameters [36]. However, many papers used SINS to predict various target event for various tumors, which may make confusion of utility of SINS [13,21,23,34]. We excluded papers regarding metastatic epidural compression, giant cell tumor, sarcoma, or myeloma, and papers estimating survival or re-treatment. VCF was known to be a fairly low-risk adverse event after conventional radiotherapy, whereas a wide risk range estimates for VCF after spinal SSRS [20]. Because the incidence of VCF affected the accuracy and precision, a clear definition of a target will be needed. Consistent use of the SINS and/or the components of the SINS to study spinal instability will facilitate uniform reporting of results and could ultimately enhance the quality of the research and patient outcomes.

CONCLUSION

The SINS system may be accurate and reliable in predicting the occurrence of post-radiotherapy or radiosurgery VCFs after spinal RT/SRS for spinal metastases. In terms of each component of the SINS, bony collapse, alignment, and bony lesion were accurate, but their precisions were fair to moderate. Location was high in reliability but low in accuracy. Diagnostic ability of location and bony lesion can have substantial weak points and need to be revised.