Validation of the Shin Pain Scoring System: A Novel Approach for Determining Tibial Bone Stress Injuries.

BACKGROUND: The incidence of adolescent overuse injuries, including bone stress injuries (BSIs), is on the rise. The identification of a BSI in the early stages is key to successful treatment. The Shin Pain Scoring System (SPSS) was developed to aid clinicians in identifying patients with a BSI. HYPOTHESIS: The SPSS will correlate with magnetic resonance imaging (MRI) grading of a BSI in an adolescent population. STUDY
DESIGN: Cohort study (diagnosis); Level of evidence, 2.
METHODS: Enrolled in this study were 80 adolescent high school athletes between the ages of 13 and 18 years participating in a variety of sports with more than 1 week of atraumatic shin pain. The SPSS questionnaire was completed for each participant, and physical examination findings were recorded. Each question and physical examination item was allotted a point value, which totaled 29 points. Radiographs and MRI scans of both lower legs were obtained for each participant. The SPSS score was statistically analyzed using logistic regression, a classification matrix, and a 2 × 2 contingency table to evaluate validity and predictability.
RESULTS: Logistic regression analysis of our data determined that 3 categories of SPSS scores provided the highest diagnostic value when compared with MRI grading based on the Fredericson classification (0-4). The SPSS correctly identified 43.5% of injuries for category 1 (MRI grades 0-1), 62.5% for category 2 (MRI grade 2), and 50.0% for category 3 (MRI grades 3-4). Overall, the SPSS correctly identified the degree of BSI in 54.4% of all tibias studied. Binary analysis for validity demonstrated a sensitivity of 96%, specificity of 26%, positive predictive value of 76%, and negative predictive value of 71% for the SPSS relative to the "gold standard" MRI results.
CONCLUSION: The SPSS is a potentially valid method to identify tibial BSIs, given the sensitivity and negative and positive predictive values. It also provides helpful categorization to alert clinicians to the presence of a BSI and direct further diagnostics and/or interventions. The SPSS should be considered as an additional tool to use when evaluating adolescents with atraumatic tibial BSIs.

According to the National Federation of State High School Associations,[27] participation in high school sporting activity continues to grow, as does the incidence of adolescent overuse injuries, including bone stress injuries (BSIs).[5] Young athletes appear to be more susceptible than adults to BSIs.[7,15] Studies have found that 40% to 50% of all reported BSIs occur in those younger than 20 years.[1,13,18,21]

The literature notes a positive impact of early recognition on return to sporting activity and reduction of longer term disability.[10,32,34] The key is to identify BSIs in the early stages before they can progress to stress fractures.[14] Ohta-Fukushima et al[34] found that those adolescents who reported their injury >3 weeks after the initiation of symptoms showed a significant increase in return time. Niemeyer et al[32] noted that a BSI in athletes with an immature skeletal system must be taken seriously and does not always culminate in a good outcome from treatment.

Clinicians frequently find it difficult to make a formal clinical diagnosis for adolescents who present with shin pain without appropriate imaging. The complaint of shin pain may be caused by a wide range of potential differential diagnoses, including bony pain related to a BSI, compartment syndrome, a soft tissue injury, popliteal artery entrapment, an infection, medial tibial stress syndrome, periostitis, or tumor.[11,24]

It is difficult to correctly differentially diagnose shin pain utilizing clinical examinations alone, and the diagnosis of a BSI is contingent on a detailed examination[9] that incorporates patient history, contributing risk factors, and a thorough physical examination and radiographic examination.[37,39,40] Clinical tests have included palpation, a fulcrum test, single-leg hop, and use of a tuning fork. Individually, these methods lack both sensitivity and specificity.[10,38] Ultrasound also has been utilized as a screening tool, with varying sensitivity and specificity.[17,35,38]

Radiographs are frequently utilized as the initial screening tool; however, initial radiographs have limited usefulness because of their inability to detect bony changes during the early development of a BSI. Radiographs are only acutely positive in 3% to 23% of cases, while serial radiographs are only positive in 24% to 51% of cases.[4,8,11,12,19,36] Initial radiographs do have value to rule out other significant problems such as frank fractures, bony tumors, or infections.[20]

Advanced imaging for determining the existence and severity of a BSI includes magnetic resonance imaging (MRI), computed tomography (CT), and single-photon emission computed tomography (SPECT). Advanced imaging is expensive, is often not approved by insurance, and in the case of CT and SPECT results in significant radiation loads for the patient. MRI is highly sensitive and specific and offers no radiation load for the patient and therefore is considered the “gold standard” for identifying adolescent BSIs.[2,9,10,12,14,16]

With the use of MRI for adolescent tibial pain, this clinical picture has been described as a stress reaction in the early stages and as a stress fracture in later stages.[29] The recent literature suggests the value of early recognition on return to sporting activity and reduction of longer term disability.[31,32,34] However, the early use of MRI has been limited, as it is expensive and frequently not approved by insurance providers as an acute diagnostic modality for adolescent tibial pain.[20]

Without imaging and a definitive clinical examination, many clinicians diagnose the active adolescent as having “shin splints,” which is a common benign term and lacks formal distinction, as it does not define the location or source of pain.[20,39] Nor does it direct a specific intervention.

Several authors have recommended the use of a screening tool for identifying those who may be at risk for overuse BSIs.[25,26,28,30,33] The purpose of our study was to validate the Shin Pain Scoring System (SPSS), which we had developed for use as an evaluation tool in the diagnosis of adolescent BSIs. Our hypothesis was that the SPSS would correlate with the severity of BSI as diagnosed by MRI.

Methods

A total of 80 adolescent high school athletes (52 female, 28 male) aged 13 to 18 years (mean age, 15.4 years) who were involved in multiple sports, including track, cross-country, basketball, soccer, gymnastics, and lacrosse, participated in this institutional review board–approved study. The participants had all reported to a single institution with a >1-week history of atraumatic shin pain. Participants were enrolled on a consecutive basis, and assent and parental consent were provided to participate. Each participant completed the SPSS questionnaire (Appendix) and underwent a clinical examination by 1 of 4 sports medicine fellowship–trained orthopaedic surgeons.

The SPSS consisted of 8 questions related to medical history and health (8 points) as well as a clinical examination (21 points). The clinical examination included palpation to identify the location of bony tenderness, a tap test utilizing 2 fingers to “tap” the bone for tenderness, vibration testing utilizing a 128-Hz tuning fork, and fulcrum testing to assess the tibia for pain. An assessment of active range of ankle dorsiflexion motion via a weightbearing lunge test, as described by Chisholm et al,[6] was performed, and an assessment of function utilizing a single-leg hop test (10 repetitions), noting a decrease in hop height, increase in landing time, and increase in pain versus the opposite side and expected normal function, was also included in the clinical examination. We attempted to add an assessment of training history to the model, but it had no effect on improving the predictive accuracy of the tool and consequently was excluded. Each question and clinical finding was allotted a point value, for a total score of 29 points.

Bilateral digital radiographs including full-length anteroposterior and lateral views were obtained for each participant. MRI scans of the bilateral tibia were acquired without the administration of a gadolinium-based contrast agent utilizing a standard protocol of bilateral coronal short tau inversion recovery (STIR), bilateral coronal, bilateral axial STIR, bilateral axial T1, and unilateral fat-suppressed FSE T2 for each participant on the same day of study enrollment. An independent musculoskeletal radiologist (R.E.) blinded to results of the SPSS reviewed both the radiographs and MRI scans and graded the MRIs utilizing the Fredericson grading scale (grade 0 = normal; grade 1 = only evidence of periosteal edema; grade 2 = marrow edema visualized on T2 weighted images; grade 3 = more significant marrow edema, visible on T1 & T2 weighted images; grade 4 = intracortical signal changes, multiple focal areas, linear region of intracortical signal change).[10]

Statistical Analysis

Logistic regression was used to test whether the SPSS could predict MRI grading using SPSS scores and sex as prediction variables. SPSS scores were calculated as a continuous variable and considered normally distributed upon analysis before logistic regression. MRI grade and sex remained categorical variables. A classification matrix was also developed for the categorization of MRI grades based on the SPSS score and sex. MRI grades 0 and 1 were combined, and MRI grades 3 and 4 were combined, given the low frequency of occurrence. MRI grade 2 exhibited a high frequency of occurrence so remained alone. Also, given the mild nature of grade 1 findings relative to grades 2 to 4, combining it with grade 0 given statistical findings was deemed acceptable. This resulted in 3 categories of MRI grades. See Table 1 for the model using MRI grade and sex.

Table 1

Logistic Regression Results

Coefficients	Slope	SE	t Value	P Value
SPSS score	0.29	0.046	6.33	<.00001
Sex	1.06	0.325	3.27	.00108
Interceptsb	Value	SE
0|1	1.4104	0.4717
1|2	2.2982	0.4888
2|3	4.9844	0.6170
3|4	8.1045	0.8819

For a female participant with a Shin Pain Scoring System (SPSS) score x, the estimated logistical odds of having a magnetic resonance imaging (MRI) grade ≤3 is 4.9844 + 0.29x, whereas for a male participant with SPSS score x, the estimated logistical odds of having an MRI grade ≤3 is 4.9844 + 1.06 + 0.29x.

MRI grade according to Fredericson classification.[10]

Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) calculations were also performed using a 2 × 2 contingency table with SPSS scores of 0 to 5 for women and 0 to 1 for men versus >5 for women and >1 for men, relative to MRI grades of 0 and 1 versus MRI grades of 2 to 4. The SPSS score ranges used to develop the proportions for this analysis were developed from the probability table created from the final logistic regression model. The lower SPSS score categories of <5 for women and <1 for men indicate a low risk for BSIs. Counts were tallied for each cell of low- or high-risk BSIs based on the SPSS score and MRI grade for calculation.

Results

A total of 160 lower extremities in 80 participants (52 female, 28 male) were evaluated. There were 54 participants (67%) who reported pain in both tibias and 26 (33%) who reported pain in 1 tibia, resulting in 134 of 160 tibias with self-reported pain. Of the 134 tibias reported to have pain, 118 showed evidence of a BSI on MRI (grades 1-4), while 16 reported to have pain were grade 0 on MRI. Of the 26 tibias reported to have no pain, 15 showed evidence of an injury on both the clinical examination and MRI (grades 1-4), while only 11 reported as normal were actually classified as MRI grade 0 (Table 2).

Table 2

Pain and MRI Results

	No. of Participants Self-reporting Pain	No. of Tibias	TIbias With MRI Grades 1–4	Tibias With MRI Grade 0
Self-reported pain	80	134	118 (89)	16 (59)
Bilateral pain	54 (67)	108 (68)
Unilateral pain	26 (33)	26 (16)
No pain	N/A	26 (16)	15 (11)	11 (41)
Total	80 (100)	160 (100)	133 (100)	27 (100)

Data are shown as n (%). MRI, magnetic resonance imaging; N/A, not applicable.

The net result was that 133 of 160 tibias demonstrated positive MRI findings of a BSI, while 27 of 160 had no findings and were graded as 0. The majority of these 27 tibias were from female participants (n = 21; 78%). There were 19 tibias (12%) that were determined to be grade 1, with 17 being from female participants. The largest group (50%) of tibias were determined to be grade 2 (n = 80), occurring in 44 female and 36 male tibias. There were 29 tibias (18%) that were labeled as MRI grade 3 (female: n = 17; male: n = 12). Only 5 tibias (3%) were considered grade 4 (female: n = 2; male: n = 3).

There were no frank fractures seen on radiographs. One participant did not undergo a radiographic evaluation because of a potential pregnancy concern. Four radiographs showed evidence of a prior healed stress fracture.

Logistic regression determined that the best model for classification combined the totals from the SPSS personal factors section and the clinical examination, together with sex. For female participants, the calculated probabilities classified a total score of 0-5 on the SPSS as category 1 (MRI grade 0 or 1), a score of 6-16 as category 2 (MRI grade 2), and a score of 17-29 as category 3 (MRI grade 3 or 4). For male participants, the respective SPSS score classifications were 0-1, 2-13, and ≥14. When utilizing the classification matrix for categorizing MRI grades based on the SPSS score and sex (Table 3), prediction of BSI can be noted in the horizontal axis, which corresponds to the predicted value of MRI grading. The vertical axis corresponds to the actual value of MRI grading. As noted in Table 4, the SPSS correctly identified 20 of 46 (43.5%) for category 1, 50 of 80 (62.5%) for category 2, and 17 of 34 (50.0%) for category 3. Overall, the SPSS appropriately predicted into 1 of 3 categories for 87 of 160 (54.4%) tibias in this study, with category 2 providing the best prediction for correct MRI grading.

Table 3

Classification Matrix of MRI Grades

	Grade 0-1	Grade 2	Grade 3-4
Grade 0-1	19	25	2
Grade 2	5	65	4
Grade 3-4	1	26	11

Data are shown as No. of tibias. Those with MRI grade x (vertical axes) vs number of tibia scored as x by SPSS (horizontal axes). MRI, magnetic resonance imaging.

Table 4

MRI Results by Grade, Category, and Sex

MRI Grade	SPSS Category	No. of Female Tibias		No. of Male Tibias		No. of Tibias With MRI Grade		No. of Tibias With SPSS Category
		Actual	Predicted	Actual	Predicted	Actual	Predicted	Actual	Predicted
0	1	24	13	3	0	27	13	46	20
1		17	7	2	0	19	7
2	2	44	31	36	19	80	50	80	50
3	3	17	3	12	10	29	13	34	17
4		2	2	3	2	5	4
Total		104	56	56	31	160	87	160	87

MRI, magnetic resonance imaging; SPSS, Shin Pain Scoring System.

Binary analysis for validity demonstrated the following for the SPSS relative to the “gold standard” MRI results based on the previously described 2 × 2 contingency table: sensitivity, 96%; specificity, 26%; PPV, 76%; and NPV, 71%.

Discussion

The term “stress fracture” is often used synonymously with “stress reaction,” which can create confusion in the interpretation of literature.[1,2] Similarly, the use of the term “shin splints” lacks a formal diagnosis and may present a confusing picture for the active adolescent, and therefore, its use by clinicians should be discouraged.[39] For this reason, we have opted to use the term “BSI” to describe a bony injury of the tibia, as our findings reflect athletes with varying degrees of a stress reaction, including stress fractures.

The literature notes the value of the fulcrum test, palpation, and single-leg hop test for pain in a clinical examination.[22,23,38] Fredericson et al[10] reported that increased pain with percussion and a remark of pain with ambulation were related to at least an MRI grade 3 injury in a small sample of patients. However, individually, the predictive ability of clinical tests is limited and is not validated or consistent in identifying the significance of BSIs. The SPSS incorporated the use of all of these clinical tests and also expanded the evaluation process by including an assessment of motion and function to improve predictive ability.

Surprisingly, training history did not significantly influence the prediction of an injury or severity of BSI in our predictive model. We surveyed participants on distance covered during exercise per week, changes in training, and hours per week of sports participation. Although training history is a contributing factor in BSIs,[2,18] our study found no objective value from these questions to improve the prediction of injuries or severity based on training history, and consequently, this was removed from the model.

A positive finding on the SPSS can potentially help to clarify decision making for clinicians and can substantiate the need for imaging to health care and insurance providers. It also provides clinicians an opportunity to discuss factors associated with BSIs, such as low body mass index, disordered eating, amenorrhea, training issues, sleep habits, and psychosocial stress, which all may affect bone health.

MRI is highly sensitive and specific, but it has also been shown to identify bone marrow edema in asymptomatic athletes, raising the suspicion that MRI may overclassify some patients.[3] In our sample, 26 participants reported pain in 1 tibia when presenting for an evaluation, while the clinical examination and MRI identified 15 of 26 contralateral asymptomatic tibias that had evidence of a BSI. This finding raises the possibility that athletes who claim to be asymptomatic may actually have positive findings on a clinical examination and MRI. Exercising adolescents who repetitively stress their bone may actually develop evidence of an injury that does not result in an expression of frank pain. Clinical testing may stress the bone sufficiently to induce positive findings, and MRI may help to identify those who may have bone marrow edema despite no day-to-day symptoms. This finding further underscores the confusion that exists regarding the injury process involved with BSIs. A total of 16 (12%) tibias were reported to be symptomatic but showed no evidence of an injury on MRI and were graded as 0. There were 4 female participants who scored >17 of 29 on the SPSS but who had no evidence of a BSI on MRI. Our experience indicates that there is a subset of patients, mostly female, who demonstrate significant pain and dysfunction that is evident on a clinical examination but not seen on MRI. This finding highlights the need to explore other imaging measures for identifying the source of pain and dysfunction found during a clinical examination in these patients.

Limitations

This study has several limitations that should be noted. We evaluated athletic adolescents with a history of >1 week of shin pain, with no direct comparison with a healthy adolescent control group to limit selection bias. Future studies should compare performance of the SPSS with a healthy adolescent control group.

The utilization of 3 categories versus 5 (MRI grades 0-4) was recommended after conducting statistical analysis because of the low number of reported grade 1 and grade 4 injuries, which resulted in a better predictive model. Statistical analysis showed a better ability to discriminate between sexes and MRI grades when using 3 categories versus 5. However, the number of SPSS classifications may change with an increase in sample size, with more male participants and particularly participants with grade 1 and 4 injuries. In light of these limitations, this study serves as a good pilot study that can stimulate further research and validation.

Conclusion

The SPSS is a potentially valid method to identify tibial BSIs given the sensitivity and NPV and PPV. It also provides helpful categorization to alert clinicians to the presence of a BSI and direct further diagnostics and/or interventions. The SPSS should be considered as an additional tool to use when evaluating adolescents with atraumatic tibial BSIs. Further studies are necessary to confirm our findings.