Risser sign inter-rater and intra-rater agreement: is the Risser sign reliable?

BACKGROUND: Studies directly evaluating the reliability of the Risser sign are few in number, possess small sample sizes, and offer conflicting results. This study establishes the reliability of the Risser sign on a large sample size in an effort to provide clarification on the subject.
METHODS: Two years' worth of AP pelvis radiographs from patients age 8-20 were downloaded from our institution's digital imaging system. One hundred of these images were selected for inclusion by an independent reviewer whose goal was to capture a spread of radiographs that included all Risser stages. Risser grading occurred in two rounds. In each round, three examiners randomly reviewed the 100 radiographs on three different occasions. The full AP pelvis radiograph was graded in Round 1 while only the iliac apophysis was visible in Round 2. Kappa coefficients and their confidence bounds are reported to indicate intra- and inter-observer reliability. The contrast between the rates of agreement about Risser stages in Rounds 1 versus 2 was assessed by McNemar's test. The signed-rank test was used to evaluate differences in intra-observer values between rounds.
RESULTS: Round 1 inter-observer kappa was 0.76. Round 2 inter-observer kappa was 0.51. In Round 1, 63 radiographs showed perfect agreement within the same Risser stage for all observations compared to 44 radiographs with perfect agreement within the same Risser stage in Round 2 (p = 0.004). Round 1 intra-observer kappa values were 0.92, 0.86, and 0.88. Round 2 intra-observer kappa values were 0.91, 0.77, and 0.88. Intra-observer value differences between rounds were not significant for two observers (p = 0.074, 0.061) but was significant for the third observer (p = 0.002).
CONCLUSION: The reliability of the Risser sign is acceptable and can be further improved when other markers of skeletal maturity on the pelvis radiograph are used to assist in grading.