Reliability of radiographic union scale in tibial fractures and modified radiographic union scale in tibial fractures scores in the evaluation of pediatric forearm fracture union.

OBJECTIVES: This study aims to evaluate the reliability of the radiographic union scale in tibial (RUST) fractures and modified RUST (mRUST) fractures in pediatric forearm fractures treated with elastic stable intramedullary nail (ESIN) and to investigate the effect of the experience of surgeon, thresholds for union, and delayed union decisions. PATIENTS AND METHODS: In this retrospective study, radiographic images of 20 patients (10 males, 10 females; mean age 8.6±4.3; range, 4 to 11 years) with forearm fractures treated using ESIN between January 2013 and December 2018 were scored by 20 observers based on the RUST and mRUST scores. The observers scored the radiographs at immediate postoperative period, and at 4-, 8-, and 12-week follow-up. Intra- and interobserver agreement for each cortex, RUST, and mRUST were evaluated using intraclass correlation coefficient (ICC). The Fleiss' kappa (κ) coefficient was used in the agreement between evaluators regarding union decision. Receiver operating curves were created to determine the thresholds for radiographic union and delayed union.
RESULTS: Intra- and interobserver reliability of the mRUST score (ICC: 0.84 and 0.79) were slightly higher than that of the RUST score (ICC: 0.80 and 0.72). Pediatric orthopedic and trauma surgeons had slightly higher agreement than the residents and general orthopedists for the total mRUST and RUST scores of the eight-week radiographs. Mean RUST and mRUST scores at the union for all fractures were 10.2±3.4 and 13.0±2.1, respectively. Kappa value for union was moderate (0.74). The total mRUST score had a higher predictive value for union than the total RUST score (area under the curve: 0.986 vs. 0.889). A mRUST score of ≥12 and RUST score of ≥9 were considered as the predictors of union. In addition, a mRUST score of ≤7 and RUST score of <9 were considered as the predictors of delayed union.
CONCLUSION: A moderate agreement for both RUST and mRUST scores was found. However, the agreement for mRUST was found to be slightly higher. Healing and union of forearm fractures treated with ESIN can be reliably assessed using RUST and mRUST.

Introduction

To measure the success of the treatment and predict patient outcome, the ability to describe the fracture healing and union are important.[1] However, there is no uniform description of union.[2,3] Radiographic criteria for fracture healing have been suggested as cortical continuity and fracture line visibility, and even depending on the number of bridging cortices and the impression of the surgeon.[4] Radiologic evaluation of fracture union is still challenging, and it is unknown which fracture healing assessment method is the most reliable in healing, union, and nonunion identification.[5]

The radiographic union scale in tibial (RUST) fractures is a valid and reliable measure,[6] which was recently developed to assess the healing of the adult tibial shaft fracture after intramedullary nailing using a numerical value for each cortex.[6,7] It was based on the bridging callus and fracture line visibility, which were found to be the most reliable signs of bone healing between observers.[8] It does not propose an exact score defining union. As a weakness of RUST score, the evaluation becomes dichotomous, i.e. whether the fracture line is visible or not, after a bridging callus is present.[1] A fracture line that disappears with complete remodeling leads to further subdividing the cortical assessment with the cortical bridging callus exist.[9] To describe the radiographic healing progress more accurately, the modified RUST (mRUST) score was developed.[1] In addition to the standard RUST score, fracture with callus was further subdivided as either presence of callus or presence of bridged callus.[1]

Less callus formation is required to achieve clinically stable or healed fracture in pediatric patients.[10,11] The greater subperiosteal hematoma with thicker and stronger periosteum provides a more rapid callus formation that leads to shorter fracture healing time.[12] Therefore, radiographic features of bone healing in children varies and the union decision is different from adult fractures.[13] In the literature, RUST and mRUST scores were used in the evaluation of fracture healing in tibial, femoral, humeral, and radial fractures.[1,14-17] To the best of our knowledge, the evaluation of pediatric forearm fracture healing using RUST and mRUST scores has not been performed yet. Therefore, in this study, we aimed to evaluate the reliability of the RUST fractures and mRUST fractures in pediatric forearm fractures treated with elastic stable intramedullary nail (ESIN) and to investigate the effect of the experience of surgeon, thresholds for union, and delayed union decisions.

Patients and Methods

Our orthopaedic trauma clinic database at the Baltalimani Bone and Joint Diseases Training and Research Hospital was retrospectively evaluated and pediatric patients with forearm shaft fracture (n=98) treated with ESIN between January 2013 and December 2018 were identified. Patients with a closed forearm shaft fractures aged <12 years who had full union or delayed union at the first surgical attempt and had more than one-year follow-up were included. The exclusion criteria were (i) comminuted or segmented fractures, (ii) open fractures, (iii) re-fractures, (iv) existence of neurovascular injury, (v) pathological fractures, (vi) history of local or systemic infection, malignancy, chemo- or radiotherapy, and (vii) lacking, inappropriate, and inadequate radiographs at 4, 8, and 12 weeks postoperatively. After applying the exclusion criteria, 28 fractures were included in the analysis. Radiographs of 20 patients (10 males, 10 females; mean age: 8.6±4.3; range, 4 to 11 years) were randomly selected by a person who was involved in the surgical treatment and follow-up of patients. The study protocol was approved by the Baltalimani Bone and Joint Diseases Training and Research Hospital Ethics Committee (approval number/date: 24/22.02.2018). A written informed consent was obtained from the legal guardians of each patient. The study was conducted in accordance with the principles of the Declaration of Helsinki.

Twenty radiographic sets of the forearm shaft fractures treated with ESIN were prepared. Twenty orthopedic surgeons with varying experience levels (five residents in their last year, five general orthopedic surgeons, five trauma surgeons, and five pediatric orthopedic surgeons) who were blinded to the patient and radiographic data were invited to review the 20 radiographic sets twice at 30 days apart. They were given the description of the RUST and mRUST scores based on the original papers by Whelan et al.[7] and Litrenta et al.[1] Twenty sets of images including anteroposterior (AP) and lateral radiographs obtained at 0, 4, 8, and 12 weeks postoperatively were included in a Microsoft PowerPoint file (Microsoft® Office 2011 for Mac; Microsoft, Redmond, WA, USA). Radiographs of each fracture were randomly mixed by a blinded person without chronological order. Patient information was hidden. A further randomization was performed for the second evaluation.

In the radiographic evaluation, radial and ulnar cortices (medial, lateral, volar, and dorsal) were scored according to the RUST score of 1 to 3 or mRUST score of 1 to 4. In the RUST score, a cortex with a visible fracture line and no callus was given a score of 1, a cortex with a callus and a visible fracture line was scored 2, and a cortex with bridging callus and without a fracture line within the callus bridge was scored 3.[7] A minimum total score of 4 (not healed) and a total maximum score of 12 (completely healed) can be achieved. In the mRUST score, a cortex with a visible fracture line and no callus was given a score of 1, a cortex with a callus and a visible fracture line was scored 2, a cortex with a callus and without a fracture line within the callus bridge was scored 3. In cases with a remodeled callus but without a visible fracture line, a score of 4 was given. A minimum total score of 4 (not healed) and a total maximum score of 16 (completely healed) can be achieved.[1]

Reviewers were asked to evaluate each cortex and radiograph and to assign a RUST and mRUST score to each AP and lateral radiograph. In addition, they were asked to determine whether the fracture is united or not.

Statistical analysis

Statistical analysis was performed using the IBM SPSS version 24.0 software (IBM Corp., Armonk, NY, USA). The mean, standard deviation, median, range, minimum, and maximum values were used in the descriptive statistics of data. The intraclass correlation coefficient (ICC) with 95% confidence interval (CI) was used to quantify the agreement of the RUST and mRUST scores in the treatment groups, in various surgeon experiences and different cortical evaluations. Fleiss’ kappa (κ) coefficient was used in the evaluation of union agreement regarding fracture union between observers. Based on the work of Koo et al.,[18] we defined the values less than 0.5 as indicative of poor reliability, values between 0.5 and 0.75 as moderate reliability, values between 0.75 and 0.9 as good reliability and values greater than 0.90 as excellent reliability. Receiver operating curves (ROCs) were created to determine the threshold scores for union and delayed union. To define radiographic union, a cut-off value was defined as total RUST and mRUST scores with specificity equal or higher than 0.90 (95% CI). The area under ROC curve (AUC) was interpreted as 0.5 to 0.6, failed test; 0.6 to 0.7, poor accuracy; 0.7 to 0.8, fair accuracy; 0.8 to 0.9, good accuracy; and 0.9 to 1.0, excellent accuracy.[19] The ROC characteristics were used to evaluate the value of the score obtained for each cortex to assess the diagnostic value for radiographic union. A p value <0.05 was considered as statistically significant.

Results

The radiographs of 20 pediatric forearm shaft fractures (15 unions and five delayed unions) were evaluated. Intra- and interobserver reliability of the mRUST score (ICC: 0.84 and 0.79) were slightly higher than that of the RUST score (ICC: 0.80 and 0.72).

Evaluation of the interobserver agreement for each of the four cortices in isolation showed that the lowest agreement for the medial cortex in the ulna was 0.52 and 0.59 for the RUST and mRUST, respectively. However, the lowest agreement in the radius was in the volar cortex (RUST: 0.63, mRUST: 0.60). Interobserver ICC values for the standard and mRUST scores and individual cortices are shown in Table I.

Surgeons with varying experiences had similar agreement in the mRUST and RUST scores (Table II). However, pediatric orthopedists and trauma surgeons had a slightly higher agreement in the total mRUST and RUST scores of the eight-week radiographs than residents and general orthopedists (Table II). The mean RUST and mRUST scores at union for all fractures were 10.2±3.4 and 13.0±2.1, respectively. Kappa for union decision was moderate (0.74). The total mRUST score had a higher predictive value for union than the total RUST score (AUC: 0.986 vs. 0.889) (Figure 1). A mRUST score of ≥12 (AUC: 0.986; sensitivity: 0.865 [95% CI: 0.819 to 0.891]; specificity: 0.950 [95% CI: 0.918 to 0.999], positive predictive value: 82%; negative predictive value: 91%; and positive likelihood ratio: 17.30 [95% CI: 5.47 to 23.89]) and RUST score of ≥9 (AUC: 0.908; sensitivity: 0.853 [95% CI: 0.816 to 0.904]; specificity: 0.954 [95% CI: 0.918 to 0.990], positive predictive value: 77%; negative predictive value: 86%; and positive likelihood ratio: 18.54 [95% CI: 12.89 to 23.25]) were considered as the excellent predictors of union. In addition, a RUST score of ≤7 (AUC: 0.944; sensitivity: 0.869 [95% CI: 0.827 to 0.922]; specificity: 0.936 [95% CI: 0.893 to 0.978], positive predictive value: 81%; negative predictive value: 90%; and positive likelihood ratio: 13.57 [95% CI: 9.33 to 19.47]) and mRUST score of <9 (AUC: 0.898; sensitivity: 0.883 [95% CI: 0.816 to 0.955]; specificity: 0.917 [95% CI: 0.865 to 0.991], positive predictive value: 79%; negative predictive value: 92%; and positive likelihood ratio: 10.63 [95% CI: 5.47 to 16.03]) were considered as the excellent predictors of delayed union. More than 90% of the reviewers considered a score of 10 for RUST and 13 for mRUST for union. Moreover, >90% of the reviewers considered a score of 6 for RUST and 8 for mRUST for delayed union.

Figures 2 and 3 show the examples of union and delayed union of the forearm shaft fractures treated with ESIN.

Discussion

This is the first study that supports the reliability of the RUST and mRUST scores in the assessment of healing in pediatric forearm shaft fractures treated with ESIN fixation.[20] Intra- and interobserver reliability of the mRUST score was found to be slightly higher than the RUST score. The lowest interobserver agreement was found in the medial cortex of the ulna and the volar cortex of the radius. The cut-off values of 9 and 12 were provided as confident assessment of union for RUST and mRUST, respectively. In addition, a cut-off value of 7 for RUST and 9 for mRUST were provided as confident assessment of delayed union. In addition, experience of the surgeon was found not to be different between the overall RUST and mRUST scores. However, there was a slight difference in the total mRUST and RUST scores at the eight-week follow-up radiographs among surgeons with different experiences.

Delayed union and nonunion are not common in skeletally immature patients.[21] They have been associated with infection, open fracture, pathologic fracture, and inappropriate or suboptimal surgical technique.[22] Sufficient callus formation is expected to be present at four weeks postoperatively in pediatric forearm fractures. Radiological fracture consolidation is achieved at seven to eight weeks.[23-25] There is no consensus in the definition of delayed fracture union.[2] Delayed union rates in forearm shaft fractures treated with elastic intramedullary nailing have been reported to be between 1.9 and 4.4%.[26] In addition, nonunion rates have been reported to be between 0.5 and 1%.19 Mid- and proximal third fractures have higher nonunion rates than distal third fractures.[27] In this study, we evaluated the ability of RUST and mRUST scores to describe the fracture healing and union of pediatric forearm fractures treated with ESIN. Thresholds for union were identified as 9 for RUST and 12 for mRUST. In addition, thresholds for delayed union were identified as 7 for RUST and 9 for mRUST.

The RUST and mRUST scoring systems were developed to assess radiographic fracture healing of tibial shaft fractures treated with intramedullary nailing.[1,7] They have been used in other long bone fractures as well. In our study, we evaluated the interobserver reliability of the RUST and mRUST scores in pediatric forearm shaft fractures treated with ESIN for the first time. The effect of the reader’s experience level, time, and cortex dependent agreement levels were evaluated to determine the differences. Pediatric orthopedic and trauma surgeons had a slightly higher agreement than residents and general orthopedic surgeons in the total mRUST and RUST scores in the eight-week radiographs. The lowest interobserver agreement levels were found for the medial cortex of the ulna and volar cortex of the radius.

On the basis of assigning a greater range of scores during the time of healing when the callus was bridged, the mRUST score was developed to achieve a more precise definition of union.[1] In the mRUST score, each cortex is scored between 1 and 4, instead of 1 and 3 as in the standard RUST score. It specifically qualifies the amount of callus present, classified as either callus bridging or non-bridging. Given that fracture union occurs between the appearance of callus and remodeled cortices, using the mRUST score in the assessment of union provides better decision about deciding whether the fracture is healing and healed.[1,4] In our study, we found that the mRUST score is slightly more reliable than the standard RUST score according to the higher interobserver agreement of the latter. However, both scores had a moderate agreement.

Litrenta et al.[1] firstly investigated the definition of union in ICC and the percentage of raters that assigned union. They defended that union is best defined by the percentage of reviewers assigning it various scores despite moderate absolute agreement between reviewers. They reported a minimum threshold for union of 9 for RUST and 11 for mRUST in the meta-diaphyseal fractures of femur and tibia, which may be reasonable because the majority of reviewers assigned union at this point. They also suggested that definite union would be 10 and 13 with >90% of reviewers assigning union. In our study, we found that a value of ≥9 was considered

to indicate union using RUST score and a value of ≥12 was considered to indicate union using mRUST score. Also, a value of ≤7 for RUST score and ≤9 for mRUST score were considered to indicate delayed union. More than 90% of the reviewers considered a score of 10 for RUST and 13 for mRUST for union. Moreover, >90% of the reviewers considered a score of 6 for RUST and 8 for mRUST for delayed union.

There are several strengths and limitations of this study. This is the first study evaluating pediatric forearm shaft fracture healing, union, and delayed union using RUST and mRUST scores. We proposed a threshold score for union using the percentage of raters with various experience levels. We showed the lowest agreement according to time and cortex. However, there were a limited number of cases for delayed union due to its low incidence. Moreover, non-orthopedic observer evaluation was not performed, which could decrease the possibility of bias. However, observers were selected from a wide range of experienced surgeons. Correlation with physical examination findings, pain scores, and functional outcome scores could provide more precise healing and union decision.

In conclusion, a moderate agreement for both RUST and mRUST scores was found. However, the agreement for mRUST was found to be slightly higher. Pediatric forearm shaft fractures treated with ESIN can be reliably assessed with these two scoring systems. Threshold for union and delayed union may help surgeons in the postoperative management of pediatric forearm fractures.

Table 1

Intraclass correlation coefficient values with 95% confidence interval for radiographic union scale in tibial fractures and modified radiographic union scale in tibial fractures scores of all patients and individual cortices

	RUST	mRUST	RUST	mRUST
	Intraobserver ICC (95% CI)	Intraobserver ICC (95% CI)	Interobserver ICC (95% CI)	Interobserver ICC (95% CI)
Radius
Total	0.75 (0.69-0.81)	0.78 (0.73-0.83)	0.71 (0.65-0.77)	0.76 (0.72-0.80)
Medial	0.70 (0.65-0.75)	0.72 (0.66-0.78)	0.67 (0.62-0.72)	0.68 (0.58-0.78)
Lateral	0.75 (0.67-0.83)	0.79 (0.74-0.84)	0.70 (0.63-0.77)	0.74 (0.68-0.80)
Volar	0.67 (0.59-0.75)	0.69 (0.63-0.75)	0.63 (0.59-0.67)	0.60 (0.57-0.63)
Dorsal	0.69 (0.63-0.75)	0.71 (0.68-0.75)	0.65 (0.57-0.73)	0.67 (0.60-0.74)
Ulna
Total	0.77 (0.71-0.83)	0.77 (0.74-0.80)	0.69 (0.64-0.74)	0.72 (0.68-0.76)
Medial	0.70 (0.62-0.78)	0.66 (0.59-0.74)	0.52 (0.43-0.61)	0.59 (0.54-0.64)
Lateral	0.67 (0.60-0.74)	0.70 (0.65-0.75)	0.58 (0.48-0.68)	0.63 (0.55-0.71)
Volar	0.69 (0.62-0.76)	0.72 (0.66-0.78	0.60 (0.51-0.69)	0.62 (0.52-0.72)
Dorsal	0.75 (0.65-0.85)	0.77 (0.71-0.83)	0.67 (0.61-0.73)	0.70 (0.60-0.80)
RUST: Radiographic union scale in tibial fractures; mRUST: Modified radiographic union scale in tibial fractures; ICC: Intraclass correlation coefficient; CI: Confidence interval.

Table 2

Agreement levels (intraclass correlation coefficients with 95% confidence intervals) of surgeons with different levels of experience in 4-, 8- and 12-week follow-up radiographs

	4-week RUST	4-week mRUST	8-week RUST	8-week mRUST	12-week RUST	12-week mRUST
Residents	0.79 (0.75-0.84)	0.81 (0.73-0.89)	0.66 (0.58-0.74)	0.70 (0.60-0.80)	0.86 (0.80-0.91)	0.87 (0.82-0.92)
General orthopedic surgeons	0.78 (0.73-0.83)	0.81 (0.75-0.87)	0.62 (0.52-0.72)	0.65 (0.58-0.72)	0.79 (0.72-0.86)	0.83 (0.76-0.90)
Pediatric orthopedic surgeons	0.83 (0.77-0.89)	0.85 (0.81-0.89	0.77 (0.70-0.84)	0.80 (0.75-0.85)	0.85 (0.76-0.94)	0.91 (0.88-0.94)
Trauma surgeons	0.80 (0.75-0.85)	0.84 (0.79-0.89)	0.74 (0.65-0.83)	0.75 (0.69-0.81)	0.89 (0.85-0.93)	0.90 (0.85-0.95)
RUST: Radiographic union scale in tibial fractures; mRUST: Modified radiographic union scale in tibial fractures.