Reliability and efficiency of three methods of calculating migration percentage on radiographs for hip surveillance in children with cerebral palsy.

PURPOSE: Hip surveillance programmes for children with cerebral palsy (CP) utilize the migration percentage (MP) measurement to initiate referrals and recommend treatment. This study assesses the reliability and efficiency of three methods of MP measurement on anteroposterior (AP) pelvis radiographs.
METHODS: A total of 20 AP pelvis radiographs (40 hips) of children with CP were measured by three raters on two occasions using three methods: digital measurement (DM) on a Picture Archiving and Communication System monitor, computer-aided measurement (CA) using a digital templating tool and mobile device application measurement (MA) using a freely available MP measurement tool. For each method, the time required to complete the MP measurement of both hips on each AP pelvis radiograph was measured. Intra-class correlation coefficient (ICC) was used to determine reliability, and analysis of variance was used to compare groups.
RESULTS: All three methods of determining MP showed excellent inter-rater and intra-rater reliability (ICC 0.976 to 0.989). The mean absolute difference in MP measurement was not significant between trials for a single rater (DM 2.8%, CA 1.9%, MA 2.2%) or between raters (DM 3.6%, CA 2.9%, MA 3.6%). The mean time to complete MP measurement was significantly different between methods, with DM = 151 seconds, CA = 73 seconds and MA = 80 seconds.
CONCLUSION: All three MP measurement methods were highly reliable with clinically acceptable measurement error. The time required to measure a hip surveillance radiograph can be reduced by approximately 50% by utilizing a computer-based or mobile application-based MP measurement tool.

Introduction

Cerebral palsy (CP) is the most common form of motor disability[1] and up to one in three of these children will develop hip displacement or migration of the femoral head from the acetabulum.[2] Neuromuscular hip dysplasia is the second most common skeletal deformity in children with CP and represents a significant burden of disease on this population.[3] A hip surveillance programme consisting of regular supine anteroposterior (AP) pelvic radiographs from childhood to skeletal maturity in children diagnosed with CP has an established record of identifying hip dysplasia in its early stages, allowing for more effective treatment strategies.[4-5] An increasing number of state-wide and national programmes,[6-8] along with professional societies,[9-10] have established hip surveillance as the standard of care for children with CP.

All hip surveillance programmes emphasize the importance of the migration percentage (MP), or the percentage of the ossified femoral head migrated beyond the edge of the acetabulum,[12] in stratification of risk of developing progressive hip displacement. When the MP is greater than 30%, a hip is considered to be at an increased risk for progressive displacement and is promptly referred to a specialty centre for closer monitoring.[2] Reliable and accurate determination of MP, therefore, is critical for efficient use of resources in regions implementing hip surveillance guidelines.

With appropriate positioning of a child with CP for an AP pelvis radiograph, measurement of the MP has been shown to be accurate and reliable.[11] The reliability and efficiency of the method of measurement of MP, however, have not been clearly studied. Methods of measurement of MP include manual measurement on physical radiographs using pen and ruler, digital measurement on a Picture Archiving and Communication System (PACS) monitor using digital tools and measurement using commercially available digital templating software. A recently released freely available mobile device application (HipScreen, www.hipscreen.org, Shriners Hospitals for Children, Sacramento, California) also allows for measurement of MP using the device’s camera and touchscreen. This study aims to compare the reliability and efficiency of MP measurement using three methods: digital measurement on a PACS monitor (DM), computer-aided measurement with digital templating tools (CA) and mobile device application measurement (MA).

Materials and methods

Ethical approval was obtained from the institutional research committee in accordance with the ethical standards of the 1964 Helsinki declaration. Power analysis revealed that for a power of 80%, α = 0.05, and an established intra-observer variability of MP measurement of mean 3.2% (sd 3.5%),[11] a sample size of 30 hips was required to quantify an MP to within 5%. A total of 40 hips (20 AP pelvis radiographs in 20 patients) were selected using convenience sampling by retrospective review by two fellowship-trained paediatric orthopaedic surgeons (VAK and JRD). All radiographs were obtained in children with CP undergoing routine hip surveillance at a tertiary referral centre for paediatric orthopaedics following optimal positioning protocols.[11]

Reliability and efficiency testing

The MP was independently measured on two separate occasions by three raters, two fellowship-trained paediatric orthopaedic surgeons (JRD with 26 years of experience, VAK with five years of experience) and one paediatric orthopaedic fellow (ADB). Prior to making study measurements, the three raters met to agree upon measurement methodology using radiographs that were not part of the study set. Measurement of MP requires determination of Hilgenreiner’s line, Perkin’s line and the medial and lateral aspects of the ossific nucleus of the femoral head (Fig. 1a). Because of the morphological variability in the lateral edge of acetabulum, the three raters agreed upon a standardized flowchart that utilized characterization of the acetabular sourcil, acetabular bony edge and presence of Gothic Arch morphology[12] to arrive at the landmark used for selection of the lateral edge of the acetabulum (Fig. 2).

Fig. 1

Three methods of measurement of migration percentage: (a) digital measurement (DM). Hilgrenreiner’s Line (H), Perkin’s Line (P) and the medial (M) and lateral (L) edges of the ossific nucleus of the femoral head are drawn to measure the appropriate distances and calculate the MP of each hip; (b) computer-aided digital templating measurement (CA). A digital templating tool allows for manipulation of each line to the appropriate position on the radiograph, and the MP is automatically calculated by the tool; (c) mobile device application measurement method (MA). Using the touchscreen of the device, each hip is expanded and positioned on the MP ruler so that vertical lines divide the femoral head into 10% increments, allowing for determination of MP of each hip without measurement of distance.

Fig. 2

Flowchart for determination of radiographic landmark selection for lateral edge of acetabulum.

Radiographs were presented in a randomized and blinded manner to each rater by a research assistant on two occasions separated by one week. A stopwatch was used to measure the time required to measure the MP of both hips on an AP pelvis radiograph using each of the following three methods (Fig. 1):

DM (Fig. 1a): each rater used the PACS monitor digital measurement tools to draw three perpendicular lines to Hilgenreiner’s line that intersected the lateral aspect of the femoral head ossific nucleus, the lateral edge of the acetabulum and the medial aspect of the femoral head ossific nucleus. Distances were measured using the digital ruler tool and a calculator was used to arrive at the MP for each hip.

CA (Fig. 1b): an MP tool in OrthoView Digital Templating suite (Materialise OrthoView, Leuven, Belgium) was utilized to calculate the MP of each hip. Each rater moved the tool’s Hilgenreiner’s line and three perpendicular lines to the appropriate position on each hip and the MP was automatically calculated by the tool.

MA (Fig. 1c): each rater downloaded the HipScreen application to their mobile device (Apple iPhone 6s, Apple iPhone 7S Plus and Samsung Galaxy S8). Each rater took a picture of the pelvis radiograph within the HipScreen application using the embedded camera on their mobile device. The HipScreen application does not store photos on the device to reduce the concerns of health information privacy. The raters then used the rotate function to level the pelvis to Hilgenreiner’s line. Then an ‘MP Ruler’ overlay tool was activated, and the device’s touchscreen was used to expand and position the image until the lateral aspect of the femoral head touched the vertical white line and the medial aspect of the femoral head touched the vertical black line. When so placed, the femoral head is divided into 10% increments by ten vertical lines. Each rater measured the MP to the nearest 5% by determining the vertical line closest to the lateral edge of acetabulum. The same process was repeated for the contralateral hip.

Statistical analysis

Intra-rater reliability was evaluated using intra-class correlation coefficient (ICC) and 95% confidence interval (CI) for all three raters separately. ICC for inter-rater reliability was based on the MP measurements by each of the three raters at the first session. The variability in mean absolute differences between measurement methods and time required to complete measurements were calculated using analysis of variance (ANOVA). For all tests, the significance level was set to p ≤ 0.05. Statistical analysis was performed using SPSS for Windows statistical software Version 24 (IBM, Chicago, Illinois).

Results

A total of 40 hips of 20 children (14 male, six female) at Gross Motor Function Classification System (GMFCS) levels II (n = 1), III (n = 2), IV (n = 11) and V (n = 6) were included in the study. Their mean age was 6.1 years (2.3 to 10.2, sd 2.6 years).

All MP measurements by each of the raters for each method were normally distributed, with a peak frequency of MP measurement between 20% and 50%, an important range to determine whether a child requires a referral to specialty care for progressive hip displacement (Fig. 3). Both the intra-rater reliability (ICC 0.976 to 0.990) and the inter-rater reliability (ICC 0.972 to 0.989) for each measurement method was excellent.[13] ANOVA testing demonstrated no statistical difference in measurement between methods or between observers (p = 0.663 to 0.782). When comparing methods of measurement, the average absolute difference in MP measurement between trials was not significant and was clinically acceptable (Table 1). The mean time to complete MP measurement of both hips on each AP pelvis radiograph was significantly different between methods with CA = 73 seconds (sd 13), MA = 80 seconds (sd 14) and DM = 151 seconds (sd 26), demonstrating that MP measurement tools can reduce the time required to measure by approximately 50%.

Fig. 3

Frequency of migration percentage of hips in study.

Table 1

Reliability and efficiency of three methods of measuring migration percentage (MP)

Technique	Intra-rater reliability (ICC (95% CI))	Inter-rater reliability (ICC (95% CI))	Absolute % difference in MP (mean % (sd))	Average time per AP pelvis (seconds (sd))
Digital measurement (DM)	Rater   A: 0.987 (0.974 to 0.993)   B: 0.990 (0.980 to 0.994)   C: 0.993 (0.987 to 0.996)	0.966(0.943 to 0.981)	Intra-rater: 2.2 (1.8)Inter-rater : 3.6 (5.2)	151 (26)[*]
Computer-aided digital templating measurement (CA)	Rater   A: 0.990 (0.980 to 0.994)   B: 0.994 (0.988 to 0.997)   C: 0.989 (0.980 to 0.994)	0.989(0.981 to 0.994)	Intra-rater: 1.9 (2.2)Inter-rater: 2.9 (2.6)	73 (13)[*]
Mobile device application measurement (MA)	Rater   A: 0.979 (0.959 to 0.988)   B: 0.986 (0.974 to 0.993)   C: 0.976 (0.956 to 0.988)	0.972(0.953 to 0.984)	Intra-rater: 2.8 (3.2)Inter-rater: 3.6 (4.2)	80 (14)[*]

Intra-class correlation (ICC) values greater than 0.90 are considered excellent reliability

p = 0.001 for DM versus CA and DM versus MA; p = 0.02 for CA versus MA

CI, confidence interval; AP, anteroposterior

Discussion

The cornerstone of a hip surveillance programme is the ability to effectively interpret an AP pelvis radiograph and quantify the MP. Most hip surveillance programmes use an MP of 30% to initiate a referral to a specialized paediatric centre, with surgical interventions beginning at an MP of 40%.[14]

The MP has been established as a measurement with good to excellent intra- and inter-rater reliability in a systematic review of the literature.[15] Using manual measurement with physical radiographs, pens and goniometers, the standard error of measurement × 1.96 for experienced raters is 5.8% for one rater and 11% between raters,[12] and the 95% CI for orthopaedic trainees is 12.9% for one rater and 22% between raters.[16] Using digital measurement tools on a digital PACS system, Analan et al[17] found that an experienced physiatrist and a physiatry resident had high inter-rater correlations of 0.94 at each of two measurement sessions. Though Cliff et al[11] did not discuss the method of measurement, their study established that when experienced radiologists measure MP in standardized radiographs obtained using a positioning protocol, the mean absolute difference in MP was 3.2% (sd 3.5%) for a single rater and 3.7% (sd 3.8%) for multiple raters.

Our study establishes all of the three common methods of MP measurement as reliable and reproducible, and found that utilization of MP measurement tools can improve the speed of measurement by approximately 50%. We found the measurement error of MP measurement to be between 1.9% and 2.8% for a single rater and 2.9% and 3.6% for multiple raters. The magnitude of this measurement error is comparable with the established standards in the literature, indicating that all three methods of measurement are valid for clinical interpretation of MP. In addition, among raters with familiarity with the measurement techniques and interpretation of hip surveillance radiographs, the clinical experience level of the rater did not change the reliability of the MP measurement.

Despite its smaller screen size, the mobile device application measurement tool can provide clinically acceptable validity when compared with measurement on larger screens. To adapt the MP measurement to a smaller screen, the HipScreen application measures MP in a fundamentally different manner. In the DM and CA methods of measurement, the radiograph and the measurement tool maintain their relative sizes to each other, so that enlarging the image on the screen proportionally increases the size of both the radiographic image and the measurement tool. When enlarging the image in the MA method of measurement using the HipScreen application, the radiographic image increases in size, but the measurement ruler size does not change. This ‘MP Ruler overlay’ technique allows a user to scale the radiographic image to fit the measurement ruler and arrive at the MP measurement without moving additional lines on a reduced-sized touchscreen. This MP measurement technique adapted for a touchscreen device is reliable and valid relative to either of the commonly employed methods of measurement, and nearly as fast as measurement with a digital templating MP measurement tool on a PACS monitor. Utilizing a mobile device for MP measurement, however, does not allow a user to visually check the measurement at a later date since no images are saved on the device. A PACS method of measurement maintains the distinct advantage of allowing users to save the measurements to the medical record of the patient, allowing for scrutiny of the measurements at a later date and easier observation of MP temporal trends.

This study has several limitations. First, we chose radiographs for this validation study that were properly positioned with clearly visible landmarks so that we could determine difference in measurement attributable to the technique, rather than to the quality of imaging. The MP is known to vary with femoral adduction, pelvic rotation, and pelvic tilt, though some studies suggest that accurate hip surveillance can be performed even without formal positioning protocol.[18] Second, we chose observers experienced in radiographic interpretation and use of MP measurement tools. For wider use in a hip surveillance programme, users with less experience in radiographic interpretation would need training on identifying the key landmarks on the radiographs required for accurate measurement of the MP and instruction in the use of the MP measurement tools. Third, variation in screen size, differences in screen resolution, and differences of ambient lighting may alter the ability to identify key landmarks using any measurement method, leading to lower reliability. We chose to perform all measurements in the same location using the same computers and devices to minimize this variability. Fourth, we did not perform a comprehensive test of all digital measurement tools available on PACS. Some systems may have tools such as a length ratio tool which could simplify the process of manual measurement. As the functionality of these PACS tools approach that of a computer-aided tool designed for measurement of MP, it is likely that the speed of manual measurement would increase. Finally, although raters were asked to measure MP at a comfortable pace that allowed them maximal accuracy, unconscious bias could have been introduced because raters were aware that their speed was being measured.

We conclude that MP can be reliably measured by any of the three methods studied, but utilizing either commercially available digital templating software or a freely available mobile device application can significantly reduce the time required for measurement.