Intra- and inter-observer agreement and reliability of bone mineral density measurements around acetabular cup: a porcine ex-vivo study using single- and dual-energy computed tomography.

BACKGROUND: Periprosthetic bone loss is considered to be a potentially contributing factor in aseptic loosening of acetabular hip components, but no studies have shown this association. The lack of association might be caused by insufficient image quality because of metal artifacts and challenges in measuring bone density (BMD) in complex anatomic structures which might be overcome using dual-energy computed tomography (DECT).
PURPOSE: To test inter- and intra-observer agreement and reliability of in-house segmentation software measuring BMD adjacent to acetabular cup and to compare measurements performed with single-energy CT (SECT) and DECT in cemented and cementless cups.
MATERIAL AND METHODS: Twenty-four acetabular cups inserted in porcine hip specimens were scanned with SECT and DECT. Bone density was measured in a three-dimensional volume adjacent to the cup. Double measurements were performed.
RESULTS: BMD derived from SECT was approximately four times higher than that of DECT. In both scan modes, intraclass correlation coefficient (ICC) was >0.90 with no differences between repeated measurements, except for uncemented cups where a statistically significant difference of 11 mg/cm3 was found with DECT. DECT showed narrower limits of agreement than SECT. Inter-observer analysis showed small differences.
CONCLUSION: BMD can be estimated with high intra- and inter-observer reliability with SECT and DECT around acetabular cups using custom software. The intra- and inter-observer agreement of DECT is superior to that of SECT and better in the cementless concept. Good intra- and inter-observer reliability can be obtained in both cemented and cementless cups using the segmentation software. SECT and DECT cannot be used interchangeably.

Introduction

Aseptic loosening of orthopedic implants is the most common cause of failure after total hip arthroplasty (THA) (1,2) and periprosthetic bone loss is considered a predictor of aseptic loosening (3,4). To our knowledge, no studies have shown this association, but a recent study found osteoporotic patients with low systemic bone mineral density (BMD) to be at higher risk of cup migration compared with patients with normal BMD (5). Bone loss may occur around THA because of stress shielding and may be located around both the cup and stem (4,6). Most attention has been paid to the femoral side (4), but cementless press-fitted cups loading the acetabular rim may also cause stress shielding of the central iliac bone (7). Bone loss occurs in all compartments adjacent to the cup (8,9), but in cementless THA it occurs most often in the supra-acetabular ileum (10) and often also in the medial wall in both cemented and cementless cups (11). In cemented cup systems, bone loss seems to be less substantial (12). Bone loss must be considerable, i.e. 30–50%, before it can be detected on radiographs (13) and furthermore the sensitivity of radiographs is low which might explain the lack of correlation between bone loss and loosening of the prosthesis.

Dual X-ray absorptiometry (DXA) is widely used for bone density measurements in osteoporosis, but it is of limited value in the presence of metal because it acquires two-dimensional (2D) images only and it cannot obtain reliable measurements in the presence of bone cement (14). Bone density changes adjacent to acetabular hip implants have been studied with computed tomography (CT) (12,15–18), but CT is challenged in visualizing the interface between bone and implant due to metal artifacts (14), thus none of the studies have focused solely on the interface. Dual-energy CT (DECT) can theoretically remove or reduce beam hardening artifacts (19–21) and has shown promising results in bone density evaluation adjacent to spinal implants (22,23) and better delineation of the acetabular bone-implant interface in THA (24). In general, measuring BMD using CT in the acetabular region is also difficult because of the complex three-dimensional (3D) anatomic structure of the bone.

The primary objective of the present study was to test the intra- and inter-observer agreement and reliability of an in-house 3D segmentation software solution that measures BMD in close proximity to the acetabular cup with the use of volumetric CT images. The secondary objective was to compare BMD measurements performed with single-energy CT (SECT) and DECT in cemented and cementless prosthetic acetabular cups. If measurements can be performed with sufficiently high intra- and inter-observer reliability, the method might contribute to further research in bone loss and potentially impact the choice of acetabular cup in future patients.

Material and Methods

Twenty-four fresh frozen female porcine hemi-pelvic specimens were included. The animals were skeletally mature sows with closed triradiate cartilage. The exact age of the animals was unknown, but usually they are slaughtered when they reach 2–4 years. One orthopedic surgeon with more than ten years of experience in hip replacement surgery inserted 12 cementless and 12 cemented cups in the specimens. The specimens were stored at −25℃ between surgery and the imaging procedure.

Surgical procedure

The specimens were partly thawed at room temperature for approximately 3 h prior to surgery to ensure that surgery was practically feasible yet avoiding the possibility of decomposition of the specimens. Surgery was performed with Stryker components and cement (Stryker Corp., Kalamazoo, MI, USA). In the cementless concept 48-mm Trident® hemispherical acetabular shells were used in combination with Trident® X3® polyethylene liners. The acetabulum was reamed line-to-line or under-reamed according to the surgeon’s judgment. In the cemented concept, we used 48-mm Contemporary® Hooded cups. The acetabulum was over-reamed by 2 mm according to the manufacturer’s recommendations. Antibiotic Simplex® Bone Cement was used. In both concepts, an Exeter stem was placed in the cup for SECT and DECT examination. In the cementless concept, a 32-mm steel head was used and in the cemented concept a 28-mm head was used. The divergence was due to the relatively small porcine acetabulum, i.e. approximately 50 mm. As the cemented concept demands over-reaming by 2 mm, 48-mm cups were used. However, this cup size can only contain a 28-mm head. The stem was not fitted in the femoral bone because the femoral side was beyond the scope of the current study.

CT scanning

All scans were performed with a GE Discovery CT750 HD 64-channel scanner (GE Healthcare, Milwaukee, WI, USA). DECT and SECT were performed with the parameters shown in Table 1, and synthetic monochromatic DECT images were reconstructed at 130 keV. The keV level was chosen because energy levels above 110 keV provide stable density measurements (23) with an optimum at 130 keV in presence of orthopedic metal devices (25,26). Double measurements were achieved by dismantling and rebuilding the experimental setup and repeating the scans. Each specimen was positioned on a vacuum bag and a calibration phantom (MINDWAYS® QCT Pro, Mindways Software Inc., Austin, TX, USA) was placed beneath the specimen. The phantom contains five calibration rods with known densities. The mean pixel value in each rod was measured and BMD in any region of interest (ROI) could be derived (Fig. 1).

Table 1.

Acquisition parameters for SECT and DECT.

Parameter	Single-energy CT	Dual-energy CT
kVp	120	80/140 dual
Tube current (mA)	300	375
Scan time (s)	0.5	0.6
Pitch	0.984:1	0.984:1
Collimation (mm)	40	40
Image acquisition (mm)	64 × 0.625	64 × 0.625
Scan field of view	Body large (50 cm)	Body large (50 cm)
Kernel	“Bone”	“Detail”
ASIR	30%	Not available
CTDIvol (mGy)	11.35	12.92

ASIR, adaptive statistical iterative reconstruction; CTDIvol, volumetric CT dose index.

Fig. 1.

Example of an axial DECT slice with the hemispherical volume defined by the pixels included in the intersection between the circles (red) and the borders of the free-hand drawn area (blue). Underneath the specimen, five ROIs (yellow) are positioned in the calibration phantom rods.

Quantitative image analysis

We used a custom in-house 3D Fiji plugin to perform the image analysis. Fiji is a platform for biological image analysis based on ImageJ (27). Using the plugin we could draw free-hand ROIs slice by slice and position circular ROIs in the phantom calibration rods (Fig. 1). The plugin stacked the ROIs and segmented a cup-shaped 3D ROI based on a user-defined diameter and thickness, and the ROI was divided into four quadrants (Fig. 2). In this study, a combination of 50-mm diameter and 5-mm thickness was used. The plugin created a comma-separated file with the pixel values of the ROIs. A Python script (Python Software Foundation, Beaverton, OR, US) was then used to calculate the mean pixel value of each ROI using a lower pixel value threshold of –300 Hounsfield units to exclude air deposits. Image analysis was performed by a radiographer with more than 20 years of experience in CT and image analysis (BM). In the cementless concept, the analysis was repeated by a second observer (KK) with no radiologic experience in order to assess inter-observer agreement and reliability.

Fig. 2.

Example of a ROI divided into quadrants superimposed onto axial and coronal slices in SECT (left) and DECT (right). Beam hardening artifacts are present in both scan modes, but appear slightly more prominent in SECT (arrow).

Statistical analysis

According to Kress et al. (16), we used a minimal relevant difference of 5%. Based on a sample size calculation using an estimated SD of 15, a difference of 15 mg/cm3 (i.e. 5%) could be detected by the inclusion of ten specimens in each group. All variables were continuous and summarized by mean, number of observations, and 95% confidence intervals (CI). The differences between repeated BMD measurements were estimated with mixed effects regression modeling, using measurement as fixed effect. The absolute agreement between the methods and between the observers was analyzed with Bland–Altman plots (28), including mean difference and limits of agreement. The intra- and inter-observer reliability was assessed by intraclass correlation coefficients (ICC) based on two-way random effects models (29,30). The statistical significance of the difference between SECT- and DECT-based BMD measurements was assessed by comparing the 95% CIs. Repeatability coefficients (RC) were calculated for the repeated measurements according to Bartlett and Frost (30). RC is an estimate below which the absolute difference would be expected to lie with 95% certainty (30–32). To compare RC and the minimal clinically relevant difference, the RCs were converted to percentages by dividing RC with the mean BMD of measurement 1. All analyses were performed using STATA/SE 14.0 (StataCorp. LP, College Station, TX, USA). The study is reported in accordance with the Guidelines for Reporting Reliability and Agreement Studies (GRRAS) (33).

Results

Two specimens with cementless cups were excluded because of fractures which had occurred during the surgical procedure, but remained undiscovered until imaging had been performed. The repeated BMD measurements in the “all quadrants” ROI showed no statistically significant differences in the cementless cups. In the cemented concept, we found no significant difference in SECT, while DECT showed a difference of 11 mg/cm3 (P = 0.014) between the measurements (Table 2). The Bland–Altman plots (Figs. 3 and 4) showed narrower limits of agreement in DECT (−21 to 44) compared with SECT (–86 to 153) and the RCs (Table 3) were significantly smaller than those of SECT. ICC and the corresponding 95% CI in SECT was 0.90 (95% CI = 0.67–0.98) and 0.91 (95% CI = 0.70–0.97) for the cementless and cemented concept, respectively, while ICC in DECT was 0.91 (95% CI = 0.69–0.98) for the cementless and 0.93 (95% CI = 0.68–0.98) for the cemented concept. The results of the inter-observer analysis are listed in Table 4 and plotted in Fig. 5, with corresponding Bland–Altman plots given in Fig. 6. The inter-observer ICC was 0.91 (95% CI = 0.47–0.98) and 0.93 (95% CI = 0.31–0.99) for SECT and DECT, respectively.

Table 2.

Intra-observer analysis. Differences between repeated BMD measurements derived from SECT and DECT (units of mg K2HPO4/cm3).

Scan mode	Volume of interest	Mean BMD measurement 1	Mean BMD measurement 2	Difference	95% CI	P value
Single-energy CT	Cementless cups, n = 10
	All quadrants	1188	1180	−8	−40 to 24	0.64
	Superior quadrant	1351	1349	−2	−53 to 49	0.95
	Inferior quadrant	1187	1211	24	−43 to 92	0.48
	Anterior quadrant	1270	1220	−50	−93 to −8	0.02
	Posterior quadrant	598	657	59	1−116	0.045
	Cemented cups, n = 12
	All quadrants	1282	1249	−33	−67 to 1	0.054
	Superior quadrant	1334	1304	−30	−66 to 6	0.11
	Inferior quadrant	1382	1300	−82	−160 to −5	0.04
	Anterior quadrant	1476	1442	−34	−88 to 21	0.22
	Posterior quadrant	962	921	−41	−106 to 24	0.21
Dual-energy CT	Cementless cups, n = 10
	All quadrants	295	297	2	−6 to 10	0.60
	Superior quadrant	328	328	0	−7 to 8	0.93
	Inferior quadrant	276	285	9	−11 to 30	0.38
	Anterior quadrant	336	328	−8	−19 to 2	0.13
	Posterior quadrant	150	170	20	−3 to 44	0.08
	Cemented cups, n = 12
	All quadrants	305	294	−11	−21 to −2	0.014
	Superior quadrant	309	310	1	−16 to 18	0.89
	Inferior quadrant	312	281	−31	−60 to −2	0.037
	Anterior quadrant	360	349	−11	−31 to 9	0.28
	Posterior quadrant	251	232	−19	−42 to 4	0.10

Fig. 3.

Bland–Altman-plots of repeated acetabular density measurements in the cementless cup derived from SECT (left) and DECT (right). Horizontal lines indicate limits of agreement and the mean difference between the measurements. n = 10.

Fig. 4.

Bland–Altman-plots of repeated acetabular density measurements in the cemented cup derived from SECT (left) and DECT (right). Horizontal lines indicate limits of agreement and the mean difference between the measurements. n = 12.

Table 3.

Repeatability coefficients and corresponding 95% CIs in units of mg K2PO4/cm3 for repeated BMD measurements performed with SECT and DECT. The absolute difference between the measurements would be expected to lie below RC with 95% certainty.

Volume of interest	RCSECT	RCDECT
Cementless cups, n = 10	101 (64–160)	25 (16–40)
Cemented cups, n = 12	117 (77–178)	32 (21–48)

Table 4.

Inter-observer analysis. Differences between average BMD measurements in cementless cups for observers 1 and 2 in SECT and DECT (units of mg K2HPO4/cm3). n = 10.

Volume of interest	Mean BMD observer 1	Mean BMD observer 2	Difference	95% CI	P value
Single-energy CT
All quadrants	1188	1146	−42	−70 to −13	0.005
Superior quadrant	1351	1339	−12	−56 to 32	0.59
Inferior quadrant	1187	1047	−140	−193 to −86	<0.0001
Anterior quadrant	1270	1235	−35	−62 to −9	0.009
Posterior quadrant	598	620	22	−66 to 110	0.63
Dual-energy CT
All quadrants	295	285	−10	−14 to −5	<0.0001
Superior quadrant	328	324	−4	−10 to 1	0.14
Inferior quadrant	276	244	−32	−41 to −22	<0.0001
Anterior quadrant	336	331	−5	−11 to 1	0.09
Posterior quadrant	150	141	−9	−22 to 4	0.19

Fig. 5.

Box plot with medians, quartiles, and ranges comparing BMD in units of mg K2PO4/cm3 for two observers (BM and KK) in the cementless concept. n = 10. Dots are outliers.

Fig. 6.

Bland–Altman plots of density measurements from observers 1 and 2 derived from SECT (left) and DECT (right). Horizontal lines indicate limits of agreement and the mean difference between the observers. n = 10.

Discussion

In this experimental ex-vivo study with porcine specimens, we found that BMD estimates derived from SECT were approximately four times higher than those of DECT using segmentation software. The difference is supposedly caused by differences in the estimation of BMD, i.e. the pixel values in the low-density rods of the calibration phantom were comparable between SECT and DECT, while the pixel values at higher densities differed. Because the BMD estimation is basically a regression maneuver, the difference will cause SECT to result in higher BMD than DECT (34).

In the cemented concept, we found a statistically significant difference between the repeated measurements in DECT and a borderline significant difference in SECT. The difference is probably caused by the relatively low number of specimens. In all cases, the reliability expressed by ICC was good and no statistically significant difference was found between SECT and DECT.

In both scan modes, the RCs were larger than the minimal clinically relevant difference of 5%, i.e. 8% in the cementless concept and 11% in the cemented concept.

Measurements in the subdivided acetabular ROIs returned small differences between the repeated measurements and followed the same tendency between the scan modes. However, the results of this analysis must be considered explorative, because such sub-group analyses would supposedly require more specimens as reflected in the variability of the P values.

The study has some limitations. The statistically significant difference between the two observers may be caused by the difference in experience between the observers potentially leading to a consensus issue. The choice of observers was made to create a worst-case scenario in order not to overestimate the reliability and agreement of the method. Even in this worst-case scenario the absolute differences were small and most likely not clinically relevant, but experienced observers should perform all analyses in clinical studies. Further steps should be taken to ensure consensus if the method is to be used with multiple observers.

Because the study solely focused on agreement between repeated measurements and not accuracy of the BMD estimates, the external validity of the study is limited. The implants were inserted in dead porcine specimens. Thus, no biochemical reactions from the implants could occur and the cement did not infiltrate the bone tissue as much as it would do in living subjects. Furthermore, the animals were young compared with hip patients who are most often elderly with degenerative changes and lower BMD. The anatomy of porcine hips differs somewhat from human anatomy and even though the specimens were kept at low temperature there were small air deposits in the cancellous bone. We also considered positioning the specimens in a water tank to mimic the absorption of human tissue, but it was not possible to obtain stable positioning without movement during the scan procedure. Thus, more image noise would be expected in a patient compared to the experimental setup. However, the image quality of both scan modes would supposedly decrease similarly from this. Finally, the cemented cups could not contain stem heads larger than 28 mm compared with the commonly used 32-mm heads. This may in principle have caused fewer artifacts in the cemented concept.

In conclusion, the present study suggests that the intra-observer agreement of acetabular BMD measurements performed with DECT is better than that of SECT, while the intra-observer reliability is equally high for both scan modes and cup types. The inter-observer agreement was better in DECT while the inter-observer reliability was equally high in SECT and DECT. However, due to the difference in experience between the observers the results on inter-observer reliability and agreement must be interpreted with caution. Thus, DECT may be beneficial as a research tool for longitudinal studies of bone loss around acetabular cups. Further experimental in vivo studies are needed before the software can be used in longitudinal clinical studies of bone loss.