Radiographic grading of femoral stem cementation in hip arthroplasty.

OBJECTIVE: To determine intra and interobserver agreement of the grading system for femoral cementation in hip arthroplasty proposed by Barrack.
METHODS: Immediate anteroposterior and lateral postoperative radiographs of 55 primary total hip arthroplasties were assessed by two observers familiar with the use of this grading system. The assessments were performed on two separate occasions by each observer and independently. The statistical analysis measured the Kappa coefficient, which determines the degree of agreement between tests with categorical variables.
RESULTS: Intraobserver Kappa coefficient varied from 0.43 to 0.68, demonstrating moderate to substantial strength of agreement; interobserver Kappa coefficient varied from 0.19 to 0.44, demonstrating slight to moderate strength of agreement.
CONCLUSION: Intra and particularly interobserver agreement are limited in this grading system, even when used by trained individuals. Level of Evidence III, Study of nonconsecutive patients; without consistently applied reference "gold" standard.

INTRODUCTION

The main cause of failure of total hip arthroplasty is aseptic loosening; , in the case of cemented prostheses, this complication is directly influenced by the cementation quality of the components. , Although modern arthroplasty techniques have improved the quality of the cementation and reduced the occurrence of aseptic loosening, , it is still responsible for about 80% of the revisions performed.

The radiographic parameters used most often when assessing the quality of the cementation obtained in a hip arthroplasty include the homogeneity of the cement mantle, , its distal extension up to the end of the femoral stem and the presence of defects in the cement-bone interface. A grading system that includes all of these parameters was proposed by Barrack et al. to specifically evaluate the femoral component cementation and to identify stems with a risk of loosening. , ,

One of the essential characteristics for any grading system to be really useful is its reproducibility, which allows data obtained in studies conducted by different authors to be appropriately compared. Our goal is to determine the reproducibility of this grading system based on its intra and interobserver agreement.

MATERIAL AND METHODS

We analyzed the immediate postoperative radiographs in anteroposterior (AP) and lateral (L) views of 42 patients (55 hips) submitted to primary total hip arthroplasty with cemented femoral component at the Hospital das Clínicas da Faculdade de Medicina de Ribeirão Preto da Universidade de São Paulo (HCFMRP-USP). These 55 cases correspond to all the patients where neither one of the two observers responsible for the radiograph analysis had taken part directly in the surgery, in the period between 2001 and 2005. CPT(r) (Zimmer, Warsaw, IN, USA) stems were used in 28 hips while Exeter Universal(r) (Stryker-Howmedica, Newbury, UK) stems were used in 27 hips, always with third-generation cementing technique and using the same type of cement (Simplex P(r), Stryker-Howmedica, Limerick, Ireland) in all cases.

The cementation quality of the femoral component was graded according to the system proposed by Barrack et al. (Table 1) by two observers (the authors FLG and ATS), trained in and familiar with the use of this grading system. Each observer performed the radiograph readings independently, on two different occasions and with a 30-day interval between one reading and the other.

Table 1

Barrack's femoral component cementation quality grading system.

Grading	Radiographic characteristics
A	Complete filling of the medullary canal, without radiolucent lines between the cement and the bone (white-out)
B	Radiolucent line covering up to 50% of the cement-bone interface
C	Radiolucent line covering between 50% and 99% of the cement-bone interface or incomplete cement mantle
D	Complete radiolucent line (100%) at the cement-bone interface and/or absence of cement distally to the end of the stem

To compare the observers' answers we calculated the Kappa coefficient with the help of version 9.0 of the SAS/STAT(r) software (SAS Institute Inc., Cary, NC, USA). This coefficient measures the degree of agreement between tests when the variables are categorical. When it assumes its maximum value (one) we have perfect agreement; a coefficient equal to zero indicates that the agreement is the same as that expected by chance and negative values occur when the agreement is weaker than expected by chance. The strength of agreement for the Kappa coefficient is divided into six categories by Landis and Koch. (Table 2)

Table 2

Categories of strength of agreement for the Kappa coefficient.

Kappa Coefficient	Strength of agreement
Below zero	Poor
0 – 0.2	Negligible
0.21 – 0.4	Modest
0.41 – 0.6	Moderate
0.61 – 0.8	Substantial
0.81 - 1	Almost perfect

RESULTS

The general distribution of the cementation quality gradings of the two observers is shown in Tables 3 (AP radiographs) and 4 (lateral radiographs). The most frequent gradings were B and C in the AP and lateral radiographs, respectively.

Table 3

Grading of the cementation quality in the anteroposterior radiographs.

	Cementation quality
	A	B	C	D
Observer 1
Reading 1	17	25	13	0
Reading 2	13	30	12	0
Observer 2
Reading 1	16	28	10	1
Reading 2	14	34	6	1
Total	60	117	41	2

Table 4

Grading of the cementation quality in the lateral radiographs.

	Cementation quality
	A	B	C	D
Observer 1
Reading 1	3	9	43	0
Reading 2	3	11	41	0
Observer 2
Reading 1	4	20	31	0
Reading 2	3	23	29	0
Total	13	63	144	0

In Table 5 we verified that the Kappa coefficient for intraobserver agreement ranged between 0.43 and 0.68, demonstrating moderate to substantial strength of agreement. For one of the observers the level of agreement was higher in the evaluation of the AP radiographs, while the other observer presented a higher level of agreement in the evaluation of the lateral radiographs. The Kappa coefficient for interobserver agreement (Table 6) ranged between 0.19 and 0.44, demonstrating slight to moderate strength of agreement. The highest level of interobserver agreement occurred in the evaluation of the AP radiographs, in both readings.

Table 5

Intraobserver agreement of the cementation quality grading.

	Kappa Coefficient (CI 95%)
Observer 1
AP	0.49 (0.29 – 0.68)
L	0.68 (0.46 – 0.90)
Observer 2
AP	0.53 (0.35 – 0.71)
L	0.43 (0.22 – 0.65)
CI: confidence interval.

CI: confidence interval.

Table 6

Interobserver agreement of the cementation quality grading.

	Kappa Coefficient (CI 95%)
Reading 1
AP	0.44 (0.26 – 0.61)
L	0.28 (0.04 – 0.52)
Reading 2
AP	0.43 (0.24 – 0.62)
L	0.19 (-0.02 – 0.42)
CI: confidence interval.

CI: confidence interval.

DISCUSSION

The importance of adequate cementation for the late result of hip arthroplasty is well documented in the literature. , , , - Besides the existence of reports of good late results using femoral components cemented with a minimum and often incomplete cement mantle (the so-called "French Paradox"), , according to concepts established by clinical - and biomechanical studies , and widely accepted by the orthopedic community, the ideal cement mantle should be homogeneous and completely cover the implant, exhibit a thickness of 2 to 4mm and not present radiolucent lines in the cement-bone interface. Considering the crucial role that the cement mantle plays in the outcome of the arthroplasty, some femoral component cementation grading systems were proposed with the objective of identifying stems with a risk of loosening. , The system proposed by Barrack et al. is one of those most frequently used owing to its simplicity; however, for a grading system to be useful and to allow the comparison of data obtained by different investigators in a reliable manner, it must also be reproducible. This study was aimed at determining the degree of intra and interobserver agreement of this femoral component cementation quality grading system.

Just as reported by some authors, , , the most frequent gradings in our study were Barrack B and C; most of our C grades occurred in the lateral radiographs, due to the presence of an incomplete cement mantle in Gruen zones 8 or 9, where the straight femoral stem tends to approach the anterior cortex on account of the normal curvature of the proximal femur.

Our results also demonstrate that the system presents problematic reproducibility, with limited intra and especially interobserver agreement. Actually this finding is consistent with previous studies conducted by other authors (Table 7), who in some cases found a level of agreement that was even lower than expected by chance. In addition to these characteristics, some authors also report that the agreement in this system is not proportional to the observers' level of experience, i.e., more experienced observers did not obtain agreement superior to that found among less experienced observers. , In our study we were unable to corroborate the latter fact, since the readings were performed by two trained observers with similar levels of experience in use of the system. As a means of reducing the high margin of error inherent to this system and of enabling a more reliable interpretation of the data, some authors suggest the grading of the quality of cementation obtained with the Barrack system in only two categories: adequate cementation (Barrack A and B, not associated with early loosening) and inadequate cementation (Barrack C and D, associated with early loosening). However, even with this device, the use of the Barrack system could still produce errors, since some authors report that type C cementation is not associated with higher rates of loosening. All of these facts give rise to doubts about the validity of Barrack's grading system and lead us to declare that it should be used and interpreted with caution until we have a more reproducible and reliable system.

Table 7

Comparison among the data of different studies on the reproducibility of the Barrack grading system.

Author	Number of arthroplasties	Number of observers	Intra K (strength of agreement)	Inter K (strength of agreement)
McCaskie et al. 23	30	6	0.07 to 0.63 (negligible to substantial)	-0.48 to 0.18 (poor to negligible)
Kelly et al. 1	60	2	0.49 to 0.53 (moderate)	0.38 to 0.43 (negligible to moderate)
Harvey et al. 4	100	3	not tested	0.56 to 0.73 (moderate to substantial)
Ilizaliturri et al. 24	22	3	not tested	0.19 to 0.27 (negligible to modest)
This study	55	2	0.43 to 0.68 (moderate to substantial)	0.19 to 0.44 (negligible to moderate)
Intra K: intraobserver Kappa coefficient – Inter K: Interobserver Kappa coefficient.

Intra K: intraobserver Kappa coefficient - Inter K: Interobserver Kappa coefficient.

CONCLUSION

This grading system showed itself to be hardly reproducible, with limited intra and especially interobserver agreement, even when used by trained individuals.