Classifying failed hip arthroplasty: generalizability of reliability and validity.

Interrater reliability and validity of a radiographic severity classification was evaluated in 81 patients having revision hip arthroplasty. Severity was rated separately on the femoral and acetabular sides using a five-level scale ranging from no significant loss of bone stock to uncontained loss of bone stock and discontinuity. Three academic orthopaedic surgeons rated preoperative anteroposterior radiographs taken within 6 weeks of surgery. Interrater reliability was 0.54 (weighted kappa) with 57% agreement on the acetabular side and 0.56 with 52% agreement on the femoral side. Rater to intraoperative findings agreed 45% of the time and weighted kappa was 0.41 on the acetabular side and agreed 38% of the time with weighted kappa of 0.39 on the femoral side. When radiographic and intraoperative ratings disagreed, 30% of the time no bony defect was found on the acetabular side. Fifty-eight percent of femoral radiographic ratings were upgraded intraoperatively. These results differ from previously reported results of high reliability from one institution with trained raters. A reliable and valid severity classification that is generalizable to multiple raters from different institutions is required to stratify patients for intervention studies, and to aid preoperative planning. Training in the classification system may improve generalizability.