Shortest time interval for detecting the progression of knee osteoarthritis on consecutive X-rays.

OBJECTIVES: This study aims to find the shortest needed time interval between two consecutive anteroposterior (AP) knee X-rays of the same patient to determine the progression of knee osteoarthritis (KOA) by a trained eye. PATIENTS AND METHODS: In this retrospective study, 2,145 AP knee X-rays of 848 primary KOA patients (331 males, 517 females; mean age 65±9 years; range, 50 to 92 years) followed-up between January 2014 and December 2017 were used. Randomly generated 1,280 pairs of knee X-rays were shown to 14 orthopedic surgeons working in the Department of Orthopedics and Traumatology, and then the physicians were asked to select the second X-ray of the same arthritis knee. The physicians completed the test twice. The patient's age, gender, time interval between two radiographs and the responses of the physicians were recorded.
RESULTS: Our results showed that if the time interval between the two radiographs was six months or more, the correct estimation rates increased gradually. When the time interval was 36 months and more, the ratio reached 92%. The sensitivity and specificity rate of the method was 81%, while the positive predictive value was 86%. However, interestingly, age or gender did not have any effect on this result.
CONCLUSION: In our study, X-rays taken in less than six months apart could not give additional information about the radiographic progression of KOA. To discern between the progression of KOA, we recommend that there be a 12 to 18-month interval between consecutive X-rays. The data of our study can be used for a routine algorithm to be developed for the evaluation of KOA patients.

Introduction

Knee osteoarthritis (KOA) is one of the leading causes of disabling pain in elderly patients. It is known as a progressive condition and usually patients are assessed many times before some eventually end up with surgery.[1,2] Progression of the disease is believed to be faster in patients with higher body weight or concomitant injury.[3-8] Clinical progression is not necessarily correlated with radiographic progression. However, it is unlikely to have improvement radiologically.[1,9] Physicians usually assess this progression by looking at the X-rays of the patients. Anteroposterior (AP) knee X-rays are known to be reliable tools for evaluating KOA and widely used for this purpose. Kellgren-Lawrence (KL) classification, which is for grading of KOA, depends on AP knee X-rays.

How long should the time interval be between the two consecutive X-rays of the same patient for a physician to be able to spot the progression of KOA? Do we cause the patient to undergo unnecessary exposure to radiation during KOA follow-up? These questions are still unanswered. Therefore, in this study, we aimed to find the shortest needed time interval between two consecutive AP knee X-rays of the same patient to determine the progression of KOA by a trained eye.

Patients and Methods

This retrospective study was conducted at Department of Orthopedics and Traumatology, Erzincan Binali Yıldırım University Mengücek Gazi Training and Research Hospital. The study included X-rays of KOA patients followed-up between January 2014 and December 2017. Patients diagnosed with primary KOA (International Classification of Diseases 10th revision [ICD10]: M17.1 or M17.0) were found from the medical database. Exclusion criteria were as follows: lower extremity surgery, fracture in the lower extremity, inflammatory arthritis (ICD10: M05-M14) or infectious arthritis (ICD10: M00-M03). Patients with more than one weight-bearing AP knee X-ray of the same knee were selected. A visual inspection was performed to eliminate the technically inadequate X-rays and finally we found 2,145 X-rays of 848 patients (331 males, 517 females; mean age 65±9 years; range, 50 to 92 years) suitable for the study. The study protocol was approved by the Erzincan Binali Yıldırım University Mengücek Gazi Training and Research Hospital Ethics Committee. A written informed consent was obtained from each patient. The study was conducted in accordance with the principles of the Declaration of Helsinki.

Of the 848 KOA patients, all had two or more AP knee X-rays, and none had any related secondary medical diagnosis. We set up an experiment using the obtained data and a custom-made computer software.[10] The software looks for possible pairs of knee X-rays of the same KOA patient and then the selected pair is presented side-by-side in a graphical user interface to the user for visual comparison (Figure 1). The user sees two AP knee X-rays of the same patient side-by-side in a random order. The user is asked to choose the latter one and click on that button. Every time the user makes a choice and clicks the button below the X-ray, the computer shows another pair from the repository.

In case where there are only two X-rays for a patient, as one might expect, a single comparison is made between them. Whereas, for the case of three or more X-rays, all possible randomized binary combinations are shown to the examiner to make a guess on which one of the X-rays was taken later. After every choice, the next pair is shown to the user until all comparisons are made.

Fourteen orthopedic surgeons were asked to use the software. All of them performed all the comparisons twice at different times to test the reproducibility. Therefore, we conducted 28 comparison tests.

Software users were blinded to the chronological sequence of the X-rays, the gender, and the age of the patient. They knew they were looking at two X-rays of the same knee of a KOA patient taken in different times. They were asked to make their best guess on the latter X-ray of the same knee. The computer kept a record of all key information relevant to the case such as the time interval between the compared X-ray pairs, age, and the gender of the patient, and the examiners’ guess if it is right or wrong (Figure 2).

We assumed that progression would be discernible on X-rays if sufficient time passed. If physicians discern the progression on side-by-side X-rays, then they can successfully guess the chronological sequence of the X-rays shown by the computer and make the right guess. If the time interval is not long enough for a visible progression, then physicians could not detect the progression and make a right guess consciously. When the physicians could not tell the chronological sequence of the X-rays by looking at them, they were asked to make a random guess and skip to the next question.

We studied the data to find out the relations between success rates, time interval, sex and age. We also studied test-retest reliability and inter-observer agreement.

Statistical analysis

All data were analyzed using the IBM SPSS Statistics version 22.0 software (IBM Corp., Armonk, NY, USA). Fleiss’ kappa test was used to test the level of agreement between all physicians. Cohen’s kappa test was used to assess test-retest reliability of the same physician. Binary logistic regression was used to determine if there was any relationship between the time interval and the successful guess. We used Cochran’s Q test for binary data to test the differences of the guesses between the physicians.

Results

We found 848 patients with 2,145 X-rays selected for the study. Software algorithm created 1,280 pairs for comparison using those X-rays. Of the 1,280 X-ray pairs, 352 had an interval of six or less months, 216 had six to 12 months, 188 had 12 to 18 months, 175 had 18 to 24 months, 166 had 24 to 36 months and 183 had more than 36 months. The average values for those groups were calculated and shown in Table 1.

Cochran’s Q test revealed no significant differences between physicians. A binary logistic regression was performed to ascertain the effects of the time interval on the successful guesses by test objects. The logistic regression model was statistically significant, χ2 (1)=444.9, p<0.0005. The model correctly classified 81% of the correct guesses. Sensitivity was 81%, specificity was 81%, positive predictive value was 86%, and negative predictive value was 74%. The effect of age or sex was not statistically significant.

Discussion

Knee osteoarthritis is a progressive disease. Thus, we can assume that the more delayed the time of the latter roentgenogram is, the more progressed the disease would be. We designed our study based on this premise. We collected weight-bearing AP knee X-rays of primary KOA patients with two or more consecutive X-rays and tested if a physician could spot the progression by looking at them. We used a custom-made software for this purpose.[10] In some cases, the difference was barely visible and it was easy to see the more progressed one. In some cases, it was hard to see any change at first sight and it needed further scrutinizing before deciding. In some cases, it was not possible to discern any difference at all. We instructed the users to make a random guess and skip to the next question if there was no visible hint of progression. The dichotomous nature of the test had some intuitive problems. Even if it was impossible to tell the correct sequence, there was a 50% chance to make the right guess. Yet it was highly reliable; any deviation from 50% would be the proof of a conscious decision. Our experiment consisted of 1,280 comparison pairs tested by 14 subjects twice which we believe was sufficient for such an experiment.

We also analyzed our data in a democratic way and calculated the collaborative guess rates using 75% as a cut-off value. That means if 75% of the individual guesses for each case were correct, it was accepted correct. In our case, if 21 or more guesses were the same out of 28 for each pair, this was accepted as a collaborative decision. By doing so, we attempted to eliminate the chance factor in binary questions like this one while also revealing inter-observer agreement from a different angle.

We found that the progression detection rates began to increase gradually with X-ray pairs taken at least six months apart and more. If two X-rays were taken in less than six months of interval, the average correct guess ratio was 51% (range, 46-57%) which allowed us to conclude that the guess was made randomly.

However, successful guess rates increased when the time interval between X-ray pairs was prolonged, reaching 92% when the interval was 36 months or more between two AP knee X-rays (Table 1). Receiver operating characteristic analysis was performed for accurate time estimation. We think that the cut-off value for the progression of KOA is approximately one year.

There was no significant difference in accurately recognizing the progression either between or within the two participating groups, i.e. senior orthopedists and residency students. A sophomore residency student and the most senior orthopedist scored similar in this test.

We believe that the relatively poor kappa values in Fleiss’ test were a consequence of the high number of examiners guessing on such a great amount of data. However, we believe that a value of 0.3 should be interpreted as a good agreement in a scenario where testers were able to make more than 90% of correct guesses.

Progression of KOA has been studied by using extensive imaging techniques. Quantitative magnetic resonance imaging (MRI) was shown to detect progression effectively as early as 12 months.[11] Magnetic resonance imaging was shown to be superior to plain radiographs at detecting the progression up to 30 months.[12] Using joint scores and strictly standardized radiographs, progression could be spotted at 18 months.[7] Our results are compatible with the literature and further illuminating by revealing the probability of radiographic progression depending on the interval between two follow-ups. Our study design allowed us to assess X-rays that are not bound to a predetermined follow-up period. We had a vast amount of comparison pairs, which had a time interval between zero days to four years. Our study has also revealed several clues about the characteristics of progression speed of the disease. Progression speed of primary KOA appears to be similar in all ages for both genders.

It is not possible to spot the progression of KOA by solely relying on inspection of X-rays obtained in a time interval shorter than six months. There is a very little chance with trivial reliability to distinguish the progression if the interval is less than 18 months. However, progression can be discerned at least 75% of the time when two consecutive X-rays have a minimum of 18 months of interval. Reliability and reproducibility in our study were high.

This study has some limitations. Firstly, we used only weight-bearing AP knee X-rays in our study, which may be criticized. However, all relevant studies used AP views commonly and this was accepted to be adequate for evaluating KOA.[13,14] Kellgren-Lawrence classification which is widely employed in radiological grading of KOA also uses only AP knee radiographs.[15]

Moreover, roentgenograms we used in our study were not strictly standardized for any specific purpose. This may be criticized as well. However, they were accepted to be adequate for KOA evaluation since they suit real-life scenarios. Having standardized views is a problematic issue. There are at least 17 factors which affect the result of a knee X-ray.[16] Moreover, KL itself is also not tuned on the position of the knee.

Another criticism may be that we did not use any grading system for progression detection. The definition of KL consists of some radiological signs of osteoarthritis, which are osteophyte formations, narrowing of joint space, sclerosis of subchondral bone, cystic areas, and altered shape of the bone ends.[15] The physician to determine the grades defined by KL interprets these attributes and concludes on a score. Those scores are highly dependent on the interpretation of the physician.[15] Studies that use KL or similar scores to determine the progression have indeed a weak point of quantizing the progression. They accept one point of increase in radiographic score as a progression sign. If scores do not change, then it is not interpreted as a progression. However, side-by-side comparisons may yield the progression clearly, even though scores are the same. Such grading systems with ordinal nature are called Likert scales. Side-by-side (pairwise) assessment of X-rays have been shown to be superior to Likert scales in detecting changes in the images,[17] which was also the foundation of our study. In our study, we did not use any grading systems but relied on the power of human eye at side-by-side comparisons, which was proven to be better for such a task.[17] Therefore, we believe pairwise comparisons were indeed a power of our study.

In conclusion, physicians use X-rays not only as a diagnostic tool but also as a source of reassurance. Medico-legal reasons and patient pressure are alibis for unduly wanted X-rays.[13,18] Our results should be kept in mind when assessing KOA patients to save resources. In the evaluation of a KOA patient, even though the disease has a progressive nature, seeing another weight-bearing AP knee X-ray before six months presumably would not give additional information about the radiographic progression. One third of knee X-rays in our study might be labeled as unnecessary in this manner. The unnecessary repetition of radiographs results in radiation exposure both for the patients and radiographers and increased costs.[19] It may be more useful to seek low-cost tools for reassurance of these patients.

Table 1

Success rates depending on time interval between consecutive X-rays

	X-ray pairs (n=1,280) n	Individual success rates (Average-range)%	Collaborative success rates (75% Cut-off)%	Level of agreement* (Fleiss' Kappa)	Test-retest reliability* (Cohen's Kappa)
<6 months	352	51 (46-57)	8	0	0.1 (0-0.3)
6-12 months	216	60 (54-63)	14	0.1	0.2 (0-0.2)
12-18 months	188	64 (59-72)	38	0.1	0.1 (0-0.2)
18-24 months	175	76 (67-80)	74	0.2	0.4 (0.2-0.5)
24-36 months	166	81 (77-86)	87	0.2	0.5 (0.2-0.7)
>36 months	183	92 (88-95)	98	0.3	0.7 (0.3-0.9)
*0: No agreement; 0.5: Moderate agreement; 1: Perfect agreement.