Intra and interobserver concordance between the different classifications used in Legg-Calvé-Perthes disease.

OBJECTIVE: The aim of this study was to determine the intra and interobserver concordance rates of the Waldenström, Catterall and Herring classifications for Legg-Calvé-Perthes disease.
METHODS: One hundred radiographs of the pelvis in anteroposterior and Lauenstein views, from patients with this disease, were selected. The radiographs were classified by four physicians with different levels of experience who had previously been given guidance regarding the classifications used, in order to minimize any bias of interpretation. The radiographs were examined by the same observers at two different times in order to evaluate the intra and interobserver concordance. Reproducibility was assessed using the kappa index.
RESULTS: The concordance analysis was stratified into levels (poor, slight, fair, moderate, good and excellent). The intraobserver analysis showed, for the Waldenström classification, moderate concordance for three examiners and fair for one; for Herring, excellent for one examiner and good for three; and for Catterall, good for all the examiners. The interobserver analysis showed: for the three classification systems, no situations of excellent concordance; for Waldenström, four situations of fair concordance, one moderate and one slight; for Herring, four situations of moderate concordance, one good and one fair; and for Catterall, four situations of moderate concordance and two fair.
CONCLUSION: The classifications studied are the ones most used for guiding the treatment for Legg-Calvé-Perthes disease, but the degree of intra and interobserver concordance is far from ideal. Complementary staging systems need to be taken into consideration, so that there can be greater certainty regarding the treatment.

Introduction

In 1910, Legg–Calvé–Perthes disease (LCPD) was described for the first time. Since then, it has aroused great interest among researchers and has come to be one of the most controversial topics in the orthopedic literature. Several aspects of this clinical entity still remain unexplained, such as its etiology and the best way of treating it in the active phase of the disease.

For a long time, almost all authors concentrated on analyzing the radiographic aspects of LCPD. The evolutionary phases were first described by Waldenström, whose classification was subsequently simplified and correlated with the anatomopathological findings by Jonsäter. Evaluations on the compromising of the nucleus of ossification of the femoral head came to be systematized by Catterall, based on analysis on simple radiographs produced during the phase of maximum fragmentation. With the aim of determining the proportions of lesions during the initial phase or the necrosis phase, Salter and Thompson demonstrated that the size of the subchondral fracture in the Lauenstein view precisely reflected the degree to which the proximal femoral epiphysis was affected by the disease. More recently, Herring et al. proposed a new classification based on the height of the lateral column of the femoral epiphysis. Other classifications have been proposed, but the ones cited above are those that are most used today.

From radiographic analyses on the hips of affected patients, all of these authors developed classifications for use in cases of LCPD and thus sought to systematize the treatment. However, for a given classification to be considered adequate, it needs to be reproducible, i.e. there needs to be inter and intraobserver concordance, and furthermore, the classification system needs to aid in guiding disease outcomes.

Although the treatment of LCPD has been the subject of exhaustive discussions among orthopedists, there is still no clear evidence regarding the best therapeutic method for these patients, and this is not within the scope of the present study.

The objective of this study was to evaluate the intra and interobserver concordance of the classifications of Waldenström, Catterall and Herring et al., attempt to establish which of them has the greatest degree of reproducibility and thus facilitate therapeutic decision-making.

Materials and methods

This research project was submitted to the research ethics committee of the Brazil Platform and was approved for implementation under the numbers CAAE 33513214.7.0000.5505 and CEP 418466.

The patients were evaluated through analysis on simple radiographs of the pelvis in the anteroposterior and Lauenstein views. A convenience sample of 100 radiographs of patients with LCPD was collected. These examinations were selected from a database within the Discipline of Pediatric Orthopedics, relating to patients attended at the orthopedics and traumatology outpatient clinic of Hospital São Paulo. The radiographs were selected by two orthopedists who did not participate in the disease classification process, such that good-quality examinations were included and a broad spectrum of lesions was taken into consideration. With the aim of minimizing bias due to difficulties in interpreting the examinations, the observers were provided with an initial explanation of the classification systems used in this study. Furthermore, the protocol used for the data-gathering contained a diagram containing images from the classification systems of Waldenström (as modified by Jonsäter), Catterall and Herring et al.

In order to determine the interobserver concordance, each of the four researchers evaluated the radiographic examinations independently. The different examiners were not allowed to have prior knowledge regarding the patients’ histories or any clinical information about how the disease was addressed or treated. The examiners were allowed to take all the time that they needed to evaluate all of the radiographs. After making the classifications, the evaluators were asked to classify all the examinations again, 30 days after the first analysis, without having access to the first round of evaluations.

The participants were instructed not to discuss the classification systems between each other until after all the material to be analyzed in this study had been gathered.

The statistical analysis on the results obtained was performed by a professional within the field of medical statistics. The data collected were analyzed with regard to inter and intraobserver concordance, by means of the kappa index. The tests were interpreted as described by Altman, as “proportional agreement with correction for chance”. Kappa is a coefficient of concordance that has values ranging from +1 (perfect concordance), passing through 0 (concordance equal to chance) and going to −1 (complete discordance). There are no definitions regarding which concordance levels should be accepted, but in the study by Svanholm et al., it was indicated that concordance greater than 0.75 is considered excellent, 0.5–0.75 good and less than 0.5 is poor. However, we used the intervals for the kappa index that are shown in Table 1.

Table 1

Correlation between the kappa value and the degree of concordance.

Kappa value	Concordance
0	Poor
0–0.20	Low
0.21–0.40	Fair
0.41–0.60	Moderate
0.61–0.80	Good
0.81–1	Excellent

P, poor; L, low; F, fair; M, moderate; G, good; E, excellent.

Results

Table 2 shows the absolute frequencies of the classifications made by the examiners at the two different evaluation times on the radiographs studied.

Table 2

Weighted kappa and 95% CI values for the intraobserver assessment.

Observer	Waldenström	Herring	Catterall
A	0.44 (M)	0.82 (E)	0.79 (S)
	(0.31; 0.58)	(0.74; 0.90)	(0.70; 0.88)
B	0.32 (C)	0.73 (S)	0.69 (S)
	(0.18; 0.45)	(0.63; 0.83)	(0.59; 0.78)
C	0.53 (M)	0.77 (S)	0.72 (S)
	(0.38; 0.69)	(0.67; 0.87)	(0.62; 0.81)
D	0.52 (M)	0.71 (S)	0.65 (S)
	(0.38; 0.65)	(0.62; 0.80)	(0.55; 0.75)

Table 3 shows the distribution of the weighted kappa values and the 95% confidence intervals of the intraobserver concordance analysis. In this analysis, moderate agreement for three examiners and fair agreement for one examiner were obtained through application of the Waldenström classification. From the Herring classification, there was an excellent result for one examiner and good agreement for three examiners. In relation to the Catterall classification, all the results presented good agreement.

Table 3

Weighted kappa and 95% CI values for the interobserver assessment.

Classification
Obs1	Obs2	Waldenström	Herring	Catterall
A	B	0.30 (C)	0.63 (S)	0.41 (M)
		(0.15; 0.45)	(0.52; 0.74)	(0.31; 0.52)
	C	0.35 (C)	0.49 (M)	0.30 (C)
		(0.20; 0.50)	(0.35; 0.63)	(0.18; 0.42)
	D	0.38 (C)	0.53 (M)	0.32 (C)
		(0.25; 0.52)	(0.40; 0.65)	(0.21; 0.44)
B	C	0.29 (C)	0.41 (M)	0.46 (M)
		(0.14; 0.45)	(0.28; 0.54)	(0.34; 0.57)
	D	0.47 (M)	0.54 (M)	0.47 (M)
		(0.33; 0.61)	(0.41; 0.66)	(0.36; 0.58)
C	D	0.23 (Pq)	0.39 (C)	0.44 (M)
		(0.08; 0.39)	(0.27; 0.51)	(0.32; 0.56)

Table 4 shows the results relating to agreement between the observers according to kappa values with 95% confidence intervals. According to the statistical analysis, no cases of excellent interobserver concordance were found. There were four results in which the concordance was fair, one moderate and one low, in the Waldenström classification. In relation to the Herring classification, four cases of moderate, one good and one fair agreement were obtained. In relation to the Catterall classification, there were four indexes with moderate agreement and two with fair agreement.

Table 4

Absolute frequencies of the observers’ classifications in the two evaluations.

Observers
Classification	A		B		C		D
	Eval1	Eval2	Eval1	Eval2	Eval1	Eval2	Eval1	Eval2
Waldenström
1	24	14	14	9	14	10	13	8
2	44	29	35	27	38	32	33	33
3	16	38	23	30	30	40	22	31
4	17	20	29	35	19	19	33	29
Herring
1	29	29	23	25	26	24	28	23
2	48	45	51	46	55	56	34	39
3	9	12	8	13	14	13	23	21
4	15	15	19	17	6	8	16	18
Catterall
1	48	42	31	27	24	20	41	35
2	26	31	30	38	31	22	21	21
3	17	18	29	20	27	35	25	28
4	10	10	11	16	19	24	14	17

Discussion

The challenge for orthopedists in relation to LCPD lies in treating this condition. There has been much discussion about whether there is or is not any definitive possibility of altering what Catterall called the natural history of the disease.

There has also been much discussion regarding the treatment that should be applied. Because of the lack of convincing evidence regarding the effectiveness of therapies, these concepts have been applied over the course of the years, based on each author's experience of diagnosing, classifying and managing LCPD.

We are convinced that, to deal with LCPD correctly, the diagnosis needs to be systematized and the approaches used need to be based especially on classifications that direct us toward appropriate treatment. So far, this has been done on the basis of the classifications assessed in this study.

Therefore, we believe that the first step to be taken, after the diagnosis of LCPD has been established, is to attempt to properly stage it, using classical methodologies based on radiographic analyses and also on magnetic resonance imaging, arthrography and scintigraphy when necessary.

With the aim of dealing with LCPD from a therapeutic point of view, we found that several authors in the literature proposed classifications that would allow this disease to be systematized so as to be able to predict which approach would be the best one to use, with the expectation of thus obtaining better results.

In our study, the Waldenström classification presented concordance that we considered to be inadequate. Since the phases of the disease overlap, lack of knowledge of the length of time over which the disease has evolved makes it more difficult to define the current stage of the disease.

Initially, the classification system that most polarized the attention of other authors in the orthopedic literature was the one proposed by Catterall in 1971. This author radiographically assessed the behavior of the nucleus of ossification of the femoral head during the progression of the disease, at the phase of maximum fragmentation.

This classification was contested by several researchers who used it. While some authors thought that it was of fundamental importance for indicating the therapy that should be followed and that it had a positive correlation with the final results,8, 9, 10 others criticized it because it is applied at an advanced stage of the disease and has questionable concordance when used by different observers. All of these criticisms were cited by Terjesen et al. Despite the reported discordance between observers, many authors have used this system to guide the therapy that is to be instituted. However, given that we did not find any excellent or good agreement, it is possible that the therapeutic indications may become distorted according to the gradation imposed by this system.

Because the classification proposed by Salter and Thompson is used at an initial stage of the disease, it may perhaps allow early treatment. However, it presents the limitation that only 25% of the patients with LCPD have subchondral fractures that are recognizable on radiographs.5, 12

In comparison with the classification of Catterall, that of Herring et al. is easier to interpret. However, since this system too can only be used at the final fragmentation phase, we take the view that it would not be ideal for indicating early treatment. Since the time of the initial description of this classification, which was presented in 1994, and because of the difficulty of defining the patients belonging to group B, another three subgroups were created, which thus allowed the classification to have wider coverage.

In our study, we obtained four moderate, one good and one fair agreement from testing the system of Herring et al. This suggests that there is some difficulty in precisely defining each of the groups and subgroups.

Another important matter is that simple radiography does not precisely mirror what happens to the femoral epiphysis. Cartilaginous tissue, which also shows alterations caused by this disease, is present in greater amounts than bone tissue.13, 14, 15 In this light, in some cases, particularly in the early stages of the disease, studying this cartilage by means of magnetic resonance imaging2, 16, 17 and pneumoarthrography of the joint13, 14 may provide greater clarification and more effective guidance for the treatment that is to be instituted.4, 14, 15, 18, 19, 20

Therefore, we consider that knowledge of the behavior of the cartilaginous structures of the hip is fundamental for staging LCPD and indicating the appropriate treatment.4, 14, 15, 18, 19, 20

We believe that magnetic resonance imaging currently presents a variety of advantages over other examinations. This is advocated in several studies in which some authors developed their own classifications with the aim of indicating the therapy with a higher proportion of correct choices.

Although we found a certain degree of intraobserver concordance in the three classification systems studied, and interobserver concordance in two of the three systems, our data are not in agreement with what we have observed in the worldwide literature.

Most of the studies observed did not demonstrate this degree of concordance. One likely cause for this divergence may relate to the low average number of patients studied in the literature (40),21, 22, 23 in comparison with the number of patients in the present study (100).

Unfortunately, although most experienced surgeons use the classifications studied here to determine the approaches that they will use, the concordance observed after statistical analysis was not shown to be sufficient for this. Therefore, this should not be the sole factor taken into account in making therapeutic decisions relating to patients with LCPD.

Conclusion

The intraobserver concordance analysis with 95% confidence intervals showed the following through the kappa index: moderate concordance for three observers and fair for one, in the Waldenström classification; excellent concordance for one examiner and good for three, in the Herring classification; and good concordance for all the examiners, in the Catterall classification.

The interobserver concordance analysis with 95% confidence intervals showed the following through the kappa index after statistical analysis: no excellent concordance for any of the three classification systems; four fair, one moderate and one low agreement for the Waldenström classification; four moderate, one good and one fair agreement for the Herring classification; and four moderate and two fair agreements for the Catterall classification.

Although these classifications are the systems most used by orthopedists to treatments for Legg–Calvé–Perthes disease, and although the intra and interobserver concordance indices found in the present study were better than those seen in the worldwide literature, the indices found here are still far from ideal. Therefore, complementary systems for staging the disease should be taken into consideration, in order to have greater precision in treating this disease.

Conflicts of interest

The authors declare no conflicts of interest.