ASSESSMENT OF THE REPRODUCIBILITY OF THE OUTERBRIDGE AND FSA CLASSIFICATIONS FOR CHONDRAL LESIONS OF THE KNEE.

OBJECTIVE: To assess the reproducibility of the Outerbridge and the French Society of Arthroscopy classifications between different observers, and to establish a comparison between them.
METHOD: Thirty videos on randomly selected knee arthroscopy procedures demonstrating chondral lesions were used. These were classified by six observers: two third-year orthopedics residents and four orthopedic surgeons, of whom two were knee surgery specialists. The intraobserver and interobserver reliability was evaluated by means of the kappa index.
RESULTS: The result from the complete evaluation on the Outerbridge classification with all the observers gave a kappa index of 0.434411. For the classification proposed by the French Society of Arthroscopy, the kappa index was 0.45166.
CONCLUSION: The Outerbridge and French Society of Arthroscopy classifications for chondral lesions are moderately reproducible between observers. Comparing the two classifications, the proposal from the French Society of Arthroscopy was shown to be more reproducible, and the authors suggest that this classification should be used preferentially in clinical practice for evaluations on chondral lesions of the knee.

INTRODUCTION

A combination of mechanical factors and loss of the structural proteins that make up the cartilage is responsible for osteoarthrosis, which is a highly prevalent pathological condition of degenerative nature that is especially seen in the knee joint.

The joint cartilage is a type of tissue with the capacity to deal with large forces applied over many cycles, but it has a low capacity for regeneration after injury.

Traditionally, osteoarthrosis of the knee is diagnosed through clinical signs resulting from the inflammatory and/or mechanical process, accompanied by specific radiological abnormalities such as diminished joint space, subchondral sclerosis and osteophytes. However, there may be very little evidence of such abnormalities, particularly during the initial stages of the degenerative process, thus leading to delays in diagnosing this condition.

Studies have revealed the presence of advanced femorotibial and/or femoropatellar arthrosis through arthroscopy of the knee, in cases with normal radiological findings.

There are some radiographic classifications for osteoarthrosis of the knee, but they have been shown to be imprecise, especially in the initial stages.

Arthroscopic evaluation of the knee presents the characteristic of direct viewing of the joint surface, thus enabling greater detailing of the chondropathy, such as size, depth, consistency and lesion location.

Some arthroscopic classification systems for chondral lesions of the knee have been described in the literature. Among these is the Outerbridge system, which was originally described through direct viewing using arthrotomy, and the classification of the French Society of Arthroscopy.

In 1961, Outerbridge classified patellar chondral lesions into four grades, such that grade I was softening; grade II was fragmentation/fissure of 1.25 cm or less; grade III was fragmentation/fissure greater than 1.25 cm; and grade IV was bone erosion.

In 1994, the French Society of Arthroscopy proposed the classification: grade I, softening; grade II, superficial fissure; grade III, deep fissure; and grade IV, bone exposure.

Reproducibility between observers is essential for any type of classification, and arthroscopy should be considered to be an important measurement method for assessing osteoarthrosis of the knee.

The purpose of this study was to assess the reproducibility of the Outerbridge and French Society of Arthroscopy classifications between observers.

MATERIALS AND METHODS

Between the months of October 2008 and May 2009, at the Orthopedics and Traumatology Service of São José University Hospital, 30 arthroscopy procedures on knees were performed in order to treat ligament or meniscal lesions, with inventories of the medial, lateral and femoropatellar compartments. All the procedures were recorded on DVD and edited to form 30 clips of five seconds each, totaling 30 images. These images were assessed and classified by six observers, of whom two were titular members of the Brazilian Society of Knee Surgery, two were titular members of the Brazilian Society of Orthopedics and Traumatology and two were third-year medical residents, using the Outerbridge and French Society of Arthroscopy classifications for chondral lesions of the knee.

These assessments were performed on a special form that was filled out by one of the researchers. In order to minimize any bias due to difficulty in interpretation or any possible forgetfulness, the classifications were described on the response sheets, together with schematic drawings of the respective classifications, which were handed over to each observer at the time of the image assessments. There was no time limit on making the image classifications.

The data obtained were analyzed by a statistician, using the chi-square test and kappa coefficient, which were first used by Fleiss and Cohen. The kappa coefficient was used to assess the concordance between the observers within the same sample unit. This ranged from -1, when all the observers disagreed regarding all the assessments to +1, when there was complete agreement.

RESULTS

The series of images were analyzed using the Outerbridge classification (four types) and the French Society of Arthroscopy classification (four types).

The result from the complete evaluation of the Outerbridge classification with all of the observers was an inter-observer kappa index (k) of 0.434411, which produced a p-value = 0. Analysis on the two residents alone gave k = 0.395973. Analysis on the orthopedists alone gave k = 0.165379. Analysis on the knee specialists alone gave k = 0.140127 (Table 1).

Table 1

Inter-observer kappa concordance index for the Outerbridge classification.

	Orthopedist 1	Orthopedist 2	Resident 1	Resident 2	Knee specialist 1	Knee specialist 2
Orthopedist 1	1	0.165379	0.363636	0.217791	0.049296	0.437500
Orthopedist 2		1	0.010989	0.041553	0.220183	0.269406
Resident 1			1	0.395973	0.127907	0.361702
Resident 2				1	0.122807	0.212963
Knee specialist 1					1	0.140127
Knee specialist 2						1
Descriptive statistics for the 15 indices found
	Minimum	Median	Mean	Maximum	Standard deviation
	0.01099	0.21296	0.20915	0.43750	0.13408

The highest value for the intra-observer concordance index was for orthopedist 1, with k = 0.509002, and the lowest was for resident 2, with k = -0.064516, as presented in Table 2. The mean intra-observer concordance was k = 0.2955 (Table 2).

Table 2

Intra-observer concordance index for the Outerbridge classification.

Intra-observer kappa concordance index (calculated by comparing the observer’s first and second evaluations on the same 30 patients)
Observer	Kappa coefficient (K)	P-value
Orthopedist 1	0.509002	2.5207e-05
Orthopedist 2	0.40458	0.000145
Resident 1	0.233227	0.026750
Resident 2	-0.064516	0.700497
Knee specialist 1	0.244604	0.044259
Knee specialist 2	0.447853	3.48437e-05

In relation to the classification proposed by the French Society of Arthroscopy, we found the following data. The inter-observer kappa was k = 0.45166, thus resulting in p = 0, among all the six observers. Among the different groups, the index was greatest for the orthopedists, with k = 0.339623, followed by the knee surgeons, with k = 0.338983. For the residents, the concordance coefficient was k = 0.22619. These results are presented in Table 3.

Table 3

Inter-observer kappa concordance index for the classification of the French Society of Arthroscopy

	Orthopedist 1	Orthopedist 2	Resident 1	Resident 2	Knee specialist 1	Knee specialist 2
Orthopedist 1	1	0.339622	0.303483	0.062500	0.148936	0.360655
Orthopedist 2		1	0.145729	0.090909	0.117647	0.296875
Resident 1			1	0.226190	0.038462	0.150000
Resident 2				1	0.062500	0.018182
Knee specialist 1					1	0.338983
Knee specialist 2						1
Descriptive statistics for the 15 indices found
	Minimum	Median	Mean	Maximum	Standard deviation
	0.01818	0.14894	0.18004	0.36066	0.12045

The highest value for the intra-observer concordance index was for orthopedist 1 (k = 0.540034), and the lowest value was for resident 2 (k = 0.033742), as shown in Table 4. The mean kappa index for the intra-observer evaluation was k = 0.3165.

Table 4

Intra-observer concordance index for the classification of the French Society of Arthroscopy.

intra-observer kappa concordance index (calculated by comparing the observer’s first and second evaluations on the same 30 patients)
Observer	Kappa coefficient (K)	P-value
Orthopedist 1	0.540034	2.55338e-05
Orthopedist 2	0.39441	0.000306
Resident 1	0.170732	0.084768
Resident 2	0.033742	0.382217
Knee specialist 1	0.440299	0.001714
Knee specialist 2	0.322581	0.00428

DISCUSSION

Several authors have expressed the opinion that in assessing the reliability of inter-observer concordance, there is a need to incorporate concordance due to chance in the evaluation9, 10. Kappa is a coefficient of concordance that corrects for the error due to chance and is used to determine the intra and interobserver variation. It is used when two observers separately classify a sample of objects using the same category scale.

Landis classified concordance as: poor (less than 0), slight (0-0.2), weak (0.21-0.4), moderate (0.41-0.6), substantial (0.61-0.8), almost perfect (0.81-1.0).

We found some articles that assessed the reproducibility of the Outerbridge and French Society of Arthroscopy classification. Cameron et al concluded that the Outerbridge classification is moderately accurate when used to grade chondral lesions arthroscopically.

In the present study on the Outerbridge and French Society of Arthroscopy classifications, we found kappa values of 0.434411 and 0.45166 respectively. According to Landis, this indicates that the classifications have moderate concordance.

In comparing the means within the groups, we could see that in the groups using the French Society of Arthroscopy classification, the means were higher than in the other groups.

CONCLUSION

On the basis of the present assessment, it can be concluded that the French Society of Arthroscopy and Outerbridge classifications have moderate concordance between observers.

Comparing the two classifications, the proposal from the French Society of Arthroscopy was shown to be more reproducible, and the present authors suggest that this classification should be used preferentially for assessing chondral lesions of the knee in clinical practice.