Knee Cartilage Defect Patients Enrolled in Randomized Controlled Trials Are Not Representative of Patients in Orthopedic Practice.

OBJECTIVE: Knee cartilage defects represent a socioeconomic burden and may cause lifelong disability. Studies have shown that cartilage defects are detected in approximately 60% of knee arthroscopies. In clinical trials, the majority of these patients are excluded. This study investigates whether patients included in randomized controlled trials (RCTs) represent a selected group compared to general cartilage patients.
DESIGN: Published randomized clinical trials on cartilage repair studies were identified (May 2009) and analyzed to define common inclusion criteria that in turn were applied to all patients submitted to our cartilage repair center during 2008. Patient-administered Lysholm knee score was used to evaluate functional level at referral. In addition, previous surgery and size and localization of cartilage defects were recorded.
RESULTS: Common inclusion criteria in the referred patients and patients included in the published RCTs were single femoral condyle lesion, age range 18 to 40 years, and size of lesion range 3.2 to 4.0 cm(2). Six of 137 referred patients matched all the 7 RCTs. Previous cartilage repair and multiple lesions were associated with decreased Lysholm score (P < 0.002). Lysholm score was independent of age, gender, and time of symptoms from the defect.
CONCLUSION: The heterogeneity of the referred cartilage patients and the variation in inclusion criteria in the RCTs may question whether RCTs actually represent the general cartilage patients. The present study suggests that results from published RCTs may not be representative of the gross cartilage population.

Introduction

Patients with articular cartilage injuries experience decreased mobility and pain, although their symptoms differ based on affected joint. These injuries affect a large number of patients. Studies have shown cartilage injuries in 66% of the patients undergoing an arthroscopy for knee pain.[1,2] In evidence-based medicine, randomized controlled trials (RCTs) are perceived as the gold standard for evaluating treatment options. Still, only 3% to 6% of published articles in orthopedics are RCTs.[3] Several studies with the aim to measure the outcome of cartilage repair have been performed during the past decade. Numerous articles have described good or excellent results, but the methodological quality has been questioned, as evident in an analysis of cartilage repair studies from 2005.[4]

An issue that has been less discussed in the orthopedic literature is the heterogeneity in etiology and the anatomical locations of cartilage lesions. Patients with lesions in only one anatomical location resulting from one specific injury may not represent general cartilage patients. The size of the defects and the age of the patients may also result in exclusion of patients in controlled studies. These limitations, which are necessary to achieve a high internal validity due to the study design of RCTs, may naturally interfere with the external validity and clinical applicability of them.

The present study was designed to evaluate the difference between patients included in published RCTs and the total number of patients referred to a major cartilage clinic. The study’s main questions were the following: how well can the RCT inclusion criteria be applied to our general cartilage group, and are results from RCTs applicable when advising a general cartilage injured population? This offers an additional and important clinical perspective on the ability of extrapolation of RCT results on cartilage repair surgery.

Methods and Materials

Inclusion Criteria in Published RCTs

To find inclusion criteria for patients enrolled in RCTs on standard cartilage surgery, we searched PubMed and Embase, using words such as cartilage, surgery, repair, outcome, and randomized. Procedures included were microfracture (MF), mosaicplasty (MP), autologus chondrocyte implantation (ACI), characterized chondrocyte implantation (CCI), and periosteal grafting (APT). Outcome measures were the Lysholm score, Knee Injury and Osteoarthritis Outcome Score (KOOS), and Cincinnati scores. The search was performed May and September 2009; only studies published in the English language were included.

All of the RCTs were evaluated according to the PRISMA statement,[5] but not all of the criteria were applicable for the current study.

Patient Material

All patients referred to our clinic with knee symptoms suspected to be caused by focal cartilage defects were eligible for enrollment. The patients were enrolled from either a primary health service or secondary health service (orthopedic departments in other hospitals).

Patients were evaluated by an experienced cartilage orthopedic surgeon and with a patient-administered Lysholm knee score form. In the few cases of incomplete information, the primary author contacted the patients by telephone or letter and asked them to complete the form.

Our cartilage clinic has standardized the use of the Lysholm score in assessing cartilage knee problems in this patient group during their clinic visits. The Lysholm score was selected because it has been commonly used to assess knee problems, it is validated,[6] it can be filled out by the patients themselves,[7] and it quickly provides a good overview of knee symptoms presented in the outpatient clinic. Additional recent work from our clinic has demonstrated that the Lysholm score, International Knee Documentation Committee (IKDC), and KOOS maintain a close correlation in evaluating knees with cartilage defects.[8]

All patients referred to the orthopedic clinic with symptoms from their knees suspected to be caused by focal cartilage defects were examined with magnetic resonance imaging (MRI) and/or knee arthroscopy. In most cases, both arthroscopy and MRI were performed.

Demographic data, such as anatomical location and size of patients’ lesions, are reported. Arthroscopy was the gold standard in reporting size of the lesions, but in the cases where arthroscopy had not been performed, we used MRI scans. These were evaluated by an experienced radiologist not participating in this study.

Statistics

Dichotomous data are presented as numbers and percentages and continuous data as means with standard deviations (SD). The study’s main questions were the following: how well can the RCT inclusion criteria be applied to our general cartilage group, and are results from RCTs applicable when advising a general cartilage injured population? This was evaluated by simply matching the referred patients with the common inclusion criteria and for the inclusion criteria from each of the 8 RCTs.

To evaluate whether patients who had undergone previous cartilage surgery differed from those without previous surgery, we performed a t test to see if there was a statistically significant difference between these two groups.

The relationship between the Lysholm score and the total number of overall knee surgeries was also calculated. In total, comparisons between 4 parameters (no surgery, 1 surgery, 2 surgeries, and 3 or more surgeries) were performed, and Bonferroni correction with a P value of 0.01 was applied.

A multiple regression analysis was performed to evaluate the correlation between the Lysholm score and factors such as age, gender, time of symptoms, and size of lesions. The correlation between Lysholm score and localization of defects was explored with a one-way analysis of variance (ANOVA) because localization of defects cannot be analyzed with a multiple regression analysis due to its nonscale nature.

Ethics

The study was approved by the regional ethical committee.

Results

Inclusion of RCTs

We found 10 RCTs based on 8 different patient materials.[9-18] The inclusion criteria in these articles are summarized in . Number of patients and treatment allocation in the RCTs are presented in .

Patient Characteristics

During 2008, our clinic received 147 referred patients, whereas 10 were excluded from this study; this number of referral of patients is in line with our previously reported numbers regarding the incidence of these lesions in our patient population.[1] This present study included more patients than each of the 8 RCTs,[9-18] whereas Saris et al.[14] included the most, n = 118, and Horas et al.[18] included 40 patients. We therefore believe that we have included enough cartilage patients to answer our study hypothesis.

We also performed a power analysis on behalf of the statistical analysis. We wanted to simply match the characteristics of included patients with the same characteristics from the 8 RCTs. This resulted in a minimum of 101 included patients in this study. illustrates the inclusion of the patients in this present study.

Flowchart of the inclusion of the patients in the study. RCT, randomized controlled trial.

In total, 46 women and 91 men were included, with their ages ranging from 13 to 58 (median 37). Nine patients had bilateral lesions, 34 had been experiencing symptoms for less than 10 months, and 75 had not been through either cartilage repair or anterior cruciate ligament (ACL) reconstruction previously, whereas 13 had not been through any intervention at the time of inclusion. In this material, 65 patients had symptoms that could be related to one specific incident, and the defects were thereby classified as acute.

We performed an independent-samples t test on those who matched the common inclusion criteria (after excluding the article of Horas et al.[18]) and those that did not match. This yielded a nonsignificant P value (0.9).

The total number of patients not receiving any surgical treatment at the end of this study was 7. We obtained information on cartilage lesion size, International Cartilage Repair Society (ICRS) grade, and localization from MRI on these.

Analyses of the mean values of the Lysholm score based on the medical history of previous cartilage surgery patients did reveal a statistical difference. The difference between patients with previous cartilage surgery and patients with no previous cartilage surgery was evident, with P < 0.008 (). illustrates confidence intervals on the Lysholm score with regard to previous cartilage surgery. As evident in , more than 1 lesion was significantly associated with a lower Lysholm score. A t test comparing the Lysholm score between those with 1 lesion and those with 2 or more yielded P < 0.002.

Confidence intervals on Lysholm score with respect to previous surgery.

A comparison of Lysholm scores demonstrated that there was no correlation with age, gender, or time of symptoms in our patient data, which were analyzed using a multiple regression analysis. Regarding the localization of the defect, the P value was 0.001; however, as this is not a continuous variable, we performed further analysis with one-way ANOVA, with Lysholm as the dependent variable and anatomical localization as the independent variable. There was no significant correlation.

Demographic data in our referred patients () showed that the medial femoral condyle was the most common location with large mean size (3.12 cm2) of the cartilage defect and low mean Lysholm score (60). Cartilage defects located on the patellae were few and associated with a low mean Lysholm score (40). However, as illustrated in , there was no clear relation between the size of the lesion and the registered Lysholm score. Coinjuries were common, with meniscus injury as the most common one, as illustrated in .

Applicability of RCTs

We included 8 randomized studies that each use specific criteria when including participants. We assessed the inclusion criteria from these articles, as shown in . Only 6 of the 137 patients matched all the inclusion criteria in the RCTs on cartilage surgery. When analyzing the remaining patients, we found that 2 did not fit the RCT inclusion criteria due to age, 3 due to anatomical localization of lesions, 2 due to the occurrence of several lesions, and 55 due to the size of their lesions. In addition, 42 patients were excluded due to 2 nonmatching factors, 21 due to 3 nonmatching factors, and 4 due to nonmatch in all 4 factors. Two patients had missing data. provides more detailed information on why the patients did not the match the RCT inclusion criteria.

Reasons for ineligibility due to size, localization, age, and number of lesions. Fifty-five patients did not match only due to size of lesion. The figure also accounts the 6 matching patients.

We also matched the patients with the common inclusion criteria after excluding the article by Horas et al.,[18] and we then found that 27 patients (20.3%) would have been eligible for inclusion in all of the remaining RCTs.

When we matched our patients with each of the studies, 42 could have been included in Knutsen et al.,[9,10] 51 in Saris et al.,[14] 41 in Gudas et al.,[12,13] 101 in Bentley et al.,[11] 9 in Horas et al.,[18] 61 in Dozin et al.,[15] 106 in Bartlett et al.,[17] and 109 in Gooding et al.[16]

Discussion

This study suggests that the potential of extrapolating results from RCTs to the general cartilage patient population is limited. Commonly, our scientific evidence used for clinical decisions concerning cartilage repair is based on the 8 RCTs referenced in the current study. However, as evidenced by the current study, there is considerable variation in the number of patients to whom it can be applied.

RCTs

The eligibility rate of patients from our center to the various RCTs ranged from 7% to over 80%. The reason for a relatively high patient eligibility rate in 3 of the articles[11,16,17] seems to be the fact that a wide range of defect sizes and all anatomical locations in the knee were accepted. On the other hand, Horas et al.[18] had very strict inclusion criteria concerning size of lesion, and this accounts for the main disparity between the referred patients in this study and our referred patients.

As mentioned, the greatest variable resulting in bias is the inclusion criterion regarding size. Other variables in our data set that have led to exclusion are age, localization, and number of lesions. Although there is lack of knowledge concerning the importance of each parameter for the prognosis, there is evidence that anatomical localizations do affect the result, with lateral femoral condyle as the most favorable one and patellae as the most challenging one.[19] Lesions on the femoral condyles also show more improvement when treated with ACI than lesions on the patellae and trochlea.[20]

One of the main findings of this study is that only 4.4% of patients referred to our cartilage clinic in 2008 would have been eligible for inclusion in all of the available RCTs on cartilage surgery. The study’s hypothesis, that patients included in RCTs on cartilage repair represent a selected group, has been verified. Even though only 6 patients satisfied all the inclusion criteria of the RCTs, a larger number of them would have been eligible for one or more of the RCTs. The large variation in eligibility is of major concern for the current literature in the field.

The results from this current study did not demonstrate a statistical difference in Lysholm score, reflecting knee symptoms, between those eligible for inclusion and those not eligible for inclusion. This suggests that the patients included in RCTs are not more disabled than the remaining knee cartilage defect patients.

Cartilage defects on the patellae, although few, were associated with a lower Lysholm score than the defects on the medial femoral condyle.

Applicability

The large variation in eligibility illustrates the variability between the RCT results and the population of cartilage defect patients. Exclusion of the article from Horas et al.[18] and a new eligibility test expanded the range of sizes and thereby the number of patients eligible, but still only 20.3% of our patients’ material matched the inclusion criteria of the remaining 7 RCTs.[9,11,13-17] To our knowledge, the number of referred patients who should match the inclusion criteria presented by an RCT before the results in the RCT are applicable to the general patient seen in the clinic has not been addressed in the orthopedic literature. Authors from other fields of medicine have focused on this discrepancy. In a study regarding patient enrollment in large RCTs of secondary prevention after transient ischemic attack (TIA) or stroke, it was found that the patients seen in private practice were not representative of the patients in the published RCTs, as 33% to 75% were not eligible for participation.[21] They concluded therefore that the inclusion criteria, which resulted in only partial applicability, were too strict. In a review by the US National Institute of Health of 41 US institutions on the same matter, an average exclusion rate of 73% was reported.[22] The conclusion in these studies was that this was not acceptable, yet in our current study, the exclusion rate is within the same range or even higher.

Clinical Value

In this study of the unselected enrollment of patients with symptomatic focal cartilage lesions in the knee, we found that 95.6% were ineligible for participation if all the published RCT inclusion criteria were to be used. When looking at one article after another, we found an enrollment percentage ranging from 6.6% to 79.6%, which in the best case excludes 1 out of 5 patients. This large variance also shows little consistency regarding inclusion criteria between different studies. In terms of advising our patients, the study of Gooding et al.[16] is the one with the highest applicability.

One way to elucidate the problem of inconsistency between RCTs and patients seen in the clinic would be for the journals to demand that all RCTs present a flowchart so that the exclusion rate of the full patient selection process is visible.[23] The main goal of an RCT is to compare two treatment options or modalities and not necessarily generalize to the entire population of patients with a certain diagnosis. Nevertheless, for the RCTs to be clinically helpful, there is a need to analyze if there is discrepancy between the group of patients seen in the clinic and the inclusion criteria of the RCTs you are leaning on when advising patients. RCTs are stated to be the gold standard of study designs due to low chance of bias when randomization, concealment of treatment allocation, and blinding have been performed. Even though the study design does not lead to bias, because the tests themselves are not biased, the reports still might present bias to the readers. Narrow inclusion criteria are necessary to minimize interindividual differences with regard to the study analysis. Thereby, there might exist a bias toward the population of patients seen in the clinic because this often is a much more heterogeneous group.

In our study, we have found that there is a bias between the population presented in the studies and the population of cartilage patients in the clinic. This is mainly due to the strict and varying inclusion criteria in the referenced RCTs.

There are both advantages and disadvantages related to RCTs in the orthopedic field. In their article, McLeod et al.[24] describe the problem with generalizing data and applying RCT results to all patients with the current disease because of strict inclusion criteria and inherent differences in patients who volunteer for trials. Randomized controlled trials may help clarify whether there are differences among the various treatment modalities, but there are definitely challenges in applying the results to “common” patients because an RCT never will include exactly “common” patients.

We have searched for good, randomized controlled studies in order to define the injuries of the group of patients who account for the population of cartilage patients presented in the “best” studies. Our study aimed to question whether current methods may be extrapolated to everyone with cartilage injuries.

Our study reveals a substantial possibility of bias between the population presented in RCTs on cartilage surgery and those referred to a major orthopedic center. This study illustrates that the inclusion criteria in RCTs do not necessarily match the majority of patients. More general agreement among clinicians on inclusion criteria may result in more representative studies. Another solution is to use data from a cartilage registry when informing the patients, as was recently done for ACL surgery patients.[25]

A registry on cartilage repair of the knee would give an extended understanding of the long-term outcome of cartilage defects. The treatment modalities in this consent will then be chosen based on clinical impression, so a distinction between the different techniques will of course be impossible. But we believe there is a clinical value in founding such a registry.

Cartilage defect patients represent a mixed group in terms of age, size of defect, anatomical location of defect, coinjuries, and previous surgery, as illustrated in the current study. A reader of an RCT that does not present a flowchart of the patient selection runs the risk of misjudging the results when interpreting the study. Additionally, the variations found in inclusion criteria in the published RCTs represent a concern related to whether the studies actually include the same patient groups. This is also a problem in other fields of medicine, as mentioned earlier and stated by the two articles regarding the applicability of RCTs to the general patient population.[21,22]

Conclusion

The results of the present study establish that RCTs on cartilage repair are not representative of the general cartilage patient population. New clinical trials conducted in line with the CONSORT rules[23] and with inclusion criteria constructed to include a larger proportion of the general cartilage patients are necessary to provide more definitive guidance for cartilage defect patients concerning treatment.

Table 1.

Assessment of the Inclusion Criteria of the 8 Articles and Common Inclusion Criteria

	Number	Size, cm2	Age	Localization	% Eligibility
Knutsen et al.[9,10]	Single lesion	2-10	18-45	Femoral condyle	31
Saris et al.[14]	Single lesion	1-5	18-50	Femoral condyle	37
Gudas et al.[12,13]	Single lesion	1-4	<40	Weight-bearing femoral condyle	30
Bentley et al.[11]	Symptomatic lesion	1-12	16-49	Whole knee joint	74
Bartlett et al.[17]	Lesion	>1	15-50	Whole knee joint	77
Gooding et al.[16]	Symptomatic lesion	1-12	15-52	Whole knee joint	80
Dozin et al.[15]	Focal defect	>1	16-40	Weight-bearing condyle	45
Horas et al.[18]	Single lesion	(3.2-5.6 as descriptive)	18-45	Weight-bearing femoral condyle	7
Common	Single, symptomatic lesion	3.2-4	18-40	Weight-bearing femoral condyle	4

Eligibility is due to the matching patients from our included patients.

Table 2.

The 8 Included RCTs, the 2 Compared Cartilage Repair Procedures for Each Study, and Number of Included Patients

RCT	Procedure 1	Procedure 2	Number of Included Patients
Knutsen et al.[9,10]	ACI	MF	80
Saris et al.[14]	CCI	MF	118
Gudas et al.[12,13]	MOAT	MF	60
Bentley et al.[11]	ACI	MP	100
Bartlett et al.[17]	ACI	Matrix-induced ACI	91
Gooding et al.[16]	ACI (periosteum)	ACI (collagen type I/III)	68
Dozin et al.[15]	ACI	MP	47
Horas et al.[18]	ACI	OCT	40

RCT, randomized controlled trial; ACI, autologus chondrocyte implantation; MF, microfracture; CCI, characterized chondrocyte implantation; MOAT, mosaic osteochondral autologus transplantation; MP, mosaicplasty; OCT, osteochondral cylinder transplantation.

Table 3.

Group Statistics: t Test Comparing Mean Lysholm Score between Previous Cartilage Surgery and No Previous Cartilage Surgery

	Additional Surgery	n	Mean	Standard Deviation	Standard Error Mean
Lysholm	No	74	60.86	17.010	1.977
	Yes	48	53.38	13.570	1.959

Table 4.

Group Statistics: t Test Comparing Mean Lysholm Score between Patients with 1 Lesion and Patients with Several Lesions

	Number	n	Mean	Standard Deviation	Standard Error Mean
Lysholm	≥2	25	49.44	12.842	2.568
	<2	93	60.60	16.300	1.690

Table 5.

Lysholm Score Due to Different Size of the Lesions

Anatomical Location	Size, Mean ± SD (n), cm2	Range, cm2	Lysholm Score, Mean ± SD (n)	Range
Patella	6.2 ± 3.5 (5)	3.2-10.0	40.4 ± 9.5 (5)	27-49
Tibiae plateau	1.5 ± 1.0 (5)	0.5-3.0	66.2 ± 12.6 (6)	51-85
Both femoral condyles	4.5 ± 3.0 (6)	1.6-10.0	52.8 ± 11.3 (5)	39-65
Trochlea	2.1 ± 1.5 (19)	0.1-6.8	58.6 ± 11.4 (16)	36-73
Medial femoral condyle	3.1 ± 3.1 (65)	0.2-16.0	59.6 ± 17.8 (62)	27-95
Lateral femoral condyle	2.4 ± 2.1 (20)	0.5-10.0	60.2 ± 14.0 (19)	41-85
Kissing lesion	3.2 ± 3.5 (9)	0.3-10.0	48.5 ± 15.1 (8)	22-65

Table 6.

Lysholm Score and Size of Lesions Due to Anatomical Localization

Cartilage Lesion	Lysholm	Range	Standard Deviation
0-1 cm2 (n = 27)	62.3	30-95	17.3
1-2 cm2 (n = 33)	56.0	22-88	17.1
2-3 cm2 (n = 20)	57.7	27-85	13.2
3-4 cm2 (n = 16)	55.8	27-87	14.1
4-5 cm2 (n = 4)	45.5	30-60	13.5
>5 cm2 (n = 15)	61.1	34-94	19.0

Table 7.

Additional Injuries in Included Patients

Coinjury	Number
Meniscus	34
Anterior cruciate ligament	22
Patella luxation	3
Other	25
None	49