Systematic review of bone marrow stimulation for osteochondral lesion of talus - evaluation for level and quality of clinical studies.

AIM: To clarify the quality of the studies indicating lesion size and/or containment as prognostic indicators of bone marrow stimulation (BMS) for osteochondral lesions of the talus (OLT).
METHODS: Two reviewers searched the PubMed/MEDLINE and EMBASE databases using specific terms on March 2015 in accordance with the Preferred Reporting Items for Systemic Reviews and Meta-Analyses guidelines. Predetermined variables were extracted for all the included studies. Level of evidence (LOE) was determined using previously published criteria by the Journal of Bone and Joint Surgery and methodological quality of evidence (MQOE) was evaluated using the Modified Coleman Methodology Score.
RESULTS: This review included 22 studies. Overall, 21 of the 22 (95.5%) included studies were level IV or level III evidences. The remaining study was a level II evidence. MQOE analysis revealed 14 of the 22 (63.6%) included studies having fair quality, 7 (31.8%) studies having poor quality and only 1 study having excellent quality.
CONCLUSION: The evidence supporting the use of lesion size and containment as prognostic indicators of BMS for OLTs has been shown to be of low quality.

Core tip: Bone marrow stimulation (BMS) is a reparative procedure for osteochondral lesions of the talus, promising approximately 85% success rates in the short- and mid-term. To date, the prognostic factors for BMS are lesion size and containment of the lesion. No other factors have been shown to be universal predictors. However, the level of evidence and methodological quality of evidence for clinical studies accompanying both the lesion sizes and containment are low. Overall, 95.5% of the studies included in the analysis are level IV or level III. No level I study was identified. The methodological qualities of the included studies were not strong. In particular, the scores of “primarily evaluates outcome criteria and recruitment rates” were low.

INTRODUCTION

Bone marrow stimulation (BMS) is a reparative procedure for osteochondral lesions of the talus (OLT)[1]. The aim of this arthroscopic procedure is to stimulate mesenchymal stem cells (MSCs) to promote fibrous cartilage tissue by breaching the subchondral bone plate (SBP) using an awl or wire[1]. Several investigators have demonstrated good to excellent clinical outcomes in around 85% of patients, treated with BMS for OLT, for the short to medium term[2].

The main prognostic factor in the treatment of OLT has been regarded as the lesion size[1,3,4]. The maximum size for BMS treatment is generally accepted as less than 15 mm in diameter or 150 mm2 in area. Chuckpaiwong et al[4] found that smaller than 15 mm in diameter was the critical cut-off value to obtain a successful outcome following BMS. Choi et al[5] concluded that 150 mm2 is the critical defect area beyond clinical outcomes following BMS for OLT decreased significantly. However, a recent systematic review by Ramponi et al[6] showed the critical lesion size to be 107.4 mm2 in area and/or 10.2 mm in diameter, for BMS. Containment of the lesion has also been demonstrated as a universally accepted prognostic factor for good clinical outcomes following BMS for OLT[3,7].

Recently, level of evidence (LOE) and methodological quality of evidence (MQOE) have been used to assess relative value of outcomes reported in the clinical studies[8-11]. Despite the widespread clinical use of lesion size as a cut-off value for BMS in OLT, there has been no comprehensive assessment of LOE and QOE for clinical studies accompanying both the lesion size and clinical outcomes. The same can be said for the presence or absence of containment of OLT.

The purpose of this systematic review was to clarify the LOE and MQOE of for the published literature investigating clinical outcome following BMS for OLT, with special emphasis on studies investigating lesion size and containment as predictors.

MATERIALS AND METHODS

Search strategy

A systematic literature search of the PubMed/MEDLINE and EMBASE databases was performed in March 2015 in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines[12]. Each database was searched using the following key words, (microfracture OR microdrilling OR drilling OR drill OR bone marrow stimulation OR marrow stimulation OR BMS OR abrasion chondroplasty OR arthroscopy OR arthroscopic) AND (talus OR talar OR ankle) AND (cartilage OR osteochondritis dissecans OR chondral OR osteochondral OR transchondral OR osteochondral lesion OR OCL OR OCD).

Titles and abstracts were screened using specific inclusion and exclusion criteria. Full texts of potentially relevant studies were then reviewed. Citations and references of all articles and relevant studies were manually assessed. Studies were searched and independently assessed by two independent reviewers. Differences between reviewers were discussed together and resolved by consensus or if a persistent disagreement occurred, a senior author was consulted.

Inclusion and exclusion criteria

Currently BMS is defined as microfracture, drilling, or abrasion. The inclusion criteria of the current systematic review was the following: (1) therapeutic clinical studies evaluating both lesion size of OLT and outcomes in patients who underwent BMS; (2) all patients included had more than a 24 mo follow up; (3) published in a peer-review journal; (4) published in English; and (5) full text of studies available. Exclusion criteria was the following: (1) cadaveric studies; (2) animal studies; (3) case reports; (4) review articles; (5) technique articles; (6) articles with unseparated results if more than one technique is described; (7) inadequately surgical technique description; (8) use of scaffolds; and (9) errors in reported data.

Data extraction and analysis

Two independent reviewers performed data extraction for each study. If any discrepancy existed, the senior author evaluated all available data and a consensus was reached. Studies that included more than one surgical procedure or a subgroup of patients with different follow-up times were included in the data for analysis[13,14].

The primary outcome of current study was LOE and MQOE of included studies. LOE of each study was graded based on the previously published criteria[15]. MQOE was assessed using the Modified Coleman Methodology Score (MCMS) (Table 1)[6]. This score consists of 2 parts, Part A (primarily evaluates baseline study characteristics; 0-60) and Part B (primarily evaluates outcome criteria and recruitment rates; 0-40). According to Jakobsen’s CMS, the score of excellent studies are between 85 to 100 points; good studies 70 to 84 points, fair studies 55 to 69 points and poor studies scored under 55 points[9].

Table 1

Modified Coleman Methodology Score[6]

Section	No. or factor	Score
Part A: Only one score to be given for each section
1 Study size - number of patients
	> 60	10
	41-60	7
	20-40	4
	< 20, not stated	0
2 Mean follow up (mo)
	> 24	5
	12-24	2
	< 12, not stated or unclear	0
3 Number of different surgical procedures included in each reported outcome. More than one surgical technique may beassessed but separate outcomes should be reported
	One surgical procedure	10
	More than one surgical procedure, but > 90% of subjects undergoing the one procedure	7
	Not stated, unclear, or < 90% of subjects undergoing the one procedure	0
4 Type of study
	Randomized controlled trial	15
	Prospective cohort study	10
	Retrospective cohort study	0
5 Diagnostic certainty (MRI)
	In all	5
	In > 80%	3
	In < 80%	0
6 Description of surgical procedure given
	Adequate (technique stated and necessary details of that type of procedure given)	5
	Fair (technique only stated without elaboration)	3
	Inadequate, not stated, or unclear	0
7 Description of postoperative rehabilitation
	Well described (ROM, WB and sport)	10
	Not adequately described (2 items between ROM and WB and sport)	5
	Protocol not reported	0
Part B: Scores may be given for each option in each of the three sections if applicable
1 Outcome criteria
	Outcome measures clearly defined	2
	Timing of outcome assessment clearly stated (e.g., at best outcome after surgery or follow-up)	2
	Objective, subjective and imaging criteria	6
	2 items between objective, subjective and imaging criteria	4
	Objective or subjective or radiological criteria	2
2 Procedure for assessing outcomes
	Subjects recruited (results not taken from surgeons files)	5
	Investigator independent of surgeon	4
	Written assessment	3
	Completion of assessment by subjects themselves with minimal investigator assistance	3
3 Description of subject selection process
	Selection criteria reported and unbiased	5
	Recruitment rate reported
	> 80% or	5
	< 80%	3
	Eligible subjects not included in the study satisfactorily accounted for, or 100% recruitment	5

MRI: Magnetic resonance imaging; ROM: Range of motion; WB: Weight bearing.

Statistical analysis

The statistical analysis was performed using a commercially available contemporary statistical software package (SAS 9.3; SAS Institute, Cary, NC, United States). In CMMS, all obtained scores were adjusted to percentage (each score/total score), the adjusted scores of CMMS were compared between Part A and Part B to determine statistical significance. As a Shapiro-Wilk’s W test showed non-normal distributed data, the Mann-Whitney U test was performed for this. Additionally, the adjusted score of each parameter were compared to investigate any difference using the Kruskal-Wallis test and Steel-Dwass test for data obtained without standard Gaussian distribution. A P-value < 0.05 was considered statistically significant.

RESULTS

The flow diagram is shown in Figure 1. After full texts articles were assessed based on the inclusion/exclusion criteria. There were 22 clinical studies included in the current systematic review[3-5,7,13,16-32].

Figure 1

PRISMA study selection flow diagram.

Demographics

Summary of the demographic data was shown in Table 2: 1.879 ankles were identified (931 males; 545 females)[3-5,7,13,16-32]. The mean lesion area was 111.9 mm2 and the mean diameter was 9.5 mm. The mean follow-up was 48.5 (range 24-146) mo.

Table 2

Studies included and demographic datas

Ref.	Year	No. of ankles	No. of males	No. of females	Follow -up (mo)	Lesion area (mm2)	Lesion diameter (mm)	Prognostic factors	LOE	MCMS (points)
[23]	2013	50	20	30	35.5	61.7	8.8	Lesion size	III	58
[29]	2015	15	7	8	94.8	87		Lesion size	IV	50
[5]	2009	120	80	37	35.6	111.7	11.4	Lesion size	III	56
[3]	2013	399			74	111.3		Lesion size, contained	III	61
[32]	2015	90	68	22	38.3	100		Lesion size	III	67
[24]	2013	298	184	114	52	98.5		Lesion size	III	57
[19]	2012	173	121	52	70.3	95.4		Lesion size	III	54
[4]	2008	105	73	32	31.6		8.84	Lesion size	IV	57
[16]	2000	17	13	4	84	85.2		Lesion size	III	33
[13]	2006	10	6	4	53	450		Lesion size	III	61
[18]	2011	22	16	6	32	76		Lesion size	IV	45
[30]	2014	50	28	22	27.1			Lesion size	III	69
[20]	2012	22	12	10	24			Lesion size	IV	56
[21]	2012	81	64	17	37.4	100		Lesion size	III	89
[17]	2010	35	27	8	33	90		Lesion size	IV	50
[31]	2014	58	37	21	35	124		Lesion size	IV	65
[25]	2013	50	30	20	141		8.8	Lesion size	IV	62
[26]	2013	38	23	15	52.8	100		Lesion size	IV	52
[27]	2013	50	22	28	36.3	62		Lesion size	IV	66
[28]	2015	41	17	24	42.5	67		Lesion size	IV	56
[22]	2012	25	19	5	32	110		Lesion size	IV	48
[7]	2011	130	64	66	37.2	84		Lesion size, contained	IV	50

LOE: Level of evidence; MCMS: Modified coleman methodology score.

LOE

Overall, 95.5% of the studies included were level IV[4,7,17,18,20,22,25-29,31] or level III[3,5,16,19,21,23,30,32]. No level I studies were included in the current review. Gobbi et al[13], was described as LOE I in the published journal, however, this study was re-assigned as LOE II (prospective cohort study). Table 2 shows information about LOE (Table 2).

MQOE

The mean MCMS was 57.5 ± 10.2 out of 100 points (range 38-89) (Table 3). Part A was 38.1 ± 8.1 (range 22-60; percentage: 63.5%) and Part B was 19.2 ± 5.5 (range 11-29; percentage: 48.0%), respectively. The adjusted MCMS of Part A were significantly higher than that of Part B (P < 0.05). In the part A, the adjusted MCMS of “Type of study” were significantly lower among all the parameters (P < 0.05). With regard to Part B, “Outcome criteria” had significantly higher scores compared with the others (P < 0.05). Of the 22 included studies, 14 studies (63.6%) were of fair quality[3-5,13,19,20,23-25,27,28,30-32], 7 (31.7%) of poor quality[7,16-18,22,26,28] and only 1 (4.5%) study[21].

Table 3

Outcome of modified Coleman methodology scores

Ref.	Part A							Part B			Total
	1 Study size - number of patients	2 Mean follow-up (mo)	3 No. of different surgical procedures included in each reported outcome	4 Type of study	5 Diagnostic certainty (MRI)	6 Description of surgical procedure given	7 Description of postoperative rehabilitation	1 Outcome criteria	2 Procedure for assessing outcomes	3 Description of subject selection process
[23]	7	5	10	0	5	3	10	8	5	5	58
[29]	0	5	10	0	0	5	10	10	5	5	50
[5]	10	5	10	0	5	5	10	8	5	0	58
[3]	10	5	10	0	5	5	10	8	8	0	61
[32]	10	5	10	0	5	5	5	6	3	8	57
[24]	10	5	10	0	5	5	10	10	9	3	67
[18]	10	5	10	0	5	5	10	8	3	0	56
[4]	10	5	10	0	5	3	10	6	8	0	57
[16]	4	5	10	0	0	3	0	8	5	3	38
[13]	4	5	0	10	5	5	10	10	9	3	61
[18]	4	5	10	0	5	5	5	6	5	0	45
[30]	7	5	10	0	5	5	10	10	9	8	69
[20]	4	2	10	0	5	3	5	10	9	8	56
[21]	10	5	10	15	5	5	10	10	9	10	89
[17]	4	5	10	0	5	5	5	8	5	3	50
[31]	7	5	10	0	5	5	10	10	5	8	65
[25]	7	5	10	0	0	3	5	10	12	5	57
[26]	4	5	10	0	5	3	10	10	5	0	52
[27]	7	5	10	0	5	3	10	8	8	10	66
[28]	7	5	10	0	5	3	10	8	5	3	56
[22]	4	2	10	0	5	5	5	8	9	0	48
[7]	10	5	0	0	5	0	10	10	5	5	50
mean	6.8	4.7	9.1	1.1	4.3	4	8.2	8.6	6.6	4	57.5
SD	3	0.9	2.9	3.8	1.8	1.3	2.9	1.4	2.4	3.5	10.2

MRI: Magnetic resonance imaging.

DISCUSSION

The aim of this systematic review is to clarify LOE and MQOE of published literature on BMS for OLT. Twenty-two studies with 1.879 patients were included, however, no level I study was identified in the study cohort. The result demonstrated that most of the studies reported the lesion sizes and the containment of the lesion were graded as low LOE. The quality of evidence in these studies demonstrated an average MCMS of 57.5 out of 100 points and only 4.5% of included studies were graded as excellent, which suggests that the methodological quality of the included studies was weak. In addition, scores of Part B (primarily evaluates outcome criteria and recruitment rates) was marked significantly lower than Part A (primarily evaluates baseline study characteristics. This systematic review has revealed that studies with low LOE and weak MQOE have supported this paradigm despite lesion size and the containment of the lesion being a common criteria value for the indication for BMS in treating OLT.

Lesion size and the containment of the lesion are accepted prognostic factors to use when making a decision in operative treatment for OLT[3,7]. In general, lesion size with less than 15 mm in diameter or less than 150 mm2 are applied for BMS. It is also well known that a non-contained OLT have a worse outcome than a contained OLT[7]. However, this systematic review has revealed that most of these studies were of low LOE, and recently, several investigators evaluated the trend of LOE of published clinical studies in sport-related journals[33]. Unfortunately greater than 80% of studies in foot and ankle surgery remain to have low LOE despite increasing numbers of the LOE I and LOE II studies in the clinical sports medicine literature[9,10,33]. High-level clinical evidence can fundamentally provide adequate treatment for patients based on the principles of evidence-based medicine[34]. Additionally, Moher et al[35] described that non-blinded clinical studies without allocation concealment tended to describe an overestimated treatment effect than blinded clinical studies and well-designed blinded case control studies are required to establish prognostic factors in BMS for OLT.

The current systematic study revealed that the MQOE of the included 22 studies have been weak (Table 3)[9]. Of those clinical studies “Procedure for assessing outcomes” and “Description of subject selection process” in Part B (primarily evaluates outcome criteria and recruitment rates) were significantly low. These findings are consistent with the outcomes found by a recent systematic review that analyzed the outcome data following microfracture for OLT in 24 clinical studies[36]. The authors found that approximately half of included studies did not have a patient history or patient-reported outcome data, despite the presence of well described general demographics and study design. Additionally, clinical variables (48%) and imaging data (39%) has been the least reported in these studies. Poor methodological quality of the clinical study decreases the reliability of study’s outcomes[37]. However, caution should be taken when interrupting the outcomes of methodological quality. The methodological deficiencies have been reported using Coleman Methodological Score for tendinopathy[8,38], knee cartilage lesion[9], fracture[39], ligament injury[40-42] and OLT[43]. However, to our knowledge, the validity and reliability of this score for OLT is unknown. Nevertheless, we believe the outcome of the current study is important because the modification for MCMS in the current study could improve the validity and reliability of this score for OLT.

Several limitations of the current study exist mainly due to the inclusion criteria. Studies published in database other than MEDLINE and EMBASE were not included. Clinical studies not written in English were not evaluated. Nevertheless, this study does demonstrate important findings of that the LOE and QOE of published literature, on using BMS for OLT, are insufficient to produce any solid conclusion. A further limitation was that the current study focused only on the available clinical studies. As a result, the outcomes have addressed very little of the underlying mechanisms and intrinsic limitations of BMS for OLT. Currently, underlying biological aspects of cartilage regeneration has been well discussed due to low intrinsic activity of reparative cartilaginous tissue following BMS and potential ability of biological factors, although a recent systematic review has suggested a comprehensive assessment of the evidence behind the translation of basic science to the clinical practice[44,45]. Thus, the usefulness of the outcomes from the current study depends essentially on critical appraisal of the literature on the clinical application.

In conclusion, lesion size and the containment of OLT is a commonly used prognostic parameter in the treatment of osteochondral lesion of the talus However, this systematic review has revealed that low levels of evidence and weak quality of evidence in clinical studies need to be improved before this paradigm can be fully supported.

COMMENTS

Background

Lesion sizes and containment are commonly used in the orthopaedic community to predict the clinical outcomes of bone marrow stimulation for osteochondral lesion of talus.

Research frontiers

The widespread use of lesion size and containment as prognostic indicators prompts a much-needed comprehensive assessment of the studies supporting this data.

Innovations and breakthroughs

The evidence supporting the use of lesion size and containment as prognostic indicators of bone marrow stimulation (BMS) for osteochondral lesion of the talus (OLTs) have been revealed in this study to be of low level of evidence (LOE) and of weak methodological quality of evidence. Future studies with more robust study designs are warranted should the current paradigm ever need to be fully supported.

Applications

This systematic review has revealed that low levels of evidence and weak quality of evidence in clinical studies need to be improved before this paradigm can be fully supported.

Terminology

BMS: Bone marrow stimulation; LOE: Level of evidence; MCMS: Modified Coleman Methodology Score; MQOE: Methodological quality of evidence; OLT: Osteochondral lesion of the talus.

Peer-review

This is a timely, objective, well-written, well-conducted systematic review of a topic relevant to the field of orthopaedics.