Failure Rates of Autograft and Allograft ACL Reconstruction in Patients 19 Years of Age and Younger: A Systematic Review and Meta-Analysis.

BACKGROUND: Graft choice for pediatric anterior cruciate ligament reconstruction (ACLR) is determined by several factors. There is limited information on the use and outcomes of allograft ACLR in pediatric patients. The purpose of this systematic review and meta-analysis was to quantify reported failure rates of allograft versus autograft ACLR in patients ≤19 years of age with ≥2 years of follow-up. We hypothesized that there would be higher rates of failure for allograft compared with autograft ACLR in this population.
METHODS: PubMed/MEDLINE and Embase databases were systematically searched for literature regarding allograft and autograft ACLR in pediatric/adolescent patients. Articles were included if they described a cohort of patients with average age of ≤19 years, had a minimum of 2 years of follow-up, described graft failure as an outcome, and had a Level of Evidence grade of I to III. Qualitative review and quantitative meta-analysis were performed to compare graft failure rates. A random-effects model was created to compare failure events in patients receiving allograft versus autograft in a pairwise fashion. Data analysis was completed using RevMan 5.3 software (The Cochrane Collaboration).
RESULTS: The database search identified 1,604 studies; 203 full-text articles were assessed for eligibility. Fourteen studies met the inclusion criteria for qualitative review; 5 studies were included for quantitative meta-analysis. Bone-patellar tendon-bone (BTB) represented 58.2% (n = 1,012) of the autografts, and hamstring grafts represented 41.8% (n = 727). Hybrid allografts (autograft + supplemental allograft) represented 12.8% (n = 18) of all allograft ACLRs (n = 141). The unweighted, pooled failure rate for each graft type was 8.5% for BTB, 16.6% for hamstring, and 25.5% for allograft. Allografts were significantly more likely than autografts to result in graft failure (odds ratio, 3.87; 95% confidence interval, 2.24 to 6.69).
CONCLUSIONS: Allograft ACLR in pediatric and adolescent patients should be used judiciously, as existing studies revealed a significantly higher failure rate for allograft compared with autograft ACLR in this patient population. Additional studies are needed to improve the understanding of variables associated with the high ACLR failure rate among pediatric and adolescent patients. LEVEL OF EVIDENCE: Therapeutic Level III. See Instructions for Authors for a complete description of levels of evidence.

The incidence of pediatric anterior cruciate ligament (ACL) injury and reconstruction is increasing[1-3]. Many factors likely contribute to this increase including increased participation in, and intensity of, youth sports[4,5]. Attitudes regarding early ACL reconstruction (ACLR) in skeletally immature athletes have also shifted as a result of literature showing increased risk of meniscal and chondral injuries associated with delayed reconstruction[6-12]. Supporting this shift has been literature showing that pediatric ACLR is relatively safe. A recent meta-analysis identified a <5% rate of growth disturbance, with less than one-third of those cases requiring secondary surgery to address this complication[13]. While reported results of pediatric ACLR are generally favorable[14-16], graft rupture has been consistently shown to occur at higher rates in this young, active population[17,18]. The increased incidence of graft rupture in this group highlights the need to identify and address potentially mitigating factors, including associated injuries (e.g., meniscal tear), postoperative rehabilitation protocols, return to sport clearance/timing, and graft choice[19,20].

Graft choice for pediatric ACLR is determined by several factors, including the amount of skeletal growth remaining, patient size, type of sport played, surgical technique, and surgeon preference. While allograft ACLR has the advantages of eliminating donor-site morbidity, control over the desired graft size, and ease of availability, some studies have shown higher failure rates in younger, more active populations when compared with autograft[13,21,22].

There is limited information on the use and outcomes of allograft ACLR specifically in pediatric patients, as a majority of the literature has evaluated adult populations. The purpose of this systematic review and meta-analysis was to quantify reported failure rates of allograft versus autograft ACLR in patients ≤19 years of age with ≥2 years of follow-up from the primary ACLR. Specifically, we aimed to answer the following: (1) what are the reported failure rates of allograft and autograft ACLR in pediatric/adolescent patients, and (2) do reported graft failure rates differ significantly between allograft and autograft ACLR in pediatric/adolescent patients? We hypothesized that there would be higher rates of failure for allograft compared with autograft ACLR in this patient population.

Materials and Methods

Literature Search

We utilized methods similar to those of a previous investigation[23]. The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines[24] were used to systematically identify literature examining allograft and autograft ACLR in pediatric/adolescent patients. Two university librarians (Brown University, Health Sciences Library) each conducted independent searches of both PubMed (which includes the MEDLINE databases) and Embase databases utilizing search-term combinations described in the Appendix. No time limit ranges were placed on the searches conducted on July 31, 2019, and August 1, 2019.

Eligibility Criteria

All studies found during the initial search were screened by title and abstract. Manuscripts were included in the full-text review if they described autograft and/or allograft ACLR in pediatric patients. If ≥2 studies described the same patient cohort, the most recently published study was included in the final analysis. Manuscripts were excluded if they primarily described revision ACLR, ACLR + meniscal allograft, primary ACL repair, ACLR + osteotomy, synthetic ACL grafts, nonhuman/animal studies, congenital ACL deficiency, multi-ligamentous knee injury, and imaging studies. Case reports, editorials, nonclinical studies (i.e., basic science, cadaveric, biomechanical), non-English-language articles, technique articles, economic-decision analyses, and meeting abstracts were excluded.

Study Selection

After title/abstract screening, full-text articles were reviewed and included in the final analysis if they met the following criteria: described a cohort of pediatric patients (average age of ≤19 years), had a minimum of 2 years of follow-up, adequately described ACL graft failure as an outcome, and had a Level of Evidence grade of I to III[25]. The average age of ≤19 years was chosen because this age range should be inclusive of those who are high school-aged and younger, an age group that is traditionally thought of as “pediatric/adolescent.” Bibliographies of systematic reviews and meta-analyses were manually reviewed for articles not captured by the initial search.

Data Extraction

After relevant full-text articles were selected, the study authors met and finalized the criteria for data extraction. Eight authors (A.I.C., J.J.B., M.D.E., S.W.M., A.T.P., Z.S.S., C.D.V., H.B.E.) extracted relevant data from included studies and recorded them into a spreadsheet. After initial data extraction, 2 authors (A.I.C., B.B.) confirmed extracted data points from individual full-text articles prior to final analysis. Study variables recorded included authors, journal, publication year, number of study subjects, sex, average age and range, average follow-up duration and range, graft type (allograft or autograft), and number of failed grafts by type. Hybrid grafts (e.g., autograft + allograft augmentation) were classified as allografts for the purposes of meta-analysis (since only 1 category could be chosen for proper analysis). Because of inconsistent reporting, graft diameter was not analyzed. Graft failure was defined as graft rupture/failure explicitly described in each individual study on the basis of clinical or radiographic criteria or a return to the operating room for failed ACLR. All included studies were assimilated in our qualitative analysis[21,26-38]. All studies included in the qualitative analysis that adequately reported data for our random-effects model were included in our quantitative meta-analysis. These studies presented data on both ACL autograft and allograft reconstruction within the same study cohort[21,28,31,35,38]. Although Razi et al.[26] reported on both autograft and allograft in the same cohort, this study was not included in the meta-analysis because there were 0 events of interest (e.g., graft failure) in either group.

Risk-of-Bias Assessment

The Newcastle-Ottawa Scale (NOS) for assessing quality in nonrandomized studies was used to evaluate the risk of bias within each study[39]. The NOS is an instrument used to assess the quality of eligible prospective and retrospective cohort studies. The NOS assesses study quality within 3 domains: selection, comparability, and outcome. According to NOS guidelines, stars (0 to 9) were awarded to each study on the basis of methodological quality. Studies with quality scores of 0 to 3 stars, 4 to 6 stars, and 7 to 9 stars were considered to have had high, moderate, and low risk of bias, respectively. Two authors independently assessed study quality (A.I.C., B.B.); any disagreements were resolved by consensus discussion.

Statistical Methods

A random-effects model was created to compare failure events among patients receiving allograft versus autograft in a pairwise fashion. Our model used an inverse variance approach. A random-effects model instead of a fixed-effects model was chosen because there were likely enough underlying differences between study populations (e.g., surgeon characteristics, follow-up duration, postoperative protocols) that it would not be accurate to assume that there was no heterogeneity in the estimates of ACL failure rates in pooled studies[40]. Data analysis was completed using RevMan 5.3 software (The Cochrane Collaboration).

Results

The database search identified 1,604 studies. After the removal of duplicates and title/abstract screening, 203 full-text articles were assessed for eligibility. Fourteen studies remained after the application of inclusion and exclusion criteria, and 5 studies were included in the quantitative meta-analysis. Figure 1 illustrates the search strategy.

Fig. 1

Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) flowchart.

Qualitative Analysis

Individual study characteristics are summarized in Tables I and II. The average age of participants in individual studies ranged from 14 years[27] to 19 years[38], and the average duration of follow-up ranged from 24 months[26] to 240 months[34]. The average age of the pediatric/adolescent cohort could not be determined in 3 studies[29,34,36]. Shelbourne et al.[29] stratified patients by age as <18, 18 to 25, or >25 years but did not describe the average age for the subgroups. Thompson et al.[34] stratified patients as ≤18 or >18 years of age and described the average age of the entire study cohort but did not describe the average age of those ≤18 years of age. Webster et al.[36] stratified patients as <20 or ≥20 years of age but did not report the average age in the subgroup <20 years of age. This study was not excluded as we postulated that patients <20 years of age in this subgroup still met our inclusion criteria of ≤19 years of age.

The sex of participants in individual studies ranged from 25% female[38] to 100% female[31]. One study described allograft ACLR only[30], 7 studies described autograft ACLR only[27,29,32-34,36,37], and 6 studies described both ACL allograft and autograft reconstruction within the same cohort[21,26,28,31,35,38].

Regarding autograft ACLR, bone-patellar tendon-bone (BTB) or hamstring grafts were the only autograft types described in the included studies. BTB grafts represented 58.2% (n = 1,012) of the autografts, and hamstring grafts represented 41.8% (n = 727). One study described hybrid ACL allografts and represented 12.8% (n = 18) of all allograft ACLRs (n = 141). The overall pooled autograft failure rate was 12.1%. The overall pooled failure rates for each graft type was 8.5% for BTB and 16.6% for hamstring (with the study by Thompson et al.[34] excluded from the calculation because graft failure was not distinguished between BTB or hamstring grafts). This is compared with a pooled allograft failure rate of 25.5% (26.0% if excluding the hybrid grafts; the hybrid grafts had a failure rate of 22.2% [4 of 18]). It should be noted that the graft failure rates described above are unweighted rates and are presented for descriptive purposes only. Statistical comparisons were limited to our quantitative meta-analysis described below.

Quantitative Analysis

The results of our statistical analysis are summarized in Figure 2. Five studies were included in the quantitative meta-analysis[21,28,31,35,38]. Allografts were significantly more likely than autografts to result in graft failure (odds ratio [OR], 3.87; 95% confidence interval, 2.24 to 6.69; I2 = 5%; studies = 5).

Fig. 2

Forest plot examining the 5 studies that were included in the quantitative meta-analysis. The plot shows that allografts were significantly more likely to result in graft failure compared with autografts (odds ratio, 3.87; 95% confidence interval [CI], 2.24 to 6.69). IV = inverse variance, and df = degrees of freedom.

Bias Assessment

Tables III and IV show the NOS bias assessment for the included studies. All 14 studies received 7 to 9 stars (low risk of bias): 6 studies had 9 stars, 6 had 8 stars, and 2 had 7 stars.

Discussion

The decision between autograft or allograft ACLR is a common topic of discussion between surgeons and patients. The ideal graft would be safe to harvest with minimal morbidity, mimic biomechanical properties of the native ACL, and minimize graft failure rates. Multiple options for grafts are available, each with unique risks and benefits. Allograft tissue has the advantage of less donor-site morbidity and possibly, quicker postoperative recovery. While allograft ACLR has been shown to be relatively safe in adult patients, its use in pediatric patients remains debatable[41,42].

With the present systematic review and meta-analysis, we aimed to assess and compare ACL graft failure rates in pediatric/adolescent patients. Among the included studies, we found failure rates of 8.5% for BTB autograft, 16.6% for hamstring autograft, 22.2% for hybrid autograft/allograft, and 25.5% for allograft ACLR. Our meta-analysis found an almost 4-fold increased risk (OR, 3.87) of failure for allograft compared with autograft (BTB/hamstring) ACLR. These findings are consistent with others in the literature. Kaeding et al. examined results of autograft versus allograft ACLR from the MOON (Multicenter Orthopaedic Outcomes Network) cohort, finding the highest risk of graft failure within the age group of 10 to 19 years in those who received allograft reconstruction[22]. Given these results, the potential benefits of allograft ACLR, compared with autograft reconstruction, in pediatric/adolescent patients are outweighed by the risks of graft failure.

Shanmugaraj et al. also performed a systematic review of allograft use in pediatric patients[43]. In their review of the literature, the authors found a 7.9% rate of retear following allograft ACLR, nearly double the reported autograft failure rate (4.2%). That study, however, focused only on skeletally immature patients, and as the rates of ACLR have been shown to be highest among older, high school-aged children[3,44], this limits the generalizability of their findings to younger patients and not necessarily those most at risk for rerupture as a whole. Shanmugaraj et al. concluded that the reported pooled graft failure rate of 7.9% following allograft ACLR in skeletally immature patients was acceptable[43]. On the basis of our findings in the current study, however, we caution against concluding that allograft ACLR in those ≤19 years of age has acceptable rates of graft failure compared with autograft. Rather, we conclude that the consistently higher reported rate of allograft failure is unacceptable in this young, active population and would advocate for the limited use of allografts in pediatric/adolescent ACLR.

Systematic reviews examining allograft ACLR in adult populations have found different results than those reported in the current study. Mariscalco et al. performed a systematic review and meta-analysis of autograft versus nonirradiated allograft ACLR in patients with an average age of 24.5 to 32 years[41]. They found no differences in the graft failure rate, laxity, or patient-reported outcome scores between the 2 groups. Cvetanovich et al. also performed a systematic review and meta-analysis of randomized controlled trials, comparing hamstring autograft and soft-tissue allograft in patients with a mean age of 29.9 years, and found no differences in outcomes[42]. It is worth noting, however, that the authors of the above studies cautioned against extrapolating their data to younger patient populations.

There was variability in allograft failure rates across the studies evaluated in our investigation, with rates ranging from 0% to 44%. This is likely multifactorial, but the choice of allograft type or allograft sterilization may play a role. Allograft types include all-soft-tissue grafts and soft-tissue-with-bone-block grafts. Our review did not find any studies directly comparing these different allograft types. It has also been shown that irradiated allografts and allografts treated with certain chemical sterilization techniques have poorer outcomes compared with autograft ACLR[41,45-47], which may be because of delayed graft incorporation or diminished structural integrity of the graft itself[47,48]. The studies that reported on allograft use in our analysis inconsistently reported the allograft type, and only 1 study reported on the specific sterilization process[21]. Because of this, we were unable to elucidate the influence of specific allograft type or sterilization technique on graft failure rates.

Failure rates after ACLR among pediatric/adolescent patients, in general, are higher than those reported for adults, with studies of pediatric ACLR outcomes noting failure rates of 6% to 25%[9,17,27,49-54], whereas failure rates of 1% to 8% have generally been found in studies in the adult literature[53,55-60]. Potential reasons for the higher failure rates among pediatric patients include a return to higher baseline levels of activity, continued musculoskeletal and neuromuscular development, and poorer adherence to activity restrictions during the postoperative rehabilitation period. Given the inherent differences between adult and pediatric/adolescent ACLR outcomes, it is important to distinguish these 2 populations when consolidating and critically evaluating the literature. ACLR in pediatric/adolescent patients should aim to reduce the risk of graft failure in this already vulnerable population, and we believe that this includes avoiding the use of allograft reconstruction, if possible.

Limitations

Limitations of this study include the inherent limitations of systematic reviews and meta-analyses, including the relative paucity of high-quality, nonbiased literature and detailed description of the data reported. We evaluated the included studies with a validated bias-assessment tool[39] and found a low risk of bias. Despite the large number of pediatric ACLRs performed each year[2], there were limited data meeting our inclusion criteria for final analysis. We included only Level-III or higher evidence in an attempt to improve data quality; however, there were only 4 Level-II studies and no Level-I studies available for inclusion. We pooled allograft data in our analysis because of the variability of allograft types used in individual studies. We therefore cannot comment on variations between allograft types or sterilization techniques, or whether certain allografts could be more desirable in pediatric ACLR. The effect on graft failure rates of other variables, such as surgical technique, graft preparation technique, fixation type, postoperative rehabilitation protocols, duration of follow-up, and return to sport evaluations/timing, could not be analyzed given the variable reporting of these data. Additionally, no weighting or adjustment was performed for autograft type (BTB versus hamstring) in our meta-analysis. These are all important considerations when evaluating the risk of ACLR failure, and authors of primary research studies should strive to consistently report these variables. Additionally, we did not specifically examine other clinical outcomes, such as postoperative radiographic changes or patient-reported outcome measures, and therefore cannot comment on these outcomes. Finally, we were required to rely on the age data presented within each study’s methods section, but we were unable to reliably pool age data given heterogeneous reporting. We could not, therefore, stratify risk of graft failure on the basis of age within this younger population. Additionally, because we included studies with an average patient age of ≤19 years, there may have been some individual patients who were >19 years of age[28,31,38]. This could have biased the results toward a decreased risk of rerupture in those studies.

Conclusions

Existing studies on ACLR in pediatric/adolescent patients revealed a significantly higher failure rate after allograft reconstruction compared with autograft reconstruction. Allograft ACLR in pediatric/adolescent patients should therefore be used judiciously. Additional studies are needed to improve the understanding of variables associated with the high ACLR failure rate in pediatric/adolescent patients.

Appendix

Supporting material provided by the authors is posted with the online version of this article as a data supplement at jbjs.org (http://links.lww.com/JBJSOA/A239).

TABLE I

Study Summary Characteristics

Study	LOE -Study Type*	No. of Participants	Age† (yr)	% Female	Duration of Follow-up† (mo)	Allograft‡			Autograft‡
						No.	No. Failed	% Failed	No.	No. Failed	% Failed
Razi[26] (2019)	II - PC	31	14.8 (ND)	32%	24 (ND)	13	0	0%	18 (HS)	0	0%
Salem[31] (2019)	III - CS	256	18.4 (15-25)	100%	43 (30-64)	18 (hybrid autograft/allograft)	4	22%	175 (BTB), 63 (HS)	12 (BTB), 7 (HS)	7% (BTB), 11% (HS)
Salmon[32] (2018)	III - CC	39	16 (14-18)	67%	ND (≥240)	0	0	NA	39 (HS)	15 (HS)	38% (HS)
Webster[33] (2016)	III - RC	316	17.2 (11-19)	37%	60 (36-120)	0	0	NA	316 (HS)	57 (HS)	18% (HS)
Thompson[34] (2016)	II - PC	29	ND (13-18)	ND	240 (ND)	0	0	NA	14 (BTB), 15 (HS)	7§	24%
Engelman[35] (2014)	III - CC	73	15.4 (11-18)	45%	41.3 (ND)	38	11	29%	35 (HS)	4 (HS)	11%
Webster[36] (2014)	III - CC	110	ND (<20)	ND	57.6 (≥36)	0	0	NA	110 (HS)	15 (HS)	14%
Mascarenhas[37] (2012)	III - CC	46	18 (ND)	57%	54 (24-120)	0	0	NA	23 (BTB), 23 (HS)	0	0%
Ellis et al.[21] (2012)	III - RC	79	16 (14-18)	62%	50.4 (24-135.6)	20	7	35%	59 (BTB)	2 (BTB)	3%
Pallis[38] (2012)	II - PC	120 (122 knees)	19 (18-25)	25%	ND (24-48)	16	7	44%	61 (BTB), 45 (HS)	7 (BTB), 6 (HS)	11% (BTB), 13% (HS)
Koizumi[27] (2013)	III - RC	15	14 (13-16)	47%	38# (25-48)	0	0	NA	15 (HS)	2 (HS)	13% (HS)
Barrett[28] (2011)	III - RC	224	17.8 (12-25)	ND	ND (≥24)	24	7	29%	152 (BTB), 48 (HS)	18 (BTB), 12 (HS)	12% (BTB), 25% (HS)
Shelbourne[29] (2009)	II - PC	528**	ND (14-18)	50%	ND (≥60)	0	0	NA	528 (BTB)	46 (BTB)	9% (BTB)
Sankar[30] (2006)	III - RC	12	15.6 (14-17)	50%	64 (30-97)	12	0	0%	0	0	NA

LOE = level of evidence, PC = prospective cohort, CS = case series, CC = case-control, and RC = retrospective cohort.

The values are given as the mean, with the range in parentheses, unless otherwise specified. ND = no data.

NA = not applicable, BTB = bone-patellar tendon-bone, and HS = hamstring.

No data on graft type.

Median.

Patients ≤18 years of age.

TABLE II

Summary of Concomitant Meniscal Procedures, Associated Ligament Injuries, and Definition of Graft Failure

Study	Graft Type* (no. [%])			Definition of Graft Failure
	Meniscal Repair	Meniscectomy	Associated Ligament Injury
Razi[26] (2019)	Allo, 4 (30.8%)Auto, 6 (33.3%)	Allo, 2 (15.4%)Auto, 5 (27.8%)	Allo: LCL, 1 (7.7%)MCL, 0 (0%)Auto: LCL, 0 (0%)MCL, 1 (5.6%)	Not explicitly defined
Salem[31] (2019)	BTB, 42 (24%)HS, 29 (46%)	BTB, 32 (18.3%)HS, 15 (23.8%)	Not reported	“Graft rupture” not explicitly defined
Salmon[32] (2018)	HS, 20 (10%)†	HS, 20 (10%)†	Not reported‡	“Graft rupture” not explicitly defined
Webster[33] (2016)	Not reported	Not reported	Not reported	“Medical records… were initially checked to identify patients who had sustained a second ACL injury…”;“… structured questions regarding any further injuries to the ACL-reconstructed knee or the contralateral knee”
Thompson[34] (2016)	HS, 10 (11.1%)†BTB, 7 (7.8%)†	HS, 9 (10%)†BTB, 6 (6.7%)†	Not reported‡	“Graft rupture” not explicitly defined
Engelman[35] (2014)	Allo, 2 (5.3%)Auto, 4 (11.4%)	Allo, 24 (63.2%)Auto, 12 (34.3%)	Not reported	Need for revision ACL surgery and/or MRI (magnetic resonance imaging) confirmation of ACL graft failure
Webster[36] (2014)	Not reported	HS, 207 (36.9%)†	Not reported	Patient self-report. “[They] answered structured questions regarding any further injuries to the ACL-reconstructed knee or the contralateral knee”
Mascarenhas[37] (2012)	BTB, 3 (13%)HS, 5 (21.7%)	BTB, 5 (21.7%)HS, 3 (13%)	None reported (Grade-III collateral ligament injuries excluded)	Not explicitly defined
Ellis[21] (2012)	Not reported	Not reported	Not reported	Patient self-report. “Subjects were also asked if they had sustained a reinjury to the respective knee or if they had undergone revision ACL surgery”
Pallis[38] (2012)	Not reported	Not reported	Not reported	ACL injury verified through MRI and/or diagnostic arthroscopy
Koizumi[27] (2013)	HS, 4 (26.7%)	HS, 2 (13.3%)	Not reported	“Rerupture” not explicitly defined
Barrett[28] (2011)	Not reported	Not reported	Associated collateral ligament injuries excluded	Reconstruction failure defined as 2+ Lachman, positive pivot shift, and KT-1000 results >5 mm side-to-side difference
Shelbourne[29] (2009)	Not reported	Not reported	Not reported	“ACL injury was counted if it had been confirmed by physician examination or if the patient reported the ACL injury on a subjective survey or by phone”
Sankar[30] (2006)	Not reported	Not reported	Allo: MCL, 12 (100%)	Not explicitly defined

Allo = allograft, Auto = autograft, LCL = lateral collateral ligament, MCL = medial collateral ligament, BTB = bone-patellar tendon-bone autograft, and HS = hamstring.

Percentages of overall cohort (i.e., inclusive of pediatric and adult-aged subgroups).

Exclusion criteria = “associated ligament injury requiring surgical treatment.”

TABLE III

Quality Assessment of the Included Cohort Studies by the Newcastle-Ottawa Scale†

Study	Selection				Comparability	Outcome
	Represen-tativeness of Exposed Cohort	Selection of Non-Exposed Cohort	Ascertainment of Exposure	Demonstration That Outcome of Interest Was Not Present at Start of Study	Comparability of Cohort on the Basis of Design or Analysis	Assessment of Outcome	Was Follow-up Long Enough for Outcomes to Occur?	Adequacy of Follow-up of Cohorts
Razi[26] (2019)	*	*	*	*	**	*	*	*
Salem[31] (2019)	*	*	*	*	*	*	*	*
Webster[33] (2016)	*	*	*	*	**	*	*	*
Thompson[34] (2016)	*	*	*	*	**	*	*	*
Ellis[21] (2012)	*	*	*	*	X	*	*	*
Pallis[38] (2012)	*	*	*	*	*	*	*	*
Koizumi[27] (2013)	*	*	*	*	*	*	*	*
Barrett[28] (2011)	*	*	*	*	*	*	*	*
Shelbourne[29] (2009)	*	*	*	*	**	*	*	*
Sankar[30] (2006)	*	*	*	*	X	*	*	*

Star (*) = item present, and X = item absent. A maximum of 1 star is possible for the Selection and Outcome domains, and a maximum of 2 stars for the Comparability domain.

TABLE IV

Quality Assessment of the Included Case-Control Studies by the Newcastle-Ottawa Scale†

Study	Selection				Comparability	Exposure
	Adequacy of Case Definition	Represen-tativeness of Cases	Selection of Controls	Definition of Controls	Comparability of Cases and Controls on the Basis of the Design or Analysis	Ascertainment of Exposure	Same Method of Ascertainment for Cases and Controls	Non-response Rate
Salmon et al.[32] (2018)	*	*	*	*	**	*	*	X
Engelman et al.[35] (2014)	*	*	*	*	**	*	*	X
Webster et al.[36] (2014)	*	*	*	*	**	*	*	*
Mascarenhas et al.[37] (2012)	*	*	*	*	**	*	*	*

Star (*) = item present, and X = item absent. A maximum of 1 star is possible for the Selection and Exposure domains, and a maximum of 2 stars for the Comparability domain.