An Evaluation of Publication Bias in High-Impact Orthopaedic Literature.

BACKGROUND: Statistical analysis of systematic reviews allows the results of previous studies to be combined and synthesized to assess the overall health effect of the intervention in question. Systematic reviews can also be used to guide the creation of clinical practice guidelines and are considered to have a high level of evidence. Thus, it is important that their methodological quality is of the highest standard. Publication bias presents 2 problems: (1) studies with significant results may be overrepresented in systematic reviews and meta-analyses ("false positives") and (2) studies without significant results may not be included in systematic reviews and meta-analyses ("false negatives") because each study, on its own, was underpowered, meaning that some treatment options that may have clinical benefit will not be adopted.
METHODS: We performed a study to evaluate the techniques used by authors to report and evaluate publication bias in the top 10 orthopaedic journals as well as 3 orthopaedic-related Cochrane groups. Two authors independently screened the titles and abstracts to identify systematic reviews and meta-analyses. We assessed publication bias in the systematic reviews that did not assess publication bias themselves.
RESULTS: Our final sample included 694 systematic reviews or meta-analyses that met our inclusion criteria. Our review included 502 studies (72%) that focused on clinical outcomes, with the majority of the remaining studies focused on predictive and prognostic accuracy (20%) or diagnostic accuracy (5%). Publication bias was discussed in 295 (42.5%) of the included studies and was assessed in 135 (19.5%). Of the studies that assessed publication bias, 31.9% demonstrated evidence of publication bias. Only 43% and 22% of studies that involved use of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines discussed and assessed publication bias, respectively.
CONCLUSIONS: Publication bias is infrequently discussed and assessed in the high-impact orthopaedic literature. Furthermore, nearly one-third of the studies that assessed for publication bias demonstrated evidence of publication bias. In addition to these shortcomings, fewer than half of these studies involved use of the PRISMA guidelines and yet only one-fourth of the studies assessed for publication bias. CLINICAL RELEVANCE: By understanding the degree to which publication bias is discussed and presented in high-impact orthopaedic literature, changes can be made by journals and researchers alike to improve the overall quality of research produced and reported.

The quality of a systematic review and meta-analysis is directly proportional to its methodological quality and the quality of the individual studies it includes[1,2]. A potential shortcoming in systematic reviews and meta-analyses is the overrepresentation of studies with significant results. This overrepresentation is a remnant of publication bias in clinical trials. Publication bias results from the non-publication of studies that do not demonstrate significant results or studies that demonstrate negative results. However, even in the most methodologically sound systematic review, low levels of evidence and underpowered studies in the literature can still lead to distorted conclusions[3] because of the overrepresentation of studies with significant results (“false positives”). Furthermore, treatment options that may have clinical benefit will not be adopted because studies without statistical significance may not be included in systematic reviews and meta-analyses because each study on its own was underpowered (“false negatives”). It has been demonstrated that, despite a large increase in the quality and quantity of orthopaedic studies over time, the levels of evidence still remain low[4].

Evaluation of publication bias within systematic reviews is mentioned in many pertinent reporting guidelines[5-7]. For instance, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines clearly state that authors should “specify any assessment of risk of bias that may affect the cumulative evidence (e.g., publication bias, selective reporting within studies)”[7]. Adherence to the PRISMA guidelines is recommended by many of the journals included in the present study[8]. Despite this recommendation, it has been found that publication bias is underreported in systematic reviews published in high-impact journals and that there are high levels of publication bias in those journals[9]. Studies have shown that, in systematic reviews published in anesthesiology and dermatology journals, assessment for publication bias is rare[10,11]. The consequences of not evaluating for publication bias can lead to treatments based on only part of the available evidence and can potentially lead to serious clinical consequences.

A powerful example of the ramifications of unchecked publication bias is the use of reboxetine for the treatment of depression. As a result of the positive findings of initial research, mainly sponsored by the pharmaceutical industry, reboxetine was approved for the acute treatment of major depression in the United Kingdom and parts of Europe. However, a systematic review that included more unpublished data regarding the safety and efficacy of the drug found that it was ineffective and in fact was harmful as an antidepressant[12]. This is a striking example of the potential harm that publication bias can cause and illustrates the urgent need for more transparency and accountability in published research.

Given the prevalence of publication bias in other medical specialties in addition to findings of publication bias in orthopaedic and surgical research[13], the primary objective of the present study was to assess the method by which publication bias is evaluated within high-ranking orthopaedic journals. In instances in which publication bias had been evaluated, our secondary objective was to perform post-hoc publication bias assessments of the systematic reviews that met certain inclusion criteria.

Materials and Methods

We performed a study to evaluate the techniques used by authors to report and evaluate publication bias in the top 10 orthopaedic journals as well as 3 orthopaedic-related Cochrane groups: (1) the Bone, Joint and Muscle Trauma group, (2) the Back and Neck group, and (3) the Musculoskeletal group. The top 10 orthopaedic journals, which were identified by consulting Google Scholar’s H5-Index scores for “Orthopedic Medicine & Surgery”, included The American Journal of Sports Medicine; The Journal of Bone & Joint Surgery; Spine; Clinical Orthopaedics and Related Research; Arthroscopy: The Journal of Arthroscopic and Related Surgery; Knee Surgery, Sports Traumatology, Arthroscopy; The Journal of Arthroplasty; European Spine Journal; Journal of Shoulder and Elbow Surgery; and International Orthopaedics. The present study includes only previously published research and therefore is not subject to institutional review board oversight. We applied Statistical Analyses and Methods in the Published Literature (SAMPL) guidelines for reporting descriptive statistics when necessary.

Search Strategy

On August 11, 2017, we conducted a search of PubMed for systematic reviews and meta-analyses that had been published between 2013 and 2016. The search string is available in the Appendix.

We located relevant Cochrane reviews from the selected groups by accessing the associated Cochrane Library web page for the group and manually extracting references for those reviews that met our inclusion criteria.

Using Rayyan (Qatar Computing Research Institute), a tool for optimizing work flow of systematic reviews, 2 authors independently screened the titles and abstracts to identify systematic reviews and meta-analyses. We determined an article to be a systematic review if the methods involved summarizing evidence across multiple studies that had been identified through a comprehensive, well-described literature search. Meta-analyses were defined as any quantitative synthesis of data from multiple studies. We did not include any primary studies (i.e., randomized controlled trials, cohort studies, case-control studies), reviews of other reviews, letters to the editor, or commentaries in our review. We used Paperpile software to retrieve and access full texts of the studies meeting our inclusion criteria.

Data Extraction and Analysis

Using a pilot-tested Google Form, we extracted elements from each systematic review and/or meta-analysis. We stored the results in Microsoft Excel. Data were analyzed with use of Microsoft Excel and Comprehensive Meta-Analysis software (Biostat).

Assessing for Publication Bias

We assessed publication bias in the systematic reviews from our sample that did not assess publication bias themselves and that met the following eligibility criteria: (1) the systematic review had to include a meta-analysis, (2) the meta-analysis had to include ≥10 primary studies, and (3) sufficient data from the primary studies (e.g., odds ratios, risk ratios, mean differences, or other effect size measurements with confidence intervals) had to be included in the systematic review. Our minimum required size of 10 primary studies was based on previous research in The BMJ[14] indicating that statistical power is too low to distinguish chance from actual asymmetry in <10 studies. In the event that a systematic review had >1 meta-analysis that met our inclusion criteria, we assessed publication bias of the meta-analysis with the most primary studies. When necessary in order to replicate the analyses and perform publication bias assessments, we made minor adjustments to the upper confidence interval until symmetry was reached. We used funnel plots, the Egger regression test, and the Duval and Tweedie trim-and-fill method to assess publication bias in those meta-analysis that met inclusion criteria. We set p < 0.05 as the significance level for the Egger regression test. All publication bias assessments were performed with use of Comprehensive Meta-Analysis software.

Search Results

Our PubMed and Cochrane Library searches returned 848 and 74 studies, respectively (Fig. 1). We uploaded our search results to Rayyan and screened the returned studies by title and abstract, yielding 717 studies. The retained studies were uploaded to Paperpile to assist in full-text retrieval and data extraction. Our final sample included 694 systematic reviews or meta-analyses that met our inclusion criteria.

Fig. 1

PRISMA flowchart demonstrating the study acquisition process. MA = meta-analysis, and PB = publication bias.

Results

Characteristics of Included Studies

Our review includes 502 studies (72%) that focused on clinical outcomes, with the majority of the remaining studies focused on predictive and prognostic accuracy (20%) or diagnostic accuracy (5%). The majority of studies included >1 study type, with 56% of the systematic reviews including randomized controlled trials. Studies from 32 countries, most frequently the United States (206 studies; 30%) and China (138 studies; 20%) were included. Fifty percent (346) of the included studies mentioned the use of the PRISMA reporting guidelines. Grey literature searches (171 studies; 24.6%) and bibliography or hand searches (559 studies; 80.5%) were common methods used to reduce publication bias.

Publication Bias Reporting

Publication bias was discussed in 295 (42.5%) of the included studies and was assessed in 135 (19.5%) (Table I). The studies by the 3 Cochrane groups discussed publication bias 90% of the time, whereas the studies in Arthroscopy discussed publication bias only 18.8% of the time. Publication bias was assessed most often in The Journal of Arthroplasty (36.4%) and least often in Arthroscopy (6.0%) (Table II). Construction and visual analysis of funnel plots was the most common method for assessing publication bias (107 studies; 79.3%). Other methods for assessing publication bias included the Begg rank correlation test, Egger regression, Duval and Tweedie trim-and-fill test, and Rosenthal failsafe-N test. Of the studies that assessed publication bias, 31.9% reported evidence of publication bias and another 3.7% did not specify or were inconclusive as to whether publication bias was present. Only 43% and 22% of studies that were performed with use of the PRISMA guidelines discussed and assessed publication bias, respectively.

Publication Bias Assessments

We found 52 studies that met our inclusion criteria for publication bias assessment. Sufficient information was available in 41 systematic reviews to perform publication bias assessment (see Appendix). Funnel plot asymmetry was present in 36 analyses (88%). Studies were most often missing from the left side of the pooled analysis (54%). The Duval and Tweedie trim-and-fill method indicated that a mean of 3.5 added studies (range, 0 to 12 added studies) would be needed to produce symmetry of the funnel plot. Fourteen meta-analyses (34%) had evidence of publication bias when assessed with Egger regression, whereas 9 (22%) had evidence when assessed with Begg rank correlation. Evidence of publication bias from all 3 methods was noted in 7 studies (17%). No evidence of publication bias from any of the 3 methods was found in 3 studies (7%). The mean absolute percent difference between adjusted and observed point estimates was 17.2% (median, 9.2%; range, 0% to 123.1%) (Table III).

Discussion

The primary objective of the present study was to assess the methods used to evaluate for publication bias in high-ranking orthopaedic journals as well as in journals from the orthopaedic-related Cochrane groups. We found that even though publication bias was discussed in 43% of the reviews published in these journals, a formal assessment for publication bias only occurred in roughly 20% of these reviews.

Our study demonstrated a higher percentage of reviews lacking formal publication bias assessment than the findings of other studies in anesthesiology[11] and dermatology[10], which demonstrated that publication bias was assessed in 43% and 21% of reviews, respectively. The same can be said in comparison with a similar study published in 2014[9] that investigated rates of publication bias assessments across multiple medical specialties. Of the 694 reviews that were included in our study, 559 gave no indication if publication bias affected their results. Of the studies that did formally evaluate for publication bias, 31.9% reported evidence of it. It appears that, compared with other fields of medical research, the orthopaedic literature is assessing for publication bias in systematic reviews at a lower rate. This finding is concerning given the disproportionately high levels of studies with positive findings in the surgical literature[13]. The effects of the overrepresentation of studies with positive findings as the only studies that should be investigated for clinical benefit leads us to the other shortcoming of publication bias: a drug or treatment that actually has substantial clinical benefit may not be adopted because, although several independent studies indicated a positive clinical effect, each study on its own lacked the power (did not include enough patients) to show this effect as being statistically significant (e.g., p < 0.05). For example, if 3 independent studies were to show a similar substantial improvement in patient outcome, but the p value in each study was p ≥ 0.05, the individual studies would be reported as demonstrating “no significant effect.” However, if the 3 studies were to be combined using meta-analysis, their combined p value may be significant.

There are many ways to prevent the effect of publication bias in medical literature. The driving force to implement these measures must come from the journals and publishers[15-17]. One method to increase the number of relevant studies included in systematic reviews is a search of the grey literature[18]. Grey literature has been defined as “that which is produced on all levels of government, academics, business and industry in print and electronic formats, but which is not controlled by commercial publishers,” as defined by the Luxembourg Convention on Grey Literature (i.e., conference proceedings, technical documents or government reports).[19] Although searching for grey literature can be difficult, it provides a more complete view of the evidence. There are several guides that provide both reliable sources of grey literature and sound search methods[18,20-23]. Furthermore, publication bias also could be avoided if authors were required by journals to report the total sample size, the magnitude of the observed effect (mean and standard deviation), and the magnitude of the associated p value (and/or confidence interval), regardless of whether the p value was above or below an arbitrary value (e.g., p < 0.05) specified as corresponding to “statistical significance.” This would allow reviewers to more accurately assess the total magnitude of the observed effect and the significance of a study’s findings and would permit a meta-analysis that includes all of the studies, rather than just the ones with individually positive outcomes. In addition, we believe that if the medical community were to abandon the arbitrary use of the p value as a measure of “success,” the degree of publication bias would drastically decrease, as proposed in recent high-profile literature[24-26]. Such efforts have the power to mitigate the proven negative effects of publication bias and prevent an ineffective or harmful treatment from becoming the standard, or conversely, a potentially effective treatment from being dismissed as having “no significant effect” simply because the study or studies examining it individually lacked the power (i.e., sufficient numbers of patients) to show the effect at p < 0.05, or any other arbitrary value. As systematic reviews are considered high-level evidence in the creation of clinical practice guidelines, there is a need to address publication bias.

Reporting guidelines for systematic reviews also have the potential to address the issue of publication bias. However, these guidelines can be effective only if they are used properly and if adherence to the guideline is required by the journal. In our study, 50% of the reviews mentioned the use of the PRISMA guideline. Of those studies, 43% discussed publication bias and only 22% evaluated for it. The Cochrane group, well known for their support of reporting guidelines such as PRISMA[27], discussed publication bias at the highest rate (90%).

One limitation of the present study is that other databases may have identified additional systematic reviews outside of PubMed and the Cochrane reviews. However, given that the journals included in our study are indexed in PubMed, this is a very low possibility. There are also other journal metrics, such as the impact factor, available to assess the rank of a journal. We chose Google Scholar’s H5-Index on the basis of its open availability. As the present study was limited to high-ranking journals, the results should not be generalized to lower-ranking journals.

In conclusion, the high level of publication bias found in the present study, coupled with the low levels of adherence to systematic review reporting guidelines, is of concern. Discussion and evaluation of publication bias in the development of systematic reviews should be standard practice. However, the quality of reporting for systematic reviews in high-ranking orthopaedic journals has been found to be poor[28]. Therefore, readers of systematic reviews should not assume that lack of assessment for publication bias indicates that publication bias was not present. The publication bias present in orthopaedic journals is worrisome, considering that randomized controlled trials published in orthopaedic journals likely have exaggerated treatment effects due to bias[29]. It has been shown that in high-ranking orthopaedic journals, such as The Journal of Bone & Joint Surgery, positive and nonpositive studies were accepted at similar rates[30]. These findings should encourage orthopaedic researchers to submit studies regardless of the direction of the results.

Appendix

Supporting material provided by the authors is posted with the online version of this article as a data supplement at jbjs.org (http://links.lww.com/JBJSOA/A93).

TABLE I

Publication Bias Variables and Counts

	No. of Studies
Variable	Yes	No	Unspecified
Publication bias discussed (n = 694)	295 (42.5%)	399 (57.5%)	—
Publication bias evaluated (n = 694)	135 (19.5%)	559 (80.5%)	—
Funnel plots presented (n = 135)	107 (79.3%)	28 (20.7%)	—
Publication bias present (n =135)	43 (31.9%)	87 (64.4%)	5 (3.7%)
Grey literature search (n = 694)	171 (24.6%)	523 (75.4%)	—
Bibliography or hand search (n = 694)	559 (80.5%)	133 (19.2%)	2 (0.3%)

TABLE II

Publication Bias Results by Journal

	No. of Studies
Journal	Yes	No
Publication bias discussed
The American Journal of Sports Medicine (n = 72)	16 (22.2%)	56 (77.8%)
The Journal of Bone & Joint Surgery (n = 17)	8 (47.1%)	9 (52.9%)
Spine (n = 49)	25 (51.0%)	24 (49.0%)
Clinical Orthopaedics and Related Research (n = 56)	26 (46.4%)	30 (53.6%)
Arthroscopy: The Journal of Arthroscopic and Related Surgery (n = 117)	22 (18.8%)	95 (81.2%)
Knee Surgery, Sports Traumatology, Arthroscopy (n = 114)	35 (30.7%)	79 (69.3%)
The Journal of Arthroplasty (n = 55)	27 (49.1%)	28 (50.9%)
European Spine Journal (n = 82)	47 (57.3%)	35 (42.7%)
Journal of Shoulder and Elbow Surgery (n= 18)	8 (44.4%)	10 (55.6%)
International Orthopaedics (n = 44)	18 (40.9%)	26 (59.1%)
Cochrane groups (n = 70)	63 (90.0%)	7 (10.0%)
Publication bias assessed
The American Journal of Sports Medicine (n = 72)	12 (16.7%)	60 (83.3%)
The Journal of Bone & Joint Surgery (n = 17)	5 (29.4%)	12 (70.6%)
Spine (n = 49)	17 (34.7%)	32 (65.3%)
Clinical Orthopaedics and Related Research (n = 56)	15 (26.8%)	41 (73.2%)
Arthroscopy: The Journal of Arthroscopic and Related Surgery (n = 117)	7 (6.0%)	110 (94.0%)
Knee Surgery, Sports Traumatology, Arthroscopy (n = 114)	16 (14.0%)	98 (86.0%)
The Journal of Arthroplasty (n = 55)	20 (36.4%)	35 (63.6%)
European Spine Journal (n = 82)	21 (25.6%)	61 (74.4%)
Journal of Shoulder and Elbow Surgery (n = 18)	5 (27.8%)	13 (72.2%)
International Orthopaedics (n = 44)	11 (25.0%)	33 (75.0%)
Cochrane groups (n = 70)	6 (8.6%)	64 (91.4%)

TABLE III

Publication Bias Assessment Results*

Article†	Begg P Value	No. of Studies Trimmed	Side of Mean Missing	Model	Observed Point Estimate (95% CI)	Adjusted Point Estimate (95% CI)	Egger Intercept	T Value
Andriolo et al.	0.06	2	Left	Random	78.93 (71.13 to 86.74)	77.15 (69.63 to 84.67)	3.98	2.75†
Avenell et al.	0.36	1	Right	Random	0.74 (0.49 to 1.11)	0.77 (0.51 to 1.17)	−0.17	0.28
Beckmann et al.	0.0001	5	Left	Fixed	0.0019 (−0.006 to 0.0099)	0.0014 (−0.0064 to 0.0093)	0.47	5.19†
Beckmann et al.	0.02	8	Left	Random	0.0076 (0.0047 to 0.01)	0.0069 (0.004 to 0.0098)	1.02	5.54†
Beckwee et al.	0.27	2	Left	Random	1.09 (1.04 to 1.14)	1.07 (1.01 to 1.13)	1.30	1.75
Buckland et al.	0.31	12	Left	Fixed	53.71 (53.47 to 53.95)	50.91 (50.68 to 51.15)	21.99	2.99†
Chee et al.	0.26	3	Left	Random	0.79 (0.46 to 1.39)	0.70 (0.42 to 1.16)	−0.12	0.28
Desai et al.	0.47	0	NA	Random	0.36 (0.22 to 0.51)	0.36 (0.22 to 0.51)	−0.37	0.36
Hewison et al.	0.15	1	Right	Random	0.49 (0.32 to 0.78)	0.51 (0.33 to 0.80)	−0.93	1.02
Higgins et al.	0.33	0	NA	Random	0.43 (−0.04 to 0.90)	0.43 (−0.04 to 0.90)	0.66	0.19
Houwert et al.	0.20	0	NA	Random	1.32 (0.65 to 2.67)	1.32 (0.65 to 2.67)	−0.62	0.58
Kamper et al.	0.24	3	Left	Random	1.02 (0.74 to 1.40)	0.96 (0.70 to 1.32)	0.54	1.14
Kang et al.	0.31	4	Left	Random	0.03 (0.02 to 0.04)	0.02 (0.01 to 0.03)	0.75	1.94
Keurentjes et al.	0.00006	3	Left	Random	91.53 (89.53 to 93.53)	90.49 (87.63 to 93.36)	−6.11	3.29†
Kim et al.	0.45	1	Right	Random	1.06 (0.98 to 1.15)	1.07 (0.98 to 1.18)	0.33	0.44
Kim et al.	0.31	6	Right	Random	0.32 (0.14 to 0.49)	0.49 (0.31 to 0.67)	1.97	0.95
Li et al.	0.35	4	Left	Random	−0.13 (−0.51 to 0.25)	−0.29 (−0.62 to 0.03)	5.41	1.43
Lieberman et al.	0.00001	0	NA	Random	−0.91 (−1.21 to −0.62)	−0.91 (−1.21 to −0.62)	−4.53	6.35
Liu et al.	0.09	2	Right	Fixed	1.37 (0.99 to 1.88)	1.43 (1.05 to 1.95)	−0.53	1.29
Ma et al.	0.21	3	Right	Random	0.36 (0.08 to 0.65)	0.54 (0.24 to 0.85)	1.74	1.11
Medina et al.	0.0004	7	Left	Random	17.94 (13.16 to 22.72)	12.98 (7.72 to 18.24)	2.68	3.98
Meijer et al.	0.24	2	Right	Fixed	1.07 (0.78 to 1.48)	1.20 (0.88 to 1.64)	−1.38	1.06
Negahban et al.	0.003	7	Left	Fixed	0.36 (0.26 to 0.45)	0.24 (0.14 to 0.33)	3.69	2.72†
Pavon et al.	0.33	4	Right	Random	0.93 (0.58 to 1.47)	1.33 (0.81 to 2.19)	−1.01	1.39
Peersman et al.	0.02	4	Left	Random	0.0104 (0.0085 to 0.0123)	0.0103 (0.0083 to 0.0122)	0.51	1.71†
Rebal et al.	0.15	2	Left	Random	0.14 (0.09 to 0.18)	0.12 (0.07 to 0.17)	1.66	0.83
Riboh et al.	0.44	2	Left	Random	1.28 (0.46 to 3.58)	1.15 (0.43 to 3.12)	0.29	1.01
Santesso et al.	0.07	6	Left	Fixed	0.91 (0.88 to 0.94)	0.85 (0.83 to 0.88)	2.58	1.62
Saragiotto et al.	0.07	4	Right	Random	−0.49 (−0.70 to −0.29)	−0.34 (−0.56 to −0.12)	−1.69	1.79
Schneider et al.	0.00005	7	Left	Random	0.012 (0.0028 to 0.021)	0.0085 (−0.000070 to 0.02)	1.12	4.28
Si et al.	0.29	1	Left	Fixed	1.42 (1.07 to 1.88)	1.39 (1.06 to 1.84)	0.29	0.55
Song et al.	0.20	3	Right	Fixed	2.45 (2.40 to 2.50)	2.48 (2.43 to 2.53)	−0.51	0.48
Towle et al.	0.21	5	Left	Random	1.29 (1.15 to 1.47)	1.13 (0.98 to 1.29)	0.48	0.35
Van Bodegom Vos et al.	0.17	2	Right	Random	0.49 (0.39 to 0.59)	0.51 (0.41 to 0.62)	−1.06	1.46
Wiggins et al.	0.04	6	Left	Random	0.06 (0.05 to 0.07)	0.05 (0.04 to 0.06)	1.42	2.39
Woodmass et al.	0.11	4	Right	Random	0.41 (0.31 to 0.51)	0.47 (0.37 to 0.57)	−1.39	1.98
Wyles et al.	0.16	6	Left	Fixed	0.12 (0.08 to 0.19)	0.07 (0.05 to 0.10)	2.51	6.10
Xiao et al.	0.26	1	Right	Fixed	0.56 (0.41 to 0.76)	0.58 (0.43 to 0.78)	−1.56	1.36
Yang et al.	0.16	7	Right	Fixed	−6.78 (−7.78 to −5.78)	−4.90 (−5.79 to −4.01)	−1.07	1.12
Yun et al.	0.21	0	NA	Random	4.40 (3.19 to 5.61)	4.40 (3.19 to 5.61)	0.77	0.87
Zhang et al.	0.31	2	Left	Random	0.24 (−0.11 to 0.59)	0.15 (−0.23 to 0.53)	1.87	1.22

CI = confidence interval, and NA = not applicable.

The references for these studies can be found in the Appendix.