Safety and efficacy of stem cell therapy: an overview protocol on published meta-analyses and evidence mapping.

BACKGROUND: Stem cell therapy (SCT) is an emerging and promising treatment measure for many conditions (e.g., chronic liver disease, diabetes mellitus, and knee osteoarthritis). Although there are numerous meta-analyses (MAs) concerning SCT, the quality of these MAs and the efficacy and safety data for SCT reported in these MAs remain unknown. Therefore, it is of utmost importance to conduct an overview of existing MAs concerning SCT for evaluating these parameters.
METHODS: We will systematically search PubMed and EMBASE databases from inception to October 2020 for identifying MAs of SCT published in English. Two independent reviewers will select appropriate MAs against the predefined eligibility criteria. The efficacy and safety data of SCT reported in MAs will be descriptively summarized. Following this, the reporting quality and methodological quality of included MAs will be appraised using Preferred Reporting Items for Systematic reviews and Meta-analyses (PRISMA) and A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR-2) tools by two reviewers, respectively. Further, the evidence mapping method will be used to present assessment results. The key information will also be extracted by two independent reviewers. The Spearman's correlation coefficient will be used to explore the association between reporting quality and methodological quality. The factors influencing the quality will be assessed through linear regression analyses. The sensitivity analysis will also be conducted. Data analyses will be performed using Stata 16.0 and Excel 2016. P<0.05 will be considered statistically significant. DISCUSSION: This overview of MAs concerning SCTs will provide comprehensive evidence on the quality of MAs and data of interest reported in MAs. Further, these data can be used to guide clinical practice and future research. OVERVIEW REGISTRATION: International Prospective Register of Systematic Reviews (PROSPERO): CRD42020206642. 2021 Annals of Translational Medicine. All rights reserved.

Introduction

Clinical and preclinical studies have identified stem cell therapy (SCT) as an emerging and promising treatment method for several diseases (1), including but not limited to age-related macular degeneration, corneal injury, diabetes mellitus, epidermolysis bullosa, knee osteoarthritis, myocardial infarction, and fire burn (1-5). However, in the era of evidence-based medicine, the basic consensus is that all healthcare decisions should be made based on the best available scientific evidence (6). A systematic review with meta-analysis (MA) is an important evidence-based evaluation method, and MA is generally used to underpin the evidence base associated with a specific question (7). MAs always play a critical role in healthcare decision-making, and can be used to inform clinical practice and future research (7,8). Nevertheless, it is noteworthy that not all MAs are useful and valued (9), and relevant problems of MAs (e.g., methodological flaws and incomplete reporting) have been pointed out in several fields (10,11). Based on our knowledge and preliminary search in PubMed, many MAs reporting efficacy and safety data concerning SCT have been identified, involving various diseases (such as chronic liver disease, diabetes mellitus, sepsis, knee osteoarthritis, stroke), in different subspecialties (3,5,12-14). For example, a recently published systematic review and MA reported that SCT is a relatively safe and effective treatment measure for diabetes mellitus (3). Zhou and colleagues investigated the efficacy and safety of SCT for chronic liver disease based on data from 24 trials and concluded SCT was also safe and effective for the treatment of chronic liver disease (12).

A systematic review of systematic reviews or MAs (i.e., overview or umbrella review) is similar to systematic review and MA (7,15), and is also a critical evidence synthesis method that can consolidate an evidence base using data from MAs, while focusing on the quality of included MAs in terms of methodological quality (i.e., rigor methodological design and implementation) and reporting quality (i.e., complete and clear reporting on key information) (7). For example, Zhu and colleagues reviewed systematic reviews and MAs concerning the use of total glucosides of paeony in the treatment of rheumatoid arthritis and found that total glucosides of paeony are an effective and safe compound for treating rheumatoid arthritis, but the methodological quality and reporting quality of included MAs were poor (11). The methodological quality and reporting quality of MAs are of paramount importance for their clinical application, because MAs with extensive methodological flaws are unreliable and misleading (7,16), and incomplete reporting usually influences the complete use of the interventions compared in MAs (17), leading to the waste of medical research and healthcare resources is accompanying (18,19). In addition, there is a lack of an evidence summary reporting the efficacy and safety data concerning SCT from MA documents. Therefore, it is necessary to perform an overview for mapping and summarizing the evidence and quality of published MAs focusing on SCT.

Here, we accordingly report a protocol of this overview concerning SCT. The objectives of the present overview include the following: (I) summarization of efficacy and safety-related evidence from published MAs focusing on SCT; (II) assessment of the methodological and reporting quality of included MAs concerning SCT; and (III) exploration of the relationship between the methodological and reporting quality and assessment of potential factors influencing the quality of MAs related to SCT. Based on the aforementioned aims, the research status quo and evidence gap of MAs concerning SCT will be presented completely. We present the following article in accordance with the Preferred Reporting Items for Systematic Review and Meta-analysis Protocols (PRISMA-P) reporting checklist (available at http://dx.doi.org/10.21037/atm-20-6892) (20).

Methods

The overview of published MAs concerning SCT will be conducted by referencing the Preferred Reporting Items for Systematic reviews and Meta-analyses (PRISMA) guideline (21) and Cochrane handbook (22). Meanwhile, we have registered this protocol on PROSPERO website (registration number: CRD42020206642). In addition, for any deviations from this protocol, we will report the details and reasons for any change in the following full-text.

Search methods

Two largest and most frequently used English databases, including PubMed and EMBASE, will be systematically searched from inception to October 2020. We used the subject headings and keywords to establish our search strategy, with the main search words including: “stem cells”, “mesenchymal stem cells”, “stem cell transplantation”, “peripheral blood stem cell transplantation”, “cord blood stem cell transplantation”, “hematopoietic stem cell transplantation”, “systematic review”, “meta-analysis” and “meta”. There will be no other language or publication type restrictions. In addition, we will check reference lists of included MAs for retrieving other potentially appropriate MAs. The search details are presented in Supplementary file.

Eligibility criteria and study selection

We will include the MAs that fit the following criteria: (I) study type: traditional MAs of clinical and preclinical primary studies investigating the efficacy and (or) safety of SCT vs. any other interventions, published in English; the definition of MA is according to the Cochrane handbook “The statistical combination of at least two studies to produce a single estimate of the effect of the healthcare intervention under consideration. A MA is simply the statistical combination of research from studies” (22); (II) subject: human or animal models with any condition, e.g., chronic liver disease (12), stroke (13), and sepsis (14); (III) intervention: all available SCTs with different delivery routes, including but not limited to, mesenchymal stem cell and adipose-derived stem cell; (IV) control: any comparison group, e.g., conventional treatment or placebo; (V) outcome: any outcomes that involve efficacy and untoward reactions, e.g., pain reduction (5) and mortality (14). Studies will be excluded if we could not access the key data or the full-text of MAs; the study type is a protocol, comment, letter, or conference abstract; or if the study is a network MA, individual patient data MA, qualitative systematic review, scoping literature review, or traditional expert review. All identified records will be imported into the reference manager software EndNote X9 (Thomson Corporation, Thomson ResearchSoft, USA) to select eligible MAs according to the abovementioned eligibility criteria. While after deduplication, two independent reviewers will perform the literature selection by reading the titles and abstracts, and the full-text of potentially appropriate MAs will be downloaded for further assessing their eligibility. In case of any discrepancy in this process, two independent reviewers will reach a consensus through discussion, or by consulting a third reviewer to make the final decision.

Quality assessment

For completely mapping the quality of MAs focusing on SCT, the two most frequently used tools, PRISMA and A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR-2) (23) will be used to appraise the reporting quality and methodological quality, respectively. PRISMA was designed to improve the reporting quality of MAs, and this guideline consists of 27 items. In this overview, the reporting quality of MAs will be assessed based on the contents in the document reported by the authors of MAs, and each item will be answered as “Yes” (the study fully met the item requirements), “Partial Yes” (the study incompletely met the item requirements), or “No” (the study did not meet the item requirements). AMSTAR-2 contains 16 items and is an updated tool for original AMSTAR. Seven of the 16 items in this tool (items 2, 4, 7, 9, 11, 13, and 15) are critical items based on the report of this tool. Similarly, three options including “Yes”, “Partial Yes”, or “No” will be used to complete the methodological quality assessment, and the overall confidence or methodological quality of a specific MA will be classified by assessing the critical and noncritical items into four levels: high (a study with no more than one noncritical weakness), moderate (a study with more than one noncritical flaws), low (a study with one critical weakness, and with or without a noncritical weakness), very low (a study with more than one critical flaws, and with or without a noncritical weakness). Two experienced reviewers will independently complete the process of quality assessment; any conflict will be solved through a discussion. A third reviewer will be consulted when necessary.

Data extraction

Two reviewers will independently extract data from each MA using a predesigned and piloted data extraction form. Any discrepancy will be solved through a consensus following a discussion. The key data that will be extracted will include the following: title, first author, number of authors, clinical topics or category of diseases, research aim, subject type (human, animal models, or a combination), year of publication, registration information, origin or country (the country of the corresponding author will be considered as the origin/country of a specific MA), journal name and impact factor, number of pages, the description of intervention details (i.e., information on SCT and comparison with control), database or source searched in the MA, search timespan, sample size of the MA (i.e., number of subjects included in each MA), number of primary studies, the tool for the assessment of risk of bias or methodological quality, funding information, involvement of a methodologist (e.g., epidemiologists and statistician, deciding by the institutes conducting the MA), publication bias assessment, use of GRADE (Grading of Recommendations, Assessment, Development and Evaluation) (6), subgroup analysis, sensitivity analysis, and meta-regression, efficacy and safety outcomes with effect sizes reported in the MAs publications, key findings and conclusion.

Data analysis

A detailed qualitative description and quantitative analysis will be used in this overview. The efficacy and safety data concerning SCT reported in MAs will be summarized descriptively. The compliance rate for items of PRISMA and AMSTAR-2 will be calculated, including the numbers and their percentage of “Yes”, “Partial Yes”, or “No”. The relationship between reporting quality and methodological quality will be assessed using Spearman’s rank correlation coefficient based on the number of “Yes” response (24). Similarly, according to the number of full compliance (“Yes” response), seven factors that potentially influence the reporting quality and methodological quality [i.e., publication year, impact factor, number of pages, registration (yes/no), international cooperation (yes/no), funding (yes/no), and involvement of a methodologist (yes/no)] will be explored using univariate and multivariate linear regression analyses. The variance inflation factor (VIF) will be used to assess the multicollinearity, and the threshold will be set as 6 in our study the accepted threshold value in other relevant publications (i.e., VIF <6 is acceptable) (25). Moreover, after summing the number of “Yes” and “Partial Yes” responses, we will conduct sensitivity analyses to consider the impact of “Partial Yes” response for the results on the relationship between reporting quality and methodological quality and influencing factors exploration. In addition, the evidence mapping method (26,27) will be used to present the results for quality assessment. The evidence mapping will constitute two axes and some bubbles. The X axe will be divided into four parts corresponding to the results concerning the methodological quality (“High”, “Moderate”, “Low”, and “Very low”) of AMSTAR-2 for each MA, whereas the Y axe will be used to describe the percentage of full compliance (“Yes” response) with the items of PRISMA. Each bubble will be used to represent a specific MA. The size of the bubble will be proportioned to the sample size of the MA, the color of the bubble will be used to describe the type of MAs (red for clinical only, green for preclinical only, and yellow for a combination). All data analyses will be completed using Stata 16.0 (StataCorp, College Station, TX, USA) and Excel 2016 (Microsoft Corporation, WA, USA). P<0.05 will be considered statistically significant.

Discussion

SCT is an emerging treatment method for many diseases, e.g., chronic liver disease, diabetes mellitus, and knee osteoarthritis. Therefore, it offers huge hope for patients with these conditions. However, the clinical decision-making should be based on high-quality evidence and should weigh benefits and harms. Although many MAs (3,5,12-14) related to SCT for various diseases have been published, their quality and clinical outcomes are unknown. Therefore, we designed an overview to map the reporting quality and methodological quality and clinical effects reported in MAs concerning SCT. To the best of our knowledge, this overview of MAs constitutes the first attempt to appraise the quality of MAs focusing on SCT based on PRISMA and AMSTAR-2 tools. Moreover, the overview will be the first to provide the clinical and preclinical evidence of efficacy and safety of SCT in MAs. Nonetheless, our study has some limitations. The most noteworthy limitation of our study is that we will only search the two largest databases and only English publications will be included. Therefore, it is possible for us to miss some studies, but hand-checking reference lists of the included MAs may compensate for this shortcoming and decrease this bias. In our overview, we will also explore the association between reporting quality and methodological quality, and assess the potential factors influencing the overall quality. Hopefully, the results of this overview of MAs concerning SCT will be useful for informed decision-making in a clinical setting and aiding further scientific research.

Presenting and reporting of results

The PRISMA guideline (21) will be used to support the reporting of the final full-text. The results of this overview of MAs concerning SCT will be submitted to a peer-reviewed journal for publication.

The article’s supplementary files as