Efficacy and safety of chimeric antigen receptor T-cell (CAR-T) therapy in patients with haematological and solid malignancies: protocol for a systematic review and meta-analysis.

INTRODUCTION: Patients with relapsed or refractory malignancies have a poor prognosis. Immunotherapy with chimeric antigen receptor T (CAR-T) cells redirects a patient's immune cells against the tumour antigen. CAR-T cell therapy has demonstrated promise in treating patients with several haematological malignancies, including acute B-cell lymphoblastic leukaemia and B-cell lymphomas. CAR-T cell therapy for patients with other solid tumours is also being tested. Safety is an important consideration in CAR-T cell therapy given the potential for serious adverse events, including death. Previous reviews on CAR-T cell therapy have been limited in scope and methodology. Herein, we present a protocol for a systematic review to identify CAR-T cell interventional studies and examine the safety and efficacy of this therapy in patients with haematology malignancies and solid tumours. METHODS AND ANALYSIS: We will search MEDLINE, including In-Process and Epub Ahead of Print, EMBASE and the Cochrane Central Register of Controlled Trials from 1946 to 22 February 2017. Studies will be screened by title, abstract and full text independently and in duplicate. Studies that report administering CAR-T cells of any chimeric antigen receptor construct targeting antigens in patients with haematological malignancies and solid tumours will be eligible for inclusion. Outcomes to be extracted will include complete response rate (primary outcome), overall response rate, overall survival, relapse and adverse events. A meta-analysis will be performed to synthesise the prevalence of outcomes reported as proportions with 95% CIs. The potential for bias within included studies will be assessed using a modified Institute of Health Economics tool. Heterogeneity of effect sizes will be determined using the Cochrane I 2 statistic. ETHICS AND DISSEMINATION: The review findings will be submitted for peer-reviewed journal publication and presented at relevant conferences and scientific meetings to promote knowledge transfer. PROSPERO REGISTRATION NUMBER: CRD42017075331. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

A major methodological limitation is that all eligible studies are expected to be single arm and have no comparator group. This may increase risk of bias when interpreting results.

We provide a comprehensive plan to address limitations of meta-analysis of single-arm studies that includes use of a modified Institute of Health Economics tool for assessing risk of bias in single-arm interventional studies. Our approach can serve as a model for future systematic reviews assessing early-phase clinical data that are often single arm with no control comparison.

Introduction

Among patients with relapsed and refractory malignancies, chimeric antigen receptor T (CAR-T) cell therapy is a novel immunotherapy that has shown promise in both preclinical and early clinical studies. This therapy allows for CARs directed against tumour-associated antigens (eg, CD19, HER-2) to be introduced into a patient’s T-cells; this serves to reprogram these cells to target the patient’s tumour cells. A number of small clinical trials using anti-CD19 CAR-T cells in haematological malignancies have demonstrated sustained responses in patients with advanced disease.1–5 CAR-T cell therapy for solid malignancies has also identified a number of potential cancer-specific targets and previous preclinical studies investigating efficacy and feasibility show promise.6

In spite of some evidence of efficacy of CAR-T cell therapy against some malignancies, there are a number of safety concerns that have been identified. Trials conducted by Juno Therapeutics and Kite Pharma reported mortality among patients with haematological malignancy treated with CAR-T cell therapy.7 8 The Juno Therapeutics trial was closed after the death of five patients from cerebral oedema linked to CAR-T cell therapy.9 Previous trials investigating CAR-T cell therapy among patients with solid tumours have reported adverse events from treatment including anaphylaxis and cardiac arrest.10 Past studies have also reported other challenges in applying CAR-T cell therapy to solid tumours, including a scarcity of tumour-associated antigen targets, the potent immunosuppressive effects of the tumour microenvironment and the limited trafficking of CAR-T cells to tumour sites.11 12

Due to the variability and small size of clinical trials investigating CAR-T therapy, there is a need for a systematic review to evaluate its efficacy and safety. Although no systematic review currently exists for CAR-T cell therapy among solid tumours, we have identified four publications that self-identified as systematic reviews studying the efficacy and safety of CAR-T cell therapy for patients with haematological malignancies.13–16 However, there were significant limitations in the scope and/or methodologies in these earlier reviews. Zhu et al only considered CD19-targeted CAR-T cell therapies, did not report differences between adult and paediatric populations and employed a non-systematic search strategy.14 Anwer et al only included allogeneic T cells, whereas most CAR-T cell therapy uses autologous cells.15 Another systematic review by Zhang et al only considered CD19 CAR-T cell therapies and was published in journal controlled by a predatory publisher.13 17 Finally, while titled as a systematic review, Holzinger et al is only a narrative review.16 Given these limitations of previous publications, along with rapid evolution of the CAR-T field, there is a need for a current systematic review, as we present here, that adheres to rigorous, state-of-the-art methods and summarises the findings among both solid tumours and haematological malignancies.

Our systematic review will clarify the determinants of efficacy and safety of CAR-T cell therapy and identify gaps in current practice and knowledge. This will also be the first review to investigate CAR-T cell therapy among patients with solid tumours. We expect that the results from this clinical systematic review will help inform the design of clinical trials. We also summarise our approach to appraise and analyse single-arm interventional studies that are typically conducted for early-phase biotherapeutic trials; we believe this approach may be replicated for other systematic reviews of early-phase clinical data.

Protocol

Our review protocol is reported in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses Protocol guidelines (see online supplementary research checklist).18

Research objectives

We will review controlled and uncontrolled interventional studies of CAR-T cell therapy to examine the safety and efficacy of this treatment in patients with relapsed or refractory haematological malignancies and solid tumours.

Methods and analysis

Eligibility criteria

Studies will be selected according to the eligibility criteria detailed in table 1. Interventional studies with and without comparators will be included. We anticipate that many of the included studies will be single-arm interventional studies. Full-text articles in any language will be considered. Unpublished grey literature, abstracts, commentaries, letters, reviews and editorials will be excluded.19

Table 1

Population, intervention, comparison, outcome and study design breakdown of study eligibility criteria

Category	Description of criteria
Population	Patients with solid tumour or haematological malignancies
Intervention	Chimeric antigen receptor T-cell therapy
Comparator(s)	Studies with or without any comparator will be considered
Outcome(s)	Primary outcome: Complete response Secondary outcomes: Overall response (haematological) or objective response (solid) Progressive disease Relapse Overall survival Adverse events (infection, neurotoxicity, cytokine release syndrome, B-cell aplasia, graft versus host disease; other types will be grouped by organ system affected and severity) Tertiary outcomes: Health-related quality of life Health utility measures Patient experience
Study design	Interventional: ±controlled, ±randomised

Information sources

We will search MEDLINE (OVID interface, including In-Process and Epub Ahead of Print), EMBASE (OVID interface) and the Cochrane Central Register of Controlled Trials (Wiley interface) from 1946 to 22 February 2017 and plan to update our search prior to submission for publication. Clinical trial registries will be searched to identify ongoing and completed trials. Specifically, ClinicalTrials.gov and the International Prospective Register of Systematic Reviews will be searched to identify ongoing or recently completed trials or systematic reviews. In order to further ensure a comprehensive literature search, we will examine reference lists of included studies or relevant reviews identified through the search. Authors of the studies included in the present review will be contacted for review of reported outcomes. Finally, we will circulate a bibliography of included articles to the systematic review team for feedback.

Search strategy

Specific search strategies will be created in collaboration with a Health Sciences Librarian (RS) with expertise in the design of systematic searches. The literature search strategies will be developed using keywords related to CAR-T cell therapy as well as haematological malignancies and solid tumours. Search strategies will use controlled vocabulary (eg, receptors, antigen, T-Cell) and keywords (eg, CAR-T). The syntax and subject headings used in the finalised EMBASE search strategy will be adapted to the other databases. A validated search filter for clinical studies will be applied. Both qualitative and quantitative studies will be sought. No study design, date or language limits will be imposed on the search. A peer review of Electronic Search Strategy will be performed by a second librarian who is not associated with the project.20 A draft of the MEDLINE (OVID interface) search strategy for haematological and solid tumours is shown in online supplementary appendix 1.

Study records

The literature search results will be uploaded to Distiller Systematic Review Software (DistillerSR, Evidence Partners, Ottawa, Canada). DistillerSR is a cloud-based software program that provides transparent, reproducible and audit-ready results necessary for accurate review.

Data collection process

Two review authors (EJMG, MML) will independently screen the titles and abstracts from the search results using the predefined inclusion criteria presented in table 1. A calibration test will be performed to refine the screening question prior to formally commencing the screening process. For all titles that appear to meet the inclusion criteria or where there is any uncertainty, we will access the full text. Two review authors (EJMG, FH) will assess the eligibility of full reports. Disagreement will be resolved through discussion with a third-party member (DAF, HA, MML, NK). We will record the reasons for excluding studies. In the case of screening eligibility of non-English full-text articles, Ottawa Hospital Employees with fluency in the article languages will first be contacted for assistance determining article eligibility. If the article meets the eligibility criteria for inclusion in the review, a verbatim translation using the scientific translation services at the Ottawa Hospital will be used.

Data items

Standardised drafts of data extraction forms were designed to extract all information of interest from the screened studies in adherence with the Effective Practice and Organisation of Care guidelines.21 The drafts will be used to inform the construction of the online data abstraction programme (DistillerSR). Data will be extracted independently and in duplicate from each eligible study (EJMG, FH). A calibration exercise will be conducted prior to formally starting data abstraction. Demographic information, methodology, intervention details and outcomes will be recorded. Reviewers will resolve disagreements by discussion or by conferring with one of two arbitrators (MML, DAF), who will adjudicate to resolve disagreements. Where uncertainty is identified, we will contact study authors for more information.

Study characteristics to be extracted will include the journal title, the first author, the inclusion criteria (outlined in table 1), patient characteristics (eg, mean age, sex, malignancy diagnosis), trial design, type and source of financial support, publication status from trial reports and study sample size. Study intervention characteristics to be extracted will include lymphodepletion method (preconditioning agents), previous treatment (ablative, non-ablative), failed transplant, comorbidities, concomitant medications and length of follow-up. CAR-T intervention characteristics to be extracted will detail manufacturing and cell product characteristics, including fresh or frozen, T-cell origin, selection of T-cell subsets, T-cell expansion method including cell culture duration, CAR target antigen, CAR antigen, CAR molecular structure (eg, affinity domain, hinge domain, transmembrane domain, costimulatory domain(s), signalling domain), transfection/transduction method and the therapeutic regimen (CAR-T cell dose, frequency, duration, route of administration). Absolute lymphocyte counts prior to CAR-T cell therapy administration will also be recorded as this has useful information for patient eligibility of CAR-T cells. Among solid tumours, the tumour regression grade will be reported when available. When necessary, we will obtain measures of central tendency and dispersion of data by analysing the figures and tables or by contacting the authors. Whenever possible, the results from an intention to treat analysis will be used.

Outcome justification and prioritisation

Primary outcome

Complete response, our primary outcome, will be defined by type of disease: haematological malignancies [eg, acute lymphocytic leukaemia (ALL) or acute myeloid leukaemia (AML)] and solid tumours. If complete response is not feasible, secondary response outcomes will be reported using best overall response when available. Best overall response will be defined according to the response evaluation criteria in solid tumours (RECIST) guidelines where patients will be assigned to one of the following categories: complete response, partial response, stable disease, progression or inevaluable for response.22 Studies that recruit patients in complete remission at the initiation of CAR-T cell therapy will not be included in the complete response data reported.

ALL and AML

For patients with ALL or AML, in studies that (1) do not provide a definition for complete response, it will be considered haematological response; (2) report minimal residual disease, response will be defined by any response criteria, including molecular, morphological and immunological. The sensitivity of the assay used will also be extracted for molecular response.

Solid tumours

For patients with solid tumours, target and non-target lesions recorded at measurement baseline will be defined according to the RECIST guidelines. Target lesions may include up to five lesions that will likely include lesions with the longest diameters. Non-target lesions will be inclusive of all other lesions (or disease targets), including pathological lymph nodes. Target lesion and non-target lesion complete responses will be defined as disappearance of all target lesions and non-target lesions, respectively, where non-target lesions must be accompanied by normalisation of tumour marker level as defined by RECIST guidelines.22 In patients with solid tumours, any pathological lymph nodes (among target or non-target lesions) must decrease in the short axis to less than 10 mm.22 Furthermore, in studies that report tumour regression grading of zero, response will be defined as pathological response.

Secondary outcomes

Overall response, progression of disease, relapse and adverse events are our secondary response outcomes to be measured.

Overall response or objective response

Overall response and objective response will be defined as the sum of partial or complete responses in both haematological malignancies and solid tumours. In haematological malignancies, partial response is considered when there has been a response to therapy but does not meet the criteria for complete response. In target lesion evaluation for solid tumours, partial response is defined as a 30% decrease in the sum of target lesion diameters (compared with baseline measures).22

Progressive disease

Progressive disease in haematological malignancies is considered when evidence of disease increases in the peripheral blood or bone marrow, or progression or new extramedullary disease is identified. In solid tumours, progressive disease is defined as a relative increase in the sum of target lesions by 20% (smallest sum as reference), an absolute increase in target lesions by 5 mm, as well as appearance of any new lesions.22 In both haematological and solid tumours, stable disease is defined as not meeting criteria for partial response, complete response or progression.

Relapse

Relapse is defined as a patient who has a partial or complete response but then develops disease progression. Studies that recruit patients in complete remission at the initiation of CAR-T cell therapy will be descriptively reported in the proportion of the patients that relapse. For patients with lymphoma or chronic lymphocytic leukaemia (CLL), response criteria are defined as per the RECIST guidelines.22 Lastly, if CLL is identified as circulating disease in the peripheral blood and/or bone marrow only, the response criteria that are used for AML and ALL will be employed.

Adverse events

Adverse events secondary outcomes will be used to evaluate clinical safety of CAR-T cell therapy. Adverse events are a measure of unplanned or undesired symptoms or diagnoses that occur during the study, which were absent at baseline or worsen over the course of the study.23 In the setting of CAR-T cells, adverse events of special interest include infection, neurotoxicity, cytokine release syndrome, B-cell aplasia and graft versus host disease.

Tertiary outcomes

Tertiary outcomes that will be extracted include overall survival, patient experience, health-related quality of life and health utility.

Overall survival

We will define overall survival as the time from the start of treatment to the time of death from any cause.

Patient experience

Patient experience combines a number of different dimensions including patient satisfaction, expectations and outcomes that occur throughout the experience of clinical treatment.24

Health-related quality of life

Health-related quality of life is a multidimensional concept that describes an individual’s self-perceived health status.25

Health utility

Health utility measures reflect the preference values that patients attach to their overall health status. A utility value is the global measure of health status; it summarises the effects of an intervention into one value between 0 (equal to death) and 1 (equal to perfect health). Due to the variety of measures for patient experience, health-related quality of life and health utility used in clinical trials, all reported indices will be considered.

Outcome follow-up periods

Early and durable response will be recorded among included studies. All time points will be considered due to the anticipated variability in follow-up. The details of the follow-up period will also be recorded for all studies.

Risk of bias assessment

Currently, no tool exists to assess the risk of bias for single-arm interventional studies. To assess the risk of bias tool for single-arm interventional studies, we have modified the Institute of Health Economics (IHE) risk of bias tool for case series studies26 and incorporated items from the Cochrane Collaboration’s tool for assessing risk of bias in randomised trials.27 This modified IHE tool includes assessment of the study objective, design, study population, intervention and cointerventions, outcome measures (eg, blinding, incomplete outcome data such as participants lost to follow-up, selective outcome reporting), statistical analysis, results and conclusions and conflicts of interest. Each item will be scored as high risk, moderate risk or low risk of bias. The overall risk of bias results from the quality assessment will be provided in a risk of bias graph using Review Manager 5.3 (London, UK). These judgements will be made independently by two review authors (FH, MML) based on the judging criteria provided for the modified IHE risk of bias tool for interventional study designs (see online supplementary appendix 2). Disagreements will be resolved first by discussion and then by consulting a third author (DAF) for arbitration.

Metabias assessment (or risk of bias across studies)

A recent study demonstrated that traditional funnel plots may be a potentially misleading tool to assess publication bias in meta-analyses of proportion studies, particularly where low or high event rates exist.28 The same study suggested an alternative funnel plot using study size on the vertical axis instead of standard error of log odds may be a more accurate measure of publication bias.28 Therefore, our review will follow these recommendations to assess publication bias and use an alternative funnel plot of study size against log odds for the primary outcome (complete response).

Summary measures and synthesis of results

We will perform a meta-analysis to synthesise the prevalence of outcomes reported. For patients with haematological malignancies, studies will be stratified by CD19 and non-CD19-targeted antigens. Dichotomous outcomes will be reported as proportions with 95% CIs. Continuous outcomes will be reported descriptively. A random-effects model will be employed using the DerSimonian and Laird random-effects method in order to pool outcome proportions (Comprehensive Meta-analysis 3.0, Englewood, USA). Heterogeneity of effect sizes in the pooled proportions will be calculated among included studies, for studies with n>1, using the Cochrane I statistic. The following thresholds are suggested to interpret the I statistic: 0%–40% (low heterogeneity), 30%–60% (moderate heterogeneity), 50%–90% (substantial heterogeneity) and 75%–100% (considerable heterogeneity).19 If there is considerable heterogeneity, sources of heterogeneity will be explored.

Subgroup analyses

We will perform several a priori subgroup analyses to identify any subpopulations that may be associated with different CAR-T cell therapy effectiveness. These analyses will include stratification of studies based on the type of malignancy (eg, non-Hodgkin’s lymphoma, CLL, ALL, metastatic breast cancer, etc), paediatric versus adult populations, interleukin-2 administration to cell and/or patient, lymphodepletion, T-cell origin (autologous vs allogeneic), T-cell culture time, total cell dose, T-cell persistence time, dose and persistence time, fresh versus frozen CAR-T cell product administered and CD19 CAR-T cells versus all other construct types.

Reporting of review

Our findings will be reported in agreement with the Preferred Reporting Items for Systematic Reviews and Meta-analyses statement.29 A completed copy of the checklist will be provided as a supplementary document to the main report.

Confidence in cumulative estimate

The quality of the treatment effects will be evaluated by using the systematic and comprehensive approach known as Grading of Recommendations, Assessment, Development and Evaluations (GRADE). This approach is recognised as a highly effective method in terms of comparing the treatment effectiveness and quality to clinical recommendations. The quality of evidence will be assessed across the domains of risk of bias, consistency, directness, precision and publication bias. Quality will be assigned as one of four GRADE scores (0–4) reflecting high, moderate, low or very low quality evidence.30 High-quality evidence reflects a high degree of confidence in the estimate of effect, whereas very low quality evidence indicates a high degree of uncertainty regarding the estimate of effect.

Amendments

If amendments are required for this protocol, date of each amendment will be provided with a description and rationale posted to PROSPERO.