Simultaneous Comparison of Efficacy and Tolerability of Second-Generation Antipsychotics in Schizophrenia: Mixed-Treatment Comparison Analysis Based on Head-to-Head Trial Data.

OBJECTIVE: Second-generation antipsychotics have been repeatedly shown to be superior to placebo. However, the comparative efficacy among these drugs has not been systematically evaluated. In this study, we used Mixed Treatment Comparison (MTC) procedures to elucidate the comparative efficacy and tolerability of second-generation antipsychotics.
METHODS: Seven antipsychotics were selected based on the availability of the relevant data. Data were gathered from a series of review article published by the Cochrane Collaboration. Six outcome measures were analyzed: 1) percentage of no clinically important response as defined by the original authors, 2) PANSS total score change from baseline to endpoint, 3) percentage of akathisia, 4) percentage of antiparkinson medication use, 5) percentage of total body weight increase more than 7%, and 6) percentage of drop-out due to any reasons.
RESULTS: All the second-generation antipsychotics included in this study showed fairly similar efficacy but widely different tolerability. In terms of efficacy, amisulpride, clozapine and olanzapine were ranked higher than aripiprazole, quetiapine and ziprasidone. Clozapine and olanzapine were superior in terms of akathisia and extrapyramidal symptom risk, but, far more prone to induce clinically important weight gain.
CONCLUSION: Using MTC methodology, we could line up the second generation antipsychotics according to their hierarchical superiority in terms of efficacy and tolerability. Though the wide overlap among the confidence intervals and the inconsistency between the direct and indirect comparison results may limit the validity of these results, it may still allow the important insights into the relative merits of the available drugs.

INTRODUCTION

Second-generation antipsychotics or atypical antipsychotics are generally recommended as first-line agents for the treatment and management of patients suffering from schizophrenia. However, the discrepancies among them have not been adequately appreciated, though, considerable variations in efficacy and tolerability are to be expected.1,2 This lack of evidence partly came from the scarcity of head-to-head clinical trials conducted on second-generation antipsychotics.3,4

The principles of evidence-based medicine requires physicians to search and appraise contemporary research findings and base their clinical decision on both valid and relevant evidences.5 For this purpose, clinicians often rely upon systemic reviews or meta-analysis reports. However, traditional meta-analysis only permits summarizing the comparison data of the same two treatments, but not the others. Recently, the so-called "umbrella review" or "review of reviews" format has been recommended by the Cochrane Collaboration.6,7 For a given condition, which has multiple competing treatments, the umbrella reviews summarize the results of existing systemic reviews. Even though it was helpful to some degree, there was still difficulty in forming a coherent hierarchical view regarding the relative rank of available treatments.

The latest statistical methodology called Mixed Treatment Comparison (MTC) has been introduced in order to bridge this increasing gap between the practical needs of the clinicians and the scarcity of the necessary data.8,9 This method is currently under development so that different authors use different names such as 'mixed treatment comparison', 'multiple-treatment meta-analysis', 'indirect-treatment comparison', or 'network meta-analysis'.9,10 It offers a quantitative approach of integrating all the data from available comparisons, whether they are direct or indirect comparisons.11,12,13

MTC has been widely adopted in all medical fields and successfully applied to various complex problems.14 Psychiatric literature is no exception. After Cipriani's pioneering work, which reported the comparative efficacy and acceptability of the latest antidepressants, more works have adapted the similar methodology.15 Recently, two independent groups of researchers reported the MTC results of efficacy and safety of many antipsychotics.16,17 In both studies, the authors included the available single-drug placebo-controlled trials and used the placebo group response as a common comparator throughout the evaluation. Unlike the active drug comparator used in the head-to-head studies, the term 'placebo control' means heterogeneous treatment settings in different trials and, accordingly, the placebo response has been shown to vary widely among the trials. Therefore, it is questionable to use placebo control group as a common comparator.18,19,20

Since the initial publication of "Aripiprazole versus Other Atypical Antipsychotics for Schizophrenia" in 2009, a group of authors had been publishing a series of umbrella reviews about the relative efficacy and tolerability of a second-generation antipsychotic compared to other second-generation antipsychotics.21,22,23,24,25,26,27,28,29,30,31 These reviews are unique in such a way that only the head-to-head comparison data are analyzed. These reviews provided not only the list of trials and the aggregated results, but also the summaries of raw data which would hardly be accessible even from the original article. These open data were in ideal format for a MTC procedure.

As a result, we collected and prepared data from these reviews to compile the head-to-head comparison trial data among second-generation antipsychotics. Synthesizing all the gathered evidence via direct or indirect comparisons using MTC procedure, we ranked the second-generation antipsychotics in terms of their relative efficacy, acceptability and tolerability.

METHODS

Data sources and selection criteria

The required data was obtained second-handedly from a series of Cochrane reviews. From 2009 to the present, the Cochrane Collaboration has been publishing a series of umbrella reviews about the relative efficacy and tolerability of second-generation antipsychotics.21,22,23,24,25,26,27,28,29,30,31 As a result, 11 reviews were published covering 9 antipsychotics (amisulpride, aripiprazole, clozapine, olanzapine, quetiapine, risperidone, sertindole, ziprasidone, and zotepine). These Cochrane reviews aimed to be as comprehensive as possible in the collection of available data. They included the randomized controlled trials (RCTs) comparing two or more second-generation antipsychotics directly with each other. Some of the trials were head-to-head studies, which directly compared two antipsychotics, and the others were multi-arm trials that included two or more antipsychotics with a placebo group as the common control.

The only qualified participants of the study were people with schizophrenia and other types of schizophrenia-like psychosis. The applied diagnostic criteria were inconsistent; however, all of the included studies have applied reasonable screening procedures. The durations of the trials ranged from several weeks to several months. In order to deal with this inevitable heterogeneity, the authors of the Cochrane reviews had grouped the trials into short-term (up to 12 weeks), medium-term (13 to 26 weeks), and long-term (over 26 weeks) groups. Nevertheless, the medium and long-term studies were relatively rare and the outcome measures used in these studies were not as diversified as the short-term studies. In this study, the present authors confined the scope only to the short-term studies so as to maximize the homogeneity among the data.

In addition, among the 9 second-generation antipsychotics mentioned in the Cochrane reviews, the number of trials including sertindole and zotepine was so limited (2 for sertindole and 3 for zotepine) that proper analysis seemed impractical. For this reason, only seven antipsychotics (amisulpride, aripiprazole, clozapine, olanzapine, quetiapine, risperidone, and ziprasidone in alphabetical order) were compared in this study.

Compared outcome measures

The original Cochrane reviews tried to include as many outcome variables as possible and catalogued every outcome measures used in the original studies. However, each study had utilized a unique set of outcome measures so that it was nearly impossible to extract a common set of outcome measures. For instance, certain studies had used the "presence of extrapyramidal symptoms", while other studies had used "use of antiparkinsonism medication" as a tolerability measure. In order to perform a proper analysis, an adequate number of trials must be obtained. Therefore, the outcome measures used in this study had to be quite limited. We had selected two efficacy measures ("no clinically important response", "change in PANSS total score") and four safety measures ("akathisia", "use of antiparkinsonism medication", "clinically important weight gain", "drop-out due to any reasons"), which have been widely used in the trials.

In the Cochrane reviews, the outcome measure "no clinically important response as defined by the individual studies" was selected as the primary outcome measure. Even though all of the clinical trials included somehow defined "clinically important responses", these definitions were far from standardized. Since each study applied a different study duration, selection criteria or dosage schedule, we only had to resort to the original authors' definition of "clinically important response". In order to compensate for this limitation, the change of the PANSS total score during the trial period was included as an auxiliary outcome measure.

Other outcome measures such as the "use of antiparkinsonism medication" or "presence of akathisia" also suffered from similar problems. Some studies had applied standardized rating scales, while others had not. Even among the studies which had used rating scales, differences abounded in the applied scales and the cut-off points. Therefore, rather than using the "presence of extrapyramidal symptoms" or "presence of parkinsonism", we decided on the "use of antiparkinsonism medication". In case of the "presence of akathisia", we followed the original authors' decisions to define the presence or absence of akathisia. Fortunately, for weight gain, most trials had used "more than 7% increase in total body weight"; therefore, we made use of this outcome measure.

Mixed treatment comparison procedure

The computational procedure of MTC is quite complicated and requires a special type of software. It was traditionally carried out by a general-purpose Markov Chain Monte Carlo (MCMC) software such as WinBUGS32 or JAGS.33 However, it was a very tedious process and required a detailed knowledge of the Bayesian data analysis. Recently, specialized software called GeMTC has been developed. This software automates the data preparation, model building, script generation, and report display.34 The data preparation and analysis of this study was done with GeMTC.

We ran four independent chains with different prior distributions each with 50,000 iterations. The chains were checked for convergence by the Brooks-Gelman-Rubin diagnostic tool. The Brooks-Gelman-Rubin score had to be below 1.01 to ensure adequate convergence.35 The final results were produced from the aggregation of all four chains (200,000 iterations in total). For the binary variables such as "no clinically important response", logit values produced from the MTC procedures were converted into odds ratios (ORs) to aid the interpretation. For continuous variables, the weighted mean difference (WMD) values were reported. Through the simple consistency model, we obtained mean ORs and their 95% credible intervals for the pairwise comparison of antipsychotics. This procedure was repeated for all of the pre-determined outcome measures.

In the iteration of each MCMC sampling process, the hierarchies of the compared antipsychotics (superiority or inferiority) can be determined. At the end of the analysis, the probabilities that a certain antipsychotic had ranked the first, second or nth position during the whole iteration process (=rank probabilities) could be obtained.36 From these ranked probabilities, the Surface under the Cumulative Ranking Curve (SUCRA) was calculated as suggested by Salanti et al.36 The reported rank of the second-generation antipsychotics in terms of their superiority in efficacy and tolerability was determined based on these SUCRA values.

We did not take any measures to incorporate a study-level covariate into the analysis. Furthermore, we did not weigh each study according to their study qualities. These extra measures might mitigate a possible bias due to the heterogeneity or inconsistency. However, implementing these measures into the MTC procedures would be highly sophisticated.

RESULTS

The summary of the originally included studies

Overall, a total of 73 independent randomized clinical trials (RCTs) were included. The total number of involved participants in these studies was 15,480. Meanwhile, for each measure variable, only a subset of the RCTs had to be used because all of the RCTs had utilized different sets of measure variables. The number of trials included in each analysis was shown in Table 1.

Table 1

The number of clinical trials included in MTC analyses

Since the original trials had used diverse range of measure variables, the different sets of trial had to be used in the analyses for each measure variable

Percentage of "no clinically important response"

Forty independent trials had been included in the analysis. All of the second-generation antipsychotics revealed a fairly similar percentage of "no clinically important response". Comparison pairs for which the 95% confidence interval did not include OR 1.0 were 1) clozapine and ziprasidone (OR=0.62, 95% CI=0.4-0.99), 2) olanzapine and ziprasidone (OR=0.65, 95% CI=0.47-0.93), and 3) amisulpride and ziprasidone (OR=0.62, 95% CI=0.41-0.95). The low OR indicates a lower percentage of "no clinically important response", thereby the former drug being more effective than the latter drug. In all three comparison pairs, the ziprasidone was the least effective. Although statistically insignificant, the amisulpride, clozapine and olanzapine seemed to be more effective than the aripiprazole, quetiapine and ziprasidone (Table 2). The expected ranks also revealed that the amisulpride, clozapine and olanzapine were the most effective antipsychotics (Table 7).

Table 2

Change of PANSS total score during the trial period

All the numbers should be read as drugs listed in row compared with those listed in column. For example, in the lower left corner, "1.61 (1.05-2.43)" indicates OR when ziprasidone was compared with amisulpride. In this case, ziprasidone was far more likely to get "no clinically important response", therefore, OR is greater than one. In the upper right corner, "-1.97 (-6.84-2.92)" indicates WMD when amisulpride was compared with ziprasidone. The change of PANSS total score was less negative (=smaller reduction) in ziprasidone, thus WMD is less than one. *odds ratios (ORs) for the risk of "no clinically improtant response" and weighted mean differences (WMDs) in change of PANSS total score during the trial period. Ami: amisulpride, Ari: aripiprazole, Clo: clozapine, Ola: olanzapine, Que: quetiapine, Ris: risperidone, Zip: ziprasidone, PANSS: Positive and Negative Syndrome Scale

Table 7

The expected ranks of the second generaion antipsychotics in terms of each measure variable

The expected ranks were calculated by the surface under the cumulative ranking (SUCRA) method. The higher ranking (ranks 1 or 2) indicates more effective and safest drugs. PANSS: Positive and Negative Syndrome Scale

Change in total PANSS score

When the actual changes in PANSS total score during the trial period were compared, the differences among the antipsychotics have become more indistinct. The only comparison pair with statistically significant mean difference was between olanzapine and aripiprazole (WMD=-3.15, 95% CI=-5.97--0.06) with olanzapine being more effective. The expected ranks seemed similar to those obtained from comparing the response percentages. Olanzapine, risperidone and clozapine were ranked highest in terms of the change in PANSS total score.

Percentage of akathisia

Quetiapine was found to be significantly less prone to inducing akathisia compared to aripiprazole (OR=0.46, 95% CI=0.23-0.91) and to risperidone (OR=0.50, 95% CI=0.30-0.78). Other antipsychotics except quetiapine were fairly similar in their propensity to producing akathisia (Table 3). Although statistically insignificant, risperidone and aripiprazole tended to produce akathisia more frequently than the other drugs. The highest ranked drugs were aripiprazole and risperidone, and the lowest ranked drugs were amisulpride and quetiapine.

Table 3

Odds ratios (ORs) for the risk of inducing "akathisia"

All the numbers should be read as drugs listed in row compared with those listed in column. Ami: amisulpride, Ari: aripiprazole, Clo: clozapine, Ola: olanzapine, Que: quetiapine, Ris: risperidone, Zip: ziprasidone

Percentage of antiparkinsonism medication use

A striking contrast was observed among the percentage of antiparkinsonism medication use. Table 4 showed that the participants who took risperidone and ziprasidone were most frequently prescribed with antiparkinsonism medication. In contrast, quetiapine and clozapine had the lowest risk. The percentage of antiparkinsonism medication use of risperidone was significantly higher than clozapine (OR=2.77, 95% CI=1.54-5.16), olanzapine (OR=1.58, 95% CI=1.20-2.11) and quetiapine (OR=2.44, 95% CI=1.51-4.09). In addition, that of ziprasidone was significantly higher than clozapine (OR=2.63, 95% CI=1.33-5.40), olanzapine (OR=1.50, 95% CI=1.01-2.26) and quetiapine (OR=2.33, 95% CI=1.29-4.33). Overall, clozapine, quetiapine and aripiprazole were associated with the lowest risk of antiparkinsonism medication use.

Table 4

Odds ratios (ORs) for the risk of using "antiparkinson medication"

All the numbers should be read as drugs listed in row compared with those listed in column. Ami: amisulpride, Ari: aripiprazole, Clo: clozapine, Ola: olanzapine, Que: quetiapine, Ris: risperidone, Zip: ziprasidone

Percentage of clinically important weight gain

In most studies, the clinically significant weight gain had been defined as more than 7% increase in total body weight during the study period. Although many antipsychotics were known to carry the risk of significant weight gain, our results confirmed the general impression that clozapine and olanzapine have the highest risk. The percentage of significant weight gain in patients who had taken clozapine and olanzapine were significantly higher than any of the remaining drugs. The only insignificant difference was between clozapine and olanzapine, which suggested that the two drugs have equivalent propensity to induce weight gain (Table 5). The expected ranks revealed that ziprasidone, aripiprazole and amisulpride were relatively safe in terms of weight gain.

Table 5

Odds ratios (ORs) for the risk of "clinically important weight gain" defined as more than 7% increase in total body weight during the study period

All the numbers should be read as drugs listed in row compared with those listed in column. Ami: amisulpride, Ari: aripiprazole, Clo: clozapine, Ola: olanzapine, Que: quetiapine, Ris: risperidone, Zip: ziprasidone

Percentage of dropout due to any reason

The percentage of dropout due to any reason could be regarded as a summary variable for overall effectiveness combined with tolerance. The most remarkable finding was that the risk of dropout for olanzapine was the lowest and that of ziprasidone was the highest. The remaining drugs revealed fairly similar risk of dropout. Olanzapine carried the significantly lower dropout risk more than any other drugs except for clozapine and amisulpride. Meanwhile, ziprasidone carried the significantly higher dropout risk compared to any other drugs, with the exception of quetiapine (Table 6).

Table 6

Odds ratios (ORs) for the risk of "drop out due to any reasons"

All the numbers should be read as drugs listed in row compared with those listed in column. Ami: amisulpride, Ari: aripiprazole, Clo: clozapine, Ola: olanzapine, Que: quetiapine, Ris: risperidone, Zip: ziprasidone

Consistency checking of the obtained results

In order to illustrate the possible inconsistency between direct and indirect comparisons, the consistency checking procedure was performed on the measured variable "no clinically important response". Although there were no significant differences between the logit values obtained from direct comparison and those obtained from indirect comparison (all of the 95% CIs with difference in the logit values between the two types of comparisons contained zero), several comparisons changed their signs. For example, clozapine was found to be inferior (logit value=0.51) to quetiapine in direct comparison, but superior in indirect comparison (logit value=-0.43). Careful inspection of this inconsistency revealed that the distribution of the sampled log odds ratio for direct comparison was so broad that it lacked sufficient accuracy (Figure 1). Overall, no obvious inconsistency was found between direct and indirect comparison results.

Figure 1

The inconsistency between the results obtained from direct comparison data and those from indirect comparison data. The lines depict the distribution of the obtained log odds ratios from the two types of data. This illustrative example compared the "percentage of no clinically important result" between quetiapine and clozapine. In direct comparison, quetiapine seemed to have superior efficacy, but in indirect comparison, clozapine was found to be superior. However, the narrowness of the graph indicates that the indirect data is more reliable.

DISCUSSION

In this study, we tried to integrate data from head-to-head clinical trials of second-generation antipsychotics as comprehensively as possible and to synthesize the evidence in a single integrated analysis framework called MTC. Unlike the traditional pairwise meta-analysis, the MTC framework enables simultaneous comparisons of all the included entities, and, therefore, presents a clearer and complete picture of the comparative hierarchies among second-generation antipsychotics.6

The output of the analysis was expressed as a series of pairwise ORs or WMDs. Some of these statistics could not be calculated from the pairwise meta-analysis because relevant clinical trials were rare or simply did not exist. For example, in actual clinical trials, amisulpride had never been compared with aripiprazole, clozapine and quetiapine. The MTC methodology can fill in these lacunae of information. MTC can also provide the most plausible hierarchies among the compared drugs using the SUCRA method.36 Using this method, we ran-ked seven second-generation antipsychotics in terms of their efficacy and risk of major side effects. The obtained hierarchies confirmed the general impression that all antipsychotics have fairly similar efficacy levels but differ widely in their side effects potentials.

In terms of efficacy, amisulpride, clozapine and olanzapine were ranked highest, and aripiprazole, quetiapine and ziprasidone were ranked lowest. Previous MTC results16,17 also pointed out amisulpride, clozapine and olanzapine as the most effective drugs. However, the superior efficacy of amisulpride is quite unexpected. In a meta-analysis that compared the effect size of second-generation antipsychotics with first-generation antipsychotics, clozapine and amisulpride ranked the first and the second with the greatest effects, respectively, but with a fairly large margin.1 The MTC results of the more recent study of Leucht et al. also confirmed the superiority of clozapine over amisulpride.17 As mentioned above, our amisulpride results were based on our insufficient number of comparison trials, because we did not include single-arm placebo controlled studies. The lack of a sufficient number of comparisons precludes any valid conclusion regarding the actual superiority of amisulpride.25 In this regard, the merits or disadvantages of using a placebo group as the common reference point must be mentioned. Since head-to-head trials are scarce, excluding single-arm placebo studies generates the problem of an insufficient number of comparable trials. Also, some authors warn against excluding any kind of treatment modality in the MTC procedure.37 However, placebo treatment is an artificial situation that is used only in clinical trial settings. It even differs from non-treatment.38 The placebo response can be highly influenced by the expectations of the participants or by other circumstances irrelevant to the efficacy of the active drug.39 These points suggest that the placebo response in one clinical trial cannot be simply equated with those in other trials.

Although the percentage of antiparkinson medication was not high with aripiprazole, it ranked as the worst drug in terms of the akathisia potential. In past literature, a relatively higher incidence of akathisia with aripiprazole was demonstrated.40 However, whether this phenomenon should be interpreted as akathisia or agitation has yet to be settled.41 Had the clinicians viewed the behavioral change of the patients as agitation but not as akathisia, proper management with antiparkinsonism medication would not have been implemented. Since most clinical trials used antiparkinson drug use as an indirect measure of extrapyramidal symptoms, the apparently low percentage of antiparkinson drug use with aripiprazole might have been due to the misinterpretation of akathisia as agitation or exacerbation of symptoms. Meanwhile, clozapine and quetiapine were preferred as the safest drugs in terms of extrapyramidal symptoms or akathisia; however, they were not desirable in weight gain potential. Ironically, drugs with relatively lower efficacy such as ziprasidone and aripiprazole were less prone to induce weight gain. The odds ratio of weight gain potential between the best drug (ziprasidone) and the worst drug (clozapine) was as high as 50 (Table 5). Despite the high risk of weight gain, olanzapine and clozapine were ranked highest in terms of the low overall drop-out rate, but quetiapine and ziprasidone were ranked lowest. These results coincide with those of other MTC studies and with the CATIE study that demonstrated that the discontinuation of the treatment for any cause was significantly lower in olanzapine group.17,42

The overall validity of the MTC procedure is still being hotly debated. First, for an MTC result to be valid, several assumptions must be met. The most important is the absence of unacceptable heterogeneity.6 The MTC procedure is more sensitive to homogeneity violation than the traditional meta-analysis, because, in MTC, all the included trials in the treatment network must be comparable-i.e., clinically and methodologically similar.12,43 However, these requirements are unrealistic, because the inclusion or exclusion criteria, study duration, dosage schedule, used scales and assessment method differ across studies. Strictly imposing a standard, fixed protocol to all future clinical trials for comparability may not be possible or desirable. We have no choice but to interpret the obtained results with caution, bearing in mind the heterogeneity of each study.

Violations of the homogeneity assumption may lead to the second assumption of consistency. Inconsistency points to the contradiction between the direct and the indirect evidence. For example, if A is superior to B and B is superior to C, then A must be superior to C (indirect evidence). If the direct comparison suggests that C is superior, it is inconsistent. No standard analysis method to investigate inherent inconsistency has been agreed upon yet, but Dias et al.44 proposed the "node-splitting" method to check the consistency of the MTC network. Using this method, we performed an inconsistency check for the "rate of no clinically important response" variable (Figure 1). The visual inspection clarified the relative contributions of the direct and indirect evidence to the final estimates and explained the found inconsistency between the two evidence.

Another assumption is the adequate number of included trials. The number of simultaneously compared trials in this study was at most 56 (for drop out due to any reasons). This number is far from ideal even in terms of traditional meta-analysis. Furthermore, amisulpride, aripiprazole, quetiapine and ziprasidone had rarely been included in head-to-head trials, so that collecting enough number of studies was impossible. With these drugs, the result of single unusual study can perturb the whole result. Ranking probabilities are especially known to be sensitive to small errors.37 Therefore, not only the point estimate of the relative effects or the most probable rank, but also the margin of error reflected in the confidence interval, had to be carefully inspected.37 The odds ratios from the comparisons of risperidone, olanzapine and clozapine had narrower confidence intervals, but the other comparisons had broader confidence intervals.

In this study, we used a newer method of meta-analysis called MTC to synthesize the available evidence of the relative merits or shortcomings of second-degree antipsychotics. We confirmed the general impression that the efficacy levels of these drugs are fairly comparable, but their side-effect potentials widely varied. We also demonstrated the usefulness of the MTC framework in summarizing and integrating the available data. The MTC framework is a fast-developing methodology. Its future advancement and refinement may help solidify the evidence-based medicine groundwork.