Triangular test applied to the clinical trial of azithromycin against relapses in Plasmodium vivax infections.

BACKGROUND: Sequential analysis enables repeated statistical analyses to be performed throughout a trial recruitment period, while maintaining a pre-specified power and type I error. Thus the trial can be stopped as soon as the information accumulated is considered sufficient to reach a conclusion. Sequential tests are easy to use and their statistical properties are especially suitable to trials with very straightforward objectives such as non-comparative phase II trials. We report on a phase II study based on the triangular test (TT) aiming at assessing the effectiveness of azithromycin in preventing Plasmodium vivax relapses.
METHODS: To test whether the P. vivax relapse rate was either <12% or >or= 45% in patients treated with azithromycin, a sequential analysis based on the TT was as used. Patients infected with P. vivax were treated with azithromycin, 1.2 g daily, for 7 days. The onset of a relapse infection was monitored.
RESULTS: Five patients presenting with an acute P. vivax infection were included in the study. All the patients were initially cured. Three patients reported mild gastrointestinal adverse effects. When the third patient relapsed, the sample path crossed the upper boundary of the TT, and the trial was stopped.
CONCLUSIONS: Using the triangular test, with only a small number of patients, we concluded that azithromycin was not effective enough in preventing P. vivax relapses to warrant further evaluation in phase III. It is suggested that a wider use of sequential analysis in phase II anti-infective drugs trials may have financial and ethical benefits.

Introduction

The purpose of phase II trials is to determine whether a drug is effective enough to warrant further evaluation in phase III. For ethical reasons, the smallest number of patients necessary to reach a firm conclusion should be included in a phase II trial. To this end, it is critical to choose decision methods that allows early cessation when effectiveness or ineffectiveness is clear. Sequential methods, such as the triangular test (TT), have the required properties, but although they have been described for many years, few clinical trials have yet made use of them. This paper presents a concrete example of a non-comparative phase II trial design and analysis based on the TT. In section 1, the rationale for using the TT in a phase II trial is examined. In section 2, an application of the TT to study the anti-relapse activity of azithromycin on Plasmodium vivax is described. Finally, in section 3, the impact of using the TT on the design of the present trial is outlined.

Rationale of the triangular test

Stopping rules in clinical trials

The standard approach in clinical trials is the fixed-sample analysis. It assumes a single step design in which all data are simultaneously available, and requires that only one analysis be performed after a given number N of patients is included. Stopping rules are a major ethical issue in clinical trials, since N is often large and data accumulates gradually over a period which can extend to months or years. In the usual fixed-sample design, the longer the time taken for the occurrence of the main response criterion is, the later the analysis is delayed after the beginning of the trial. Therefore, it is common practice to perform interim analyses. Supported by ethical motivation, their objective is to stop the trial early if a treatment effect is detected or alternatively if it appears that there is no treatment effect. But such repeated testing increases the probability α of wrongly rejecting the null hypothesis: for example, this probability becomes 14% if five analyses are performed at the usual 5% level [1]. The use of a more conservative nominal level of α has been proposed for each interim analysis to ensure an overall level of α. One still plans an analysis after N patients according to the fixed sample design, but interim analyses at level α' are performed before N patients and the conclusion is reached, if a significant difference is observed. Otherwise, the trial goes on until the final analysis after N patients are included [2]. This hybrid approach combines interim analyses at a level α' and then, if they are not conclusive, a final analysis at a level α as in the fixed-sample design. This does not allow early stopping if there is no treatment difference. Finally, power calculations are made for the final analysis but not for the interim analyses. For these reasons, it seems preferable to use truly sequential methods, which are designed to allow for repeated testing.

Description of sequential methods

Malariologists are familiar with the sequential analysis approach since a similar method, the double (or two steps) lot quality assurance sampling plan [3], is widely used to assess the level of in vivo antimalarial drug resistance of P. falciparum within communities. Sequential analysis methods were first used in the context of industrial quality control in the late 1920s [4]. Sequential analysis enables repeated statistical analyses to be performed throughout the trial recruitment period, while maintaining a pre-specified α and β error, enabling the trial to be stopped as soon as the information accumulated is considered sufficient to reach a conclusion. The use of sequential analysis in clinical trial has been extensively described in [5,6]. Data are analysed as the results for each participant are obtained. After each analysis, the decision is made either to continue the study by enrolling additional participants, to stop the study with the conclusion that there is a statistically significant difference, or to stop the study and conclude that there is no statistically significant difference. In the 1940s, Wald [7] developed the continuous sequential probability ratio test (SPRT). The continuous SPRT requires that the analysis be performed as the outcome for each new patient becomes available. This makes it difficult to implement in a many clinical trial settings. Whitehead [8] proposed the discrete SPRT and the triangular test (TT) for the design and analysis of phase III trials. Unlike the continuous SPRT, these two sequential methods allow for discrete data analysis (for example, after every 10 new patients) and are therefore easier to implement in a variety of settings. Both the discrete SPRT and the TT have optimal properties in terms of average sample needed [9]. They are based on straight line stopping boundaries: the discrete SPRT using an open continuation region and the TT a closed continuation region. Therefore, the TT appears more relevant, because the sample size could theoretically be infinite using the SPRT. Two statistics are used, the efficient score Z, and Fisher's information V for the parameter of interest. They are both computed under the null hypothesis H0. Each analysis results in a pair of values for the Z and V statistics. The corresponding points are plotted on the graph as they become available. The plot of Z against V is referred to as the sample path. The horizontal axis corresponds to the statistic V, representing the quantity of information accumulated since the beginning of the trial. The vertical axis corresponds to the statistic Z, representing the benefit as compared with the null hypothesis. Two straight lines, the boundaries of the test, delineate a continuation region (located in between these lines) from the region of non-rejection of the null hypothesis (i.e. no significant improvement, located beneath the bottom line) to the region of rejection of the null hypothesis (i.e. significant improvement, located above the top line). The sample path is compared with the stopping boundaries. If the plotted point lies either above the upper boundary or below the lower boundary, the trial has to be stopped. Bellissant et al.[9,10] applied the discrete SPRT and the TT to the comparison of an observed success rate p with p0, as encountered in non-comparative phase II cancer clinical trials.

Non-comparative phase II trials settings

Phase II pilot efficacy evaluation clinical trials are often non-comparative. Their usual end point is the proportion p of eligible patients who respond to the treatment as defined in the protocol. Their objective is to determine whether the response rate p is greater than a threshold value p0, i.e. the largest response rate (or clinical improvement) for which further investigations are not worthwhile. In statistical terms, the two competing hypotheses are the null hypothesis (H0: p ≤ p0) and the alternative hypothesis (Ha: p >p0). The sample size N required by the single stage design as well as the corresponding threshold C based on the exact binomial distribution of number of responses S can be computed after choosing p0, specifying Ha: p = p (p being the smallest response rate for which further investigations are worthwhile), choosing the type I error α, and choosing the power 1-β under Ha. If S is greater than C after N patients, then H0 is rejected.

Set-up of a triangular test

The description of the set-up of a TT in a non-comparative phase II trial that follows is adapted from Bellissant et al. [9,10] where greater details are given. In a phase II study design aiming to compare an observed event rate p to an expected event rate p0, the two competing hypothesis are: H0: p ≤ p0 and Ha: p >p0. The log odds ratio measures the difference between p and p0:

hence if p = p0 then θ = 0.

The log-likelihood of the data can be written as:

l(p) = S log p + (N - S)log(1 – p),

where S denotes the number of patients who experience a response (or a success), and N is the number of patient included in the study.

The Z and V statistics are respectively the first derivative and the opposite of the second order derivative of the log-likelihood evaluated at θ = 0:

If we choose p, the smallest event rate for which further investigations are worthwhile, we can specify Ha: p >p, and

The boundaries of the test are computed given the type I error α, and the power 1-β under Ha. The equations of the boundaries are given by:

Z = a + λ V and Z = -a + 3λV,

where:

and where I denotes the increment in V between two analyses, when discrete analyses are performed every n patients

I = np0 (1 - p0).

In the case where α ≠ β, based on the assumption that Z is normal, distributed with mean θV and variance V, we can use a corrected value of θ, given by the approximate formula:

where Φ(x) denotes the standard normal distribution function:

and where X ~ N(0,1).

The above section describes the usual design of the TT aiming to detect any improvement or any increase of the measured end point (response) rate. In the section that follows, a non-comparative trial based on the TT, as described by Bellissant et al.[9,10] is reported. The measured end point of the present trial being the relapse after the treatment of a P. vivax malaria attack, this trial had the peculiarity to have been designed to detect a decrease, and not an increase, of the end point rate. As a result, the region of rejection of the null hypotheses (otherwise located above the top line) was located beneath the bottom line, and the region of non-rejection of the null hypothesis (otherwise located beneath the bottom line) was located above the top line (Figure 1).

Figure 1

Design of the triangular test (upper boundary, Z = 2.58 – 1.51 V; lower boundary Z = -2.58 – 0.50 V) and the results of the three sequential analyses. The Z and V statistics were calculated each time a patient relapsed. The corresponding points were plotted on the graph and compared with the stopping boundaries. On the third analysis, the upper boundary was crossed, causing the inclusion to be stopped.

Application of the TT to study the anti-relapse activity of azithromycin on Plasmodium vivax

The prevention of relapse infections is the most important problem in the treatment of Plasmodium vivax malaria. Relapses are clinical attacks that appear in 23% to 44% of patients after schizonticidal treatment [11,12]. They are due to the persistence of hepatocytic dormant stages of the parasite, known as hypnozoites. To date, the 8-aminoquinoline primaquine is the only antimalarial drug effective against the hypnozoites that can be used to cure P. vivax infection. Tafenoquine, a recently developed 8-aminoquinoline [11] is not yet available. However, there are three main drawbacks in the use of 8-amino-quinolines: 1) they may induce haemolytic anaemia in persons deficient in glucose-6-phosphate dehydrogenase (a frequent condition in areas where P. vivax is endemic); 2) they are not registered in several countries; and 3) primaquine-resistant P. vivax is now emerging [13]. Therefore, new anti-relapse drugs need to be developed. We speculated that azithromycin, an azalid antibiotic analogue to erythromycin, might be an anti-relapse drug of interest. Indeed, azithromycin achieves high tissue concentrations; it is mainly metabolised in the liver (where the hypnozoites are sheltered) and it has an excellent therapeutic index with the low level of toxicity characteristic of macrolide antibiotics; it may also be used in children and pregnancy. Azithromycin proved efficient in the prophylactic treatment of P. vivax malaria [14], but it had a lower antimalarial activity for the treatment of P. vivax attacks than clindamycin or tetracyclines [15]. Azithromycin has no hypnozoitocidal activity in rhesus monkeys infected with P. cynomolgi B [16]. Yet, to our knowledge, the activity of azithromycin on the hypnozoites of P. vivax has not been tested. The purpose of this phase II clinical trial was to determine whether azithromycin was effective enough in preventing P. vivax relapses to warrant further evaluation in phase III.

Methods

The hospital's Ethics Advisory Board (CCPRB Marseille 2) approved the study protocol. All patients received written and oral information, and appropriate informed consent was obtained.

Patients

Criteria for inclusion in the study were: 1) P. vivax infection: body temperature ≥ 38°C and P. vivax identified in a blood smear stained with Giemsa; 2) age from 18 to 70 years; 3) agreement to comply with the study protocol. Criteria for non-inclusion in the study were: 1) other treatment usually efficient against P. vivax administered at a curative dose within 10 days before the inclusion; 2) fever related to a condition other than P. vivax infection; 3) personal history of adverse reaction to macrolide or macrolide-related drugs.

Study design

We carried out a non-comparative, open-label, phase II, prospective study. The patients were administered azithromycin 1.2 g once a day (two tablets of Azadose® 600 mg, Pfizer, 86 rue de Paris, F-91407 ORSAY Cedex) for 7 days (total dose 8.4 g). A member of the medical staff witnessed each dosing. Body temperature was measured, any adverse reaction was recorded, and blood smear examinations were carried out every day during the first week, and then once a week for five weeks. The parasite clearance time is the time taken for parasite counts to fall below detectable levels in a peripheral blood smear. The fever clearance time is the time taken for the body temperature to fall and remain below 38°C for >48 h. Transaminase levels were measured at baseline, day 4, day 7, and day 28. After the first six weeks of follow-up, the patients were interviewed by telephone about relapses until they had reached either an end point or the end of the study. When a relapse infections occurred, the patient was treated with chloroquine, 25 mg/kg total dose.

End point of interest

The end point of the study was the onset of a relapse infection. A relapse was defined as a recurrence of a P. vivax infection after day 28. Reinfection was excluded because none of the patients left metropolitan France, where P. vivax transmission do not occur, during the entire study period.

Statistical analysis

This study was based on the TT [8,10]. We planned a continuous monitoring because P. vivax malaria has a relatively low occurrence rate in our ward, and it was easy to perform the analysis as the outcome for each new patient became available. We designed the study to have a 5% type I error and a 90% power (α = 0.05 and β = 0.10). We considered that a relapse rate <12% indicated an adequate clinical protection, and thus was the minimum detectable improvement in relapse rate worthwhile to be detected at the β = 10% level, while arelapse rate ≥ 45% indicated insufficient protection [12,1]. Thus, H0 (p ≥ p0) and Ha (p

Results

We included five patients, 3 males and 2 females aged 18–32 years, in the study. Their baseline characteristics are summarised in Table 1. Two patients acquired P. vivax in the Comoro Islands, and three in India. The three patients with a previous history of malaria may be considered as immune. All the patients complied well with the treatment, they all had an immediate adequate clinical response; their body temperature returned below 38 C° in less than 4 days. All patient were cured at day-28. We recorded only mild adverse events. Three patients complained of nausea and abdominal discomfort occurring within 30 to 60 minutes after each dosing; one patient (n° 3) reported loose stool emission two hours after each dosing. The transaminase levels remained within the normal range in all the patients. Three patients relapsed (relapse rate 60%). At the time the third patient relapsed we reached the conclusion that the relapse rate was not significantly different from the null hypotheses (Table 2 & Figure 1) and thus we stopped the trial.

Table 1

Characteristics, initial treatment outcome, and relapse pattern of patients infected with Plasmodium vivax and treated with azithromycin, 1.2 g once a day, for 7 days

Patients characteristics					Initial treatment		Relapse
N°	Sex	Age (years)	Previous malaria	Area of acquisition	Parasite clearancea (hours)	Fever clearanceb (hours)	Outcome	End point (days)
5	F	26	No	India	186	89	Relapsed	43
2	F	22	Yes	India	65	51	Relapsed	45
4	M	32	No	Comoros	161	42	Relapsed	108
1	M	32	Yes	Comoros	96	24	Censored	396
3	F	18	Yes	Comoros	161	17	Censored	407

a The parasite clearance time is the time taken for the parasite count to fall below detectable levels in a peripheral blood smear. b The fever clearance time is the time taken for the body temperature to fall below 38°C and to remain below this value for >48 h.

Table 2

Sequential analysis of the Plasmodium vivax relapse rate in patients treated with azithromycin

Analysis No	S	N	p	Z	V	Z significant	Z not significant
-	0	0	0	0	0	-2.58	2.58
1	1	2	0.50	0.10	0.50	-2.83	1.83
2	2	4	0.50	0.20	0.99	-3.07	1.08
3	3	5	0.60	0.75	1.24	-3.20	0.71

The trial was designed with the triangular test for p0 = 0.45, p = 0.12, α = 0.05, and β = 0.10. S denotes the number of relapses observed; N, the number of patients included; p, is the relapse rate; Z= S – N p0; and V = N p0 (1 - p0). The setting of the triangular test is detailed in the text. At the third analysis, the Z value exceeded the upper boundary value (bolded in the table). At this time, the data indicate that the study should stop, and we can conclude that azithromycin treatment has not significantly reduced the relapse rate of P. vivax infection.

Discussion

One advantage of conducting in a non-endemic country a study aiming to evaluate the effectiveness of a drug effective on hypnozoites of P. vivax is that the relapse infections are easily distinguished from the reinfections, since reinfection can occur only if the patient has travelled to an endemic country during the follow-up period. One disadvantage, however, is the relatively low incidence of P vivax infection; and thus the relatively small number of eligible patients available. Another disadvantage is that a relapse infection might occur more than two years after the infection. Because we wanted to avoid the complexity of a multi-centre design, the analysis would have been considerably delayed to achieve an appropriate power level in a fixed-sample design. To meet all these requirements, we were compelled to choose a truly a sequential decision-making method allowing to include the smallest number of patients necessary to achieve an adequate power and to perform a continuous monitoring of a delayed and infrequent outcome: the TT.

Using the TT, our findings clearly indicate that azithromycin does not reliably prevent relapses in P. vivax infections. This prompts us to conclude that the issue of the activity of azithromycin against the hypnozoites of P. vivax does not warrant further clinical studies. While we wanted to adequately cure the P. vivax attacks, we did not want to use another antimalarial drug which might have altered the outcome of our patients. Pukrittayakamee et al. treated 18 patients infected with P. vivax with a 3-day course of azithromycin (1.5 g total dose) [15]. Recrudescence was observed in 89%. Although early relapses cannot be ruled out, they are very unlikely because these patients were followed up for only 28 days. Recrudescence might be due to the insufficient free plasma and intra-erythrocytic [17] concentrations of azithromycin resulting from an insufficient dosing and/or too short a duration of the treatment. To allow for this, we administered a higher dose of azithromycin for a longer (7-day) course. To our knowledge, this is the first clinical trial in which as much as 8.4 g of azithromycin has been administered. This regimen was well tolerated (the recorded gastrointestinal adverse episodes were mild), and successfully cured all P. vivax attacks. However, secondary outcomes derived from such a small sample size study require very careful interpretation and should not be generalised without caution.

Sequential tests are easy to use and their statistical properties are especially suitable to trials with straightforward objectives such as phase II trials [18]. Unlike the conventional fixed-sample design, the sequential design we used in this study allowed for early termination when inefficacy was clear. In sequential designs the sample size is a random variable whose distribution depends on the true treatment difference and on the stopping rule used. Usually, sequential designs will give sample size values which are well below those needed in equivalent fixed-sample designs [8]. Stimulation studies indicated that the use of a TT-based sequential analysis design brought about a 50% reduction of the average sample needed, when compared to the fixed-sample design [10]. In the present study, if we had used a fixed-sample design with identical criteria, the sample size needed would have been 15 patients. Using the a TT-based sequential analysis, sufficient information was accumulated to reach a conclusion as soon as the fifth patient reached an end point. That is, we saved 67% of the patient sample size. Moreover, inclusion in the study could be stopped shortly after one year from the inclusion of the first patient. This constituted a substantial gain in time since the inclusions were initially planned over a three year period. This makes the test not only economical but also in line with ethical requirements and could make a case for wider use of sequential designs in infectious disease clinical trials.

Authors' contributions

SR conceived the study, performed the statistical analysis and drafted the manuscript. SB performed the inclusion and follow up of the patients. JD participated in the design of the study and in the writing of the manuscript. PB participated in the design and coordination of the study. All authors read and approved the final manuscript.