A re-examination of the SPYRAL HTN-OFF MED Pivotal trial with respect to the underlying model assumptions.

BACKGROUND: The SPYRAL HTN-OFF MED Pivotal trial demonstrated that RDN was efficacious compared to a sham control. The underlying model was an extension of the analysis of covariance (ANCOVA) model, adjusted for baseline blood pressure (BP), and allowed borrowing of information from the previously reported feasibility study using a novel Bayesian method. Fundamental to the estimation of a treatment effect for efficacy are a multitude of statistical modelling assumptions, including the role of outliers, linearity of the association between baseline BP and outcome, and parallelism of the treatment effect difference over the baseline BP range. In this report, we examine the validity of these assumptions to verify the robustness of the treatment effect measured.
METHODS: We examined the requisite modelling assumptions of the ANCOVA model fitted to the SPYRAL HTN-OFF MED Pivotal trial using Bayesian methods. To address outliers, we fit a robust regression model (with heavy tailed errors) to the data with diffuse weakly informative prior distributions on the parameters. To address linearity, we replaced the linear baseline term by a natural spline term with 4 degrees of freedom. To address parallelism, we refit the ANCOVA model with an interaction term for treatment arm and baseline BP.
RESULTS: ANCOVA models were fitted to the trial data (pooled across the feasibility and pivotal cohorts) using Bayesian methodology with diffuse (non-informative) prior distributions. The modelling assumptions inherent to the ANCOVA models were shown to be broadly satisfied. A robust ANCOVA model yielded a posterior treatment effect of -4.1 mmHg (95% credible interval: -6.3 to -1.9) indicating the influence of outlier values was small. There was moderate evidence of an interaction term effect between baseline BP and treatment, but no evidence of gross violation of linearity in baseline BP.
CONCLUSION: The posterior treatment effect estimate is shown to be robust to underlying model assumptions, thus further supporting the evidence of RDN to be an efficacious treatment for resistant hypertension.

Introduction

The SPYRAL HTN-OFF MED Pivotal trial demonstrated a reduction in blood pressure after renal denervation (RDN), and employed a novel Bayesian methodology whereby an informative prior distribution was constructed from the pilot study, and a stochastic comparison approach was used to down-weight the influence depending on the similarity with the pivotal data [1]. The results were consistent with prespecified analyses whereby a classical analysis of covariance (ANCOVA) model was fitted to the pooled (pilot + pivotal) cohorts that was supported by the similarity of the data between the cohorts. Irrespective of the approach taken to combine the pilot and pivotal cohorts in the Bayesian framework, the fundamental underlying model remains consistent: the ANCOVA model [2]. It is of interest to inspect the underlying modelling assumptions in order to gauge the potential impact on the treatment effect.

Methods

Study design

The SPYRAL HTN-OFF MED Pivotal trial was a multicentre, international, prospective, single-blinded, randomized, sham-controlled trial conducted in accordance with the Declaration of Helsinki, and the trial protocol was approved by the Institutional Review Boards or Ethics Committee from 44 study sites in Europe, the United States, Japan and Australia. Patients provided written informed consent. Details regarding trial design and enrolment criteria have been published previously [1,3]. The primary efficacy endpoint was reduction in ambulatory systolic BP (ASBP) between RDN and sham control groups based on 24-h ambulatory BP measurement at 3-month follow-up.

The SPYRAL HTN-OFF MED Pivotal trial prespecified a Bayesian analysis that used data from the pilot study to construct an informative prior distribution, that was subsequently updated with data from the pivotal study. In particular, the prior distribution was constructed using a discount power prior methodology that allows for the prior information to be down-weighted according to the concordance between the prior and pivotal outcomes data. The discounting (power) parameters estimated for ASBP in the pivotal study were 1.00 (RDN) and 0.75 (control), with values approaching 1 indicating strong similarity between the pilot and pivotal datasets.

Bayesian ANCOVA

The SPYRAL HTN-OFF MED Pivotal trial adopted an unconventional parameterisation of the analysis of covariance (ANCOVA) model [1]. Namely, for subject , randomized to treatment (sham-control) or (RDN), the change from baseline in 3-month systolic blood pressure (SBP), , is modelled aswhere is the baseline SBP measurement for subject , is the sample mean of baseline SBP measures in the pivotal data, and denotes a random error term assumed to be normally distributed with mean zero and constant variance. Under this parameterisation, the sham-adjusted treatment effect is calculated as . This prespecified model allowed for different prior distributions (as well as different discounting) to be applied in the control and treatment arms. Since it was shown that there was a large amount of similarity in data-borrowing between the prior and pivotal data for the two arms, in the following, we revert to the classical ANCOVA regression model,

We apply Bayesian regression analyses by placing prior distributions on the model coefficients and variance term, which summarises prior knowledge. By applying Bayes’ theorem [4,5], we mathematically update the joint prior distribution by the likelihood (determined from the data and model) to derive a posterior distribution. This posterior distribution represents beliefs after having observed the data and can then be used to produce probability statements regarding the treatment effect of RDN. In particular, we can calculate the probability of treatment efficacy conditional on the data (), , and summarise the distribution.

Although the primary focus here is on the pooled data, Bayesian regression models were fitted separately to the pilot, pivotal, and pooled datasets to illustrate the commensurability of the treatment effects. In each case, flat (so-called ‘non-informative’) prior distributions were assumed for the unknown parameters, essentially allowing for the trial data to drive the posterior distribution.

Modelling assumptions

Independent of whether a Bayesian or frequentist statistical paradigm is adopted for an analysis, ANCOVA regression models depend on several modelling assumptions. We examine three key assumptions: 1) the influence of outliers; 2) the linearity assumption of SBP change with baseline SBP; and 3) the parallelism assumption. In each case, we restricted these supportive analyses to the pooled data.

Outliers

To address the outliers, we fit a robust ANCOVA regression model where the errors, , are assumed to be independently distributed with Student t-errors with an unknown degrees of freedom parameter. It is evident from simple graphical analyses that there are outliers in the SPYRAL HTN-OFF MED Pivotal trial. Student t-distributions have wider tails than normally distributed errors meaning they are less sensitive to outliers. We place default weakly informative prior distributions on the parameters, including the degrees of freedom parameter.

Linearity

The model assumes the endpoint is linear in baseline SBP, with slope . To evaluate this assumption, we replace the linear baseline SBP term, , by a natural spline term with 4 degrees of freedom (3 internal knots placed at quartiles).

Parallelism

The regression lines of outcome, , are assumed to be parallel for each treatment arm over the range of values. To evaluate this assumption, we refit the ANCOVA model by mean centering the baseline SBP, , and including an additional interaction term between treatment and [centered] baseline SBP, .

Statistical analysis

All Bayesian models are fitted using the Stan software with R (version 4.0.1; R Foundation for Statistical Computing, Vienna, Austria). The package ‘brms’ (version 2.13.0) is used to fit the robust regression model. The ‘rstanarm’ (version 2.19.3) package is used to fit all other models. Posterior distributions were sampled using Markov chain Monte Carlo methods with 4 chains (with between 10,000 and 50,000 samples per chain) and the first half of each chain discarded as burn-in. Gelman-Rubin statistics were used as a diagnostic to assess convergence. Posterior treatment effects are summarised by means and 95% credible intervals (95% CrI), the posterior probability that RDN is effective (i.e. ), and graphically summarised using kernel density plots.

Results

Primary endpoint results have been previously published, with a treatment difference of -3.9 mmHg (95% CrI: -6.2 to -1.6) favouring RDN over sham control for 3-month change in ASBP, and >0.999 posterior probability of efficacy [1]. Applying the simpler ANCOVA model (above) to the data yields a posterior treatment effect of -4.0 mmHg (95% CrI: -6.2 to -1.8), with corresponding probability of efficacy >0.999. Analyses demonstrated some influence of outliers leading to a slightly larger treatment effect relatively to standard normal error model (-4.1 mmHg, Fig. 1A). Relaxing the linearity assumption also did not influence the treatment effect (-4.0 mmHg; 95% CrI: -6.2 to -1.8), with demonstrable evidence of a linear association (Fig. 1B).

Fig. 1

A) Posterior distribution of ANCOVA baseline ASBP adjusted treatment effect for RDN (versus sham control) with normal errors and Student-t distributed errors. B). Posterior natural spline model fit (with 95% credible bands) of association between baseline ASBP and 3-month ASBP change from baseline for RDN.

The standard ANCOVA model posits that the baseline-adjusted treatment effect is constant over the range of baseline ASBP values. By including an interaction term between baseline ASBP and treatment we allow for differential treatment effects over varying baseline ASBP values (Fig. 2A and B). At the baseline ASBP sample mean (150.8 ± 7.7 mmHg; Fig. 2C) the posterior treatment effect is -4.0 mmHg (95% CrI: -6.2 to -1.8), with >0.999 posterior probability of efficacy. The interaction term coefficient was -0.2 mmHg (95% CrI: -0.5 to 0.1). Although the credible interval contains zero, corresponding with no interaction, the posterior probability of the interaction being negative was 0.86 (Fig. 2D).

Fig. 2

A: Posterior model fits (with 95% credible bands) based on an ANOCVA model with interaction term between treatment and baseline ASBP. B: Posterior treatment effect versus baseline ASBP, calculated as the vertical distance between the RDN and control lines in panel A. Black-dashed lines denote the 95% credible interval band. Red-dashed line denotes null effect; blue-dashed line denotes the treatment effect assuming no interaction term. C: Histogram of baseline 24-hr ASBP values. Red-dashed line denotes the sample mean. D: Posterior distribution of the interaction term from the ANCOVA model. The blue shaded area to the left of zero (red-dashed line) denotes the posterior probability of a negative interaction term (0.86). (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)

Discussion

Application of Bayesian methods to clinical trial design has become increasingly commonplace in both pharmaceutical and device clinical trials. As a primary analytical method, a Bayesian approach permits greater efficiency in trial design by reducing trial sample size and redundancy by enabling synthesis of accruing information. For the SPYRAL HTN-OFF MED Pivotal study, Bayesian methods were used to incorporate the previously published data from a corresponding pilot study within an adaptive design. There was substantial evidence to support the overall poolability of the data, which facilitates simpler modelling approaches. Another advantage of the Bayesian paradigm is the ability to report posterior probabilities of the underlying hypothesis of interest, as opposed to frequentist P values that are routinely misunderstood.

Modelling assumptions are inherent to analysis approaches employed in the SPYRAL HTN-OFF MED Pivotal trial, irrespective of whether one adopts a Bayesian or frequentist paradigm, or, in the case of the latter, whether one adopts the complex discount power prior methodology employed in the primary analysis or the simpler diffuse prior approach used in this report. To date there has been little attention given to these assumptions. Herein, we investigated these assumptions to ensure the primary reported results were robust to these assumptions, including outliers, linearity, and parallelism. In each case reported, there was overwhelming evidence to support efficacy.

Conclusions

The SPYRAL HTN-OFF MED Pivotal trial employed a novel yet complex prespecified Bayesian methodology. The underlying model assumptions are demonstrated as being suitably satisfied, or, where slight departures were observed (as was the case with outliers), the impact on the treatment effect was shown to be very small. It is therefore conclusive that RDN is efficacious in lowering blood pressure.

Funding

The study was funded by Medtronic.

Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests.Mr. Fahy, Dr. Brar, and Ms. DeBruin are employees and shareholders of Medtronic, Dr. Hickey was a former employee of Medtronic. Prof. Pocock reports consultant fees from Medtronic outside the submitted work. Dr. Weber reports personal fees from Medtronic, Ablative Solutions, ReCor, and Boston Scientific, all outside the submitted work. Prof. Böhm reports personal fees from Amgen, Bayer, Servier, Medtronic, Boehringer Ingelheim, Vifor, Bristol Myers Squibb, and Astra Zeneca, all outside the submitted work. Prof. Mahfoud is supported by Deutsche Gesellschaft für Kardiologie (DGK), and Deutsche Forschungsgemeinschaft (SFB TRR219) and has received scientific support and speaker honoraria from Bayer, Boehringer Ingelheim, Medtronic and ReCor Medical. Dr. Kandzari reports institutional research/grant support from Medtronic and Ablative Solution; and personal consulting honoraria from Medtronic.