Discontinuation and non-publication of heart failure randomized controlled trials: a call to publish all trial results.

AIMS: Discontinuation or non-publication of trials may hinder scientific progress and violates the commitment made to research participants. We sought to identify the prevalence of discontinuation and non-publication of heart failure (HF) clinical trials. METHODS AND
RESULTS: We conducted a cross-sectional search of ClinicalTrials.gov to identify all completed and discontinued HF clinical trials. We limited our search to only include trials that were completed by 31 December 2017. Trials were investigated to identify reasons for discontinuation. Informative termination was defined as trial termination due to safety or efficacy concerns. Data pertaining to the trial phase, funding, intervention, enrolment, and trial completion date were extracted for each trial. A total of 572 trials were included. Of these, 21% (n = 118) were discontinued before completion. Patient accrual was the most frequently cited reason (n = 42; 36%) for trial discontinuation, followed by informative termination (n = 16; 14%) and funding (n = 14; 12%). Overall, 24 780 patients were enrolled in trials that were terminated. Of trials that were completed and not terminated, nearly one-third (n = 131/454; 29%) were not published. Seventy-nine (24%) trials were published within 12 months, 192 (59%) within 24 months, and 252 (78%) trials within 36 months.
CONCLUSIONS: Discontinuation and non-publication of HF trials is common. This raises ethical concerns towards participants who volunteer for research and are exposed to potential risks, inconvenience, and discomfort without furthering scientific progress.

Introduction

Randomized clinical trials (RCTs) provide the highest level of evidence to confirm treatment effects and inform clinical care. The success of conducting clinical trials depends upon the willingness of human subjects who volunteer to participate in these trials, often without any prior knowledge of the benefits and risks of unproven interventions. It is vital for investigators and sponsors to conduct trials with the highest ethical standards and publish detailed trial results, regardless of their outcome. Failure to do so can hinder scientific progress and neglects the commitment from trial participants who expose themselves to potential risks, inconvenience, and discomfort for the sake of generating medical evidence. , , Moreover, failure to publish and disseminate trial results leaves other patients and investigators with incomplete information that can lead to similar or duplicative studies being carried out or similar trial mistakes being made due to general unawareness of earlier results. This is especially true for drug trials that are stopped due to safety reasons as drugs of the same or similar class may pose the same safety concerns in future.

The World Health Organization (WHO) recently mandated that the main findings of all clinical trials must be posted to a primary clinical trial registry within 12 months of the end of the study and called for reporting of older unpublished trials. Similarly, the Food and Drug Administration Amendments Act of 2007 (FDAAA) also requires the public sharing of clinical trial results of FDA regulated drugs and devices to the ClinicalTrials.gov databank. However, despite the moral and ethical responsibility to publish studies, premature discontinuation of trials with no safety or futility concerns and non‐publication of clinical trials remain prevalent across a range of specialties. , , , , , Evidence suggests that trials with positive outcomes are published more frequently as compared with those with neutral outcomes. , This may be of particular significance in trials that are slower to enrol, for example, heart failure (HF). We therefore aimed to investigate the discontinuation and non‐publication of HF trials and subsequently determine the frequency of available results for unpublished trials. We also highlight possible strategies to enhance timely publication of trial results.

Methods

Data source and search strategy

ClinicalTrials.gov is the most comprehensive publicly available clinical trial registry. Investigators are expected to continuously update information as the trials progress in order to have the most up‐to‐date data available as public record. A cross‐sectional search of trials for HF registered in the ClinicalTrials.gov database was conducted. The initial search was carried out in January 2020 by two independent investigators (N. A. and I. S.). We used the ‘advance search’ option to search for all randomized HF trials that were completed or had been discontinued since the inception of the database. Because the estimated median time to publication is 20 months, the search was limited to only include trials that were completed by 31 December 2017. Trials were included if the recruitment status was listed as ‘completed’ or ‘active, not recruiting’ to ensure that no additional participants would be enrolled in the study. Studies with and without available results were included. Trials that were registered ≥30 days after the initial start date were excluded to avoid any possible bias by the investigator to only register the study due to positive study results.

Data regarding trial phase, funding source, intervention, enrolment, and trial completion date were extracted from ClinicalTrials.gov for each study. Trials were grouped from phase I to phase IV. Trials described as phase I/II in their registry were categorized as phase II, while trials described as phase II/III in their registry were categorized as phase III. Funding source was further divided into industry funded, National Institute of Health (NIH), non‐NIH US federal funding, or other (non‐US funding, individuals, universities, or organizations), while trial intervention was further categorized as drug, behavioural, device, or procedural intervention.

For trials that were discontinued, reasons for trial discontinuation were analysed based on data provided at ClinicalTrials.gov. These included funding issue, loss of principal investigator, company/business decision, informative termination (defined as trial termination due to safety concerns, emerging efficacy data or short‐term results indicating efficacy, or futility), slow enrolment, trial conduct problems (defined as technical or logistical issues contributing to lack of trial completion), or unreported/unclear reasons.

Publication search

ClinicalTrials.gov uses a unique NCT identifier number that automatically links to relevant PubMed publication. If a publication containing trial results was not listed, we searched Medline using NCT number, trial title, author names, study intervention, number of participants, study keywords, and funding sources to identify any existing trial publication. Publications search was carried out by two independent investigators (N. A. and I. S.) in January 2020. For those trials for which no published results were found, we repeated the search strategy on EMBASE. We considered a trial published if trial results were available in a peer‐reviewed manuscript. A trial was considered unpublished if no relevant publication was identified in a peer‐reviewed manuscript; these were categorized into those with or without results available in the ClinicalTrials.gov. Abstracts were not considered as peer reviewed publications.

Statistical analysis

Trials were categorized as either discontinued or completed. Continuous variables were reported as median (interquartile range [IQR]), while categorical variables were expressed as No. (%). All trial characteristics were calculated as a percentage of total trials. Time taken to publish studies was calculated from the trial completion date as reported by ClinicalTrials.gov to the date of first online or in print publication (whichever occurred first); χ 2 tests were used to evaluate associations between trial characteristics and trial completion and publication status. Multivariable logistic regression models assessing factors including funding source, interventions, trial phase, and masking were used to determine their impact on trial completion and publication, with completion or publication status as the dependent variable and the control variables modelled as a set of dummy variables. Because trial publications can be late, a sensitivity analysis excluding trials completed after 2015 was also conducted to observe any meaningful change in trial statistics. All statistical analyses were performed using Statistical Package for the Social Sciences (SPSS) software version 23 (International Business Machines Corporation, New York, USA). All comparisons were two‐tailed, and a P‐value <0.05 was considered statistically significant.

Results

Literature search

The initial search of the ClinicalTrials.gov database yielded 2954 HF trials, of which 572 were included in our analyses. Main reasons for exclusion included recruitment status other than ‘completed or active, not recruiting’ (n = 1163), trial completion date after 31 December 2017 (n = 444), trial not randomized (n = 51), and trials that were registered with ClinicalTrials.gov more than 1 month after beginning enrolment (n = 724). Figure highlights the detailed literature search process.

FIGURE 1

Flowchart of the detailed literature search. Of the 572 included HF trials, 118 were discontinued. Of the 454 completed HF trials, 131 remain unpublished.

Trial characteristics

A total of 165 308 patients were enrolled in the 572 trials. The median number of participants enrolled across trials was 60 (IQR = 21–197). Nearly half (49%, n = 279) of trials studied drug interventions, while device, procedural, and behavioural interventions accounted for 23% (n = 134), 8% (n = 47) and 8% (n = 46) of trials, respectively. Forty‐three per cent (n = 208) of trials were either phase 2 or phase 3. Almost half of the trials (47%, n = 270) received industry funding (Table ). Enrolment of participants varied across trials; 258 trials (45%) enrolled less than 50 participants while 65 (11%) trials enrolled more than 500 participants, and 147 trials (26%) were completed before 2012, while 249 trials (44%) were listed as complete between the years 2015 and 2017. Twenty‐three trials (4%) were for HF with preserved ejection fraction, and 49 trials (9%) were for acute heart failure. Seventy‐four trials (13%) were double blind, while 247 trials (43%) were open label.

TABLE 1

Characteristics of discontinued randomized trials registered with ClinicalTrials.gov

Characteristics	All trials (n = 572), n (%)	Completed trials (n = 454), n (%)	Discontinued trials (n = 118), n (%)	P‐value
Funding source a				0.063
Industry	270 (47)	198 (44)	72 (61)
NIH	61 (11)	52 (11)	10 (8)
US Federal	20 (3)	18 (4)	3 (2)
Other (non‐US)	352 (62)	286 (63)	67 (56)
Intervention a
Drug	279 (49)	212 (47)	67 (56)	0.050
Behavioural	47 (8)	44 (10)	4 (3)
Device	134 (23)	98 (22)	37 (31)
Procedure	46 (8)	39 (9)	8 (6)
Other	125 (22)	102 (22)	24 (20)
Trial phase b
Phase 1	31 (5)	28 (6)	3 (3)	0.110
Phase 2	139 (24)	107 (24)	32 (27)
Phase 3	109 (19)	81 (18)	27 (23)
Phase 4	62 (11)	46 (10)	17 (14)
Not applicable	231 (40)	192 (42)	39 (33)
Masking
Open label	247 (43)	203 (45)	45 (38)	0.16
Single blind	76 (13)	57 (13)	19 (16)
Double blind	74 (13)	61 (13)	12 (10)
Triple blind	58 (10)	39 (9)	19 (16)
Quadruple blind	113 (20)	91 (20)	22 (19)
Unspecified	4 (1)	3 (1)	1 (1)
Enrolment size
Median enrolment (IQR)	60 (21–197.25)	62 (29.25–220.75)	26 (9–86)
SD no. of participants	869	839	625
<10	53 (9)	22 (5)	31 (26)	<0.01
<50	205 (36)	159 (35)	46 (39)
<100	105 (18)	94 (21)	12 (10)
<500	144 (25)	126 (28)	18 (14)
>500	65 (11)	53 (12)	11 (9)

NIH, National Institutes of Health.

Trials with multiple funding sources or interventions are listed within all relevant categories; totals therefore add to more than 100%.

Trials described as ‘phase I/II’ in their registry entry are categorized as phase II. Trials described as ‘phase II/III’ in their registry entry are categorized as phase III.

Discontinued trials

Of the 572 HF trials, 118 (21%) were discontinued before completion. Overall, 24 780 patients were enrolled in these trials at termination, and of these, 2849 (11.5%) were enrolled in trials that were discontinued based on informative reasons. There was a significant association between completion status and enrolment size of the trials (P < 0.01) (Table ). Trials based on behavioural interventions were less likely to be completed compared with drug trials (OR 0.24, 95% CI [0.07, 0.79]; P = 0.02) (Table ). On the contrary, device trials were significantly more likely to reach completion (OR 1.72, 95% CI [1.10, 2.70]; P = 0.02). Trials including triple blind masking were also associated with increased odds of reaching trial completion compared with open label masking (OR 1.88 [1.01, 3.50]; P = 0.04).

TABLE 2

Logistic regression analysis of factors associated with completion and publication of heart failure clinical trials

	Completion of trial		Publication of trial
	OR (95% CI)	P‐value	OR (95% CI)	P‐value
Funding source
Industry	Reference		Reference
NIH	1.21 (0.44, 3.3)	0.71	1.53 (0.57, 4.14)	0.40
US Federal	0.34 (0.11, 1.04)	0.06	1.57 (0.60, 4.10)	0.36
Other	0.84 (0.55, 1.23)	0.43	0.54 (0.35, 0.83)	0.01
Intervention
Drug	Reference		Reference
Behavioural	0.24 (0.07, 0.79)	0.02	0.78 (0.37, 1.67)	0.53
Device	1.72 (1.10, 2.70)	0.02	1.41 (0.85, 2.33)	0.18
Procedure	0.59 (0.26, 1.38)	0.22	1.15 (0.56, 2.38)	0.71
Other	1.18 (0.73, 1.91)	0.51	1.04 (0.62, 1.72)	0.89
Trial phase
Phase 1	Reference		Reference
Phase 2	1.25 (0.47, 3.3)	0.66	4.21 (1.76, 10.10)	60;0.01
Phase 3	1.44 (0.54, 3.88)	0.47	4.41 (1.77, 11.01)	60;0.01
Phase 4	1.45 (0.50, 4.17)	0.49	2.39 (0.91, 6.26)	0.08
Not applicable	1.18 (0.46, 3.03)	0.73	2.48 (1.11, 5.53)	0.03
Masking
Open label	Reference		Reference
Single blind	1.28 (0.71, 2.31)	0.42	0.79 (0.42, 1.51)	0.48
Double blind	0.83 (0.43, 1.61)	0.83	0.71 (0.37, 1.34)	0.29
Triple blind	1.88 (1.01, 3.50)	0.04	0.52 (0.23, 1.20)	0.13
Quadruple blind	0.91 (0.53, 1.59)	0.75	0.65 (0.37, 1.14)	0.13
Unspecified	3.57 (0.49, 25.96)	0.21	4.06 (0.36, 45.58)	0.26

OR, odds ratio.

The median number of participants enrolled in discontinued trials was 26 (IQR = 9–86) (Table ). Slow enrolment was the most frequently cited reason (n = 42;36%) for trial discontinuation, while informative termination and funding issue accounted for 14% (n = 16) and 12% (n = 14) of the trials discontinued, respectively. Unreported or unclear reasons accorded for 22% (n = 26) of the discontinued trials (Figure ). Majority of the trials discontinued were industry funded (n = 72; 61%), followed by funding by other organizations (n = 67; 56%), NIH funded trials (n = 10; 8%), and US Federal funding (n = 3; 2%). Of the 118 trials that were discontinued prematurely, 38 (32%) trials started before the year 2012, 20 (17%) between 2013 and 2014, while 18 (15%) trials commenced 2015 onwards, respectively.

FIGURE 2

The reasons for discontinuation of trials. Patient accrual was the most common cited reason for trial discontinuation.

Non‐published trials

Of the 572 trials included, 454 (79%) were completed. Of the completed trials, 131 (29%) were unpublished (Figure ). The median duration of time since trial completion was 55 months (IQR = 39–88). These 131 unpublished trials represented a total number of 18 471 participants. Thirty‐three (25%) of these trials had more than 100 patients enrolled, and 32 trials (24%) were placebo controlled. Median number of participants enrolled in these unpublished trials was 43 (IQR = 20–100) (Table ). Majority of these trials were based on drug interventions (n = 59; 45%), followed by device interventions (n = 34; 26%). A significant association was observed between trial publication status and funding source, trial phase, and enrolment size (P < 0.01). Thirty of the 131 (22.9%) unpublished trials comprising of 4887 participants posted results on ClinicalTrials.gov. None of these trials provided information on mortality during study time or adverse events. Seventy‐six (58%) unpublished trials were completed by 2016. Sensitivity analysis excluding trials completed after 2015 revealed a total of 314 completed trials, of which 24.2% (n = 76) remain unpublished. Similar trends of trial characteristics were observed, with industry funded trials (55%) and trials based on drug interventions (46%) accounting for the highest percentage of unpublished trials.

TABLE 3

Characteristics of completed randomized trials registered with ClinicalTrials.gov

Trial characteristics	All trials (n = 454), n (%)	Published trials (n = 323), n (%)	Non‐published trials (n = 131), n (%)	P‐value
Funding source a
Industry	198 (44)	125 (39)	73 (56)	<0.01
NIH	52 (11)	42 (13)	10 (8)
US Federal	18 (4)	12 (4)	6 (5)
Other	286 (63)	218 (67)	68 (52)
Intervention a
Drug	212 (47)	153 (47)	59 (45)	0.63
Behavioural	44 (10)	34 (11)	10 (8)
Device	98 (22)	64 (20)	34 (26)
Procedure	39 (9)	27 (8)	12 (9)
Other	102 (22)	73 (23)	29 (22)
Trial phase b
Phase 1	28 (6)	13 (4)	15 (11)	<0.01
Phase 2	107 (24)	84 (26)	23 (18)
Phase 3	81 (18)	64 (20)	17 (13)
Phase 4	46 (10)	31 (10)	15 (11)
Not applicable	192 (42)	131 (41)	61 (47)
Masking
Open label	203 (45)	136 (42)	67 (51)	0.28
Single blind	57 (13)	41 (13)	16 (12)
Double blind	61 (13)	45 (14)	16 (12)
Triple blind	39 (9)	31 (10)	8 (6)
Quadruple blind	91 (20)	69 (21)	22 (17)
Unspecified	3 (1)	1 (0)	2 (2)
Enrolment size
Median (IQR)	62 (29.25–220.75)	80 (33–262)	43 (20–99.5)
SD no. of participants	840	941	473
<10	27 (6)	7 (2)	15 (11)	<0.01
<;50	167 (37)	102 (32)	57 (44)
<100	86 (19)	68 (21)	26 (20)
<500	123 (27)	97 (30)	29 (22)
>500	51 (11)	49 (15)	4 (3)

NIH, National Institutes of Health.

Trials with multiple funding sources or interventions are listed within all relevant categories; totals therefore add to more than 100%.

Trials described as ‘phase I/II’ in their registry entry are categorized as phase II. Trials described as ‘phase II/III’ in their registry entry are categorized as phase III.

Published studies

The median number of participants enrolled in trials that were subsequently published was 80 (IQR = 33–262) (Table ), while the median time from study completion to publication was 559 (IQR = 351–887) days. Seventy‐nine (24%) trials were published within 12 months, 192 (59%) trials within 24 months, and 252 (78%) trials within 36 months of trial completion date. Other sources of funding including non‐US funding, individual funding, universities, or organizational funding had increased odds of trial publication compared with industry funded trials (OR 0.54, 95% CI [0.35, 0.83]; P = 0.01). Similarly, phase 2 (OR 4.21 95% CI [1.76, 10.10]; P < 0.01) and phase 3 trials (OR 4.41 95% CI [1.77, 11.01]; P < 0.01) were more likely to be published compared with phase 1 trials (Table ). Trials that were published within 24 months compared with other published trials did not have any statistically significant difference in terms of sample size and industry funding.

Discussion

Our study highlights that discontinuation and non‐publication of HF clinical trials is common (Figure ). Around 21% of trials registered on ClinicalTrials.gov were discontinued prior to completion, and over a quarter of completed trials were never published. Our results are consistent with previous studies observing high non‐publication and discontinuation rates across other medical specialties. , , These findings raise ethical concerns regarding the enrolment of human subjects and suggest that investigators and sponsors are not consistently fulfilling responsibilities to HF research participants. A total of 24 780 participants were enrolled in trials that were discontinued prior to completion, while 18 471 participants were enrolled in trials that never got published. This represents a large cohort of HF patients who subjected themselves to the potential risks, inconvenience, and discomfort of participating in a clinical trial without any contribution to the scientific literature, rendering their efforts futile.

FIGURE 3

The number of HF trials that were discontinued prematurely and have been unpublished. The most frequently cited reasons for premature discontinuation of HF trials and proposed strategies to decrease non‐publication of HF clinical trials are also highlighted. RIAT, Restoring Invisible and Abandoned Trials.

Problems with recruitment of patients were cited as the most common reason for trial discontinuation. This may be of particular concern in HF trials that are considered more complex, and achieving adequate enrolment may require high cost and time and addition of enrolment sites. This was seen in both Randomized Evaluation of Mechanical Assistance for the Treatment of Congestive Heart Failure (REMATCH) trial as well as Surgical Treatment for Ischemic Heart Failure (STICH) trial, where additional funding and global expansion in recruitment was required to meet the set enrolment targets. Moreover, racial and socio‐economic reasons might also serve as a potential barrier, whereby participants of different demographics may feel uncomfortable interacting with or trusting trial investigators or trial sponsors. These factors may impair overall trial enrolment while also contributing to underrepresentation and impaired representativeness of racial/ethnic minorities and women across HF clinical trials. , , ,

Prior analyses have shown that high non‐publication rates may be due to publication bias against studies that yield negative results. , , Considering that most of our trials involving non‐publication were based on drug or device interventions, it may be possible that drugs showing low efficacy and negative outcomes remained unpublished. This may be particularly true for HF trials where rates of trials meeting their primary endpoint have significantly decreased over time, potentially reducing the likelihood of trial results being reported or subsequently published. Moreover, non‐publication of negative outcomes is also observed majorly amongst industry funded studies as opposed to government funded trials. , Furthermore, we did not limit our search with the number of participants enrolling in the trial, with most unpublished trials having less than 50 participants (44%). Publication bias is often noted in trials with small sample size, with an increased likelihood of publication associated with an increase in sample size. This relationship between low enrolment and non‐publication may potentially further worsen enrolment difficulties, as patients may hesitate to participate in trials for fear of their efforts being worthless.

The emphasis by WHO on mandating access to results of all registered trials within 12 months is particularly welcomed to curtail information lost through unpublished trials. One way of practical implementation may be through audit, where a registry may be used to identify completed trials that remain unpublished even after a year of completion, post the date of results appearing in the registry, and derive performance metrics for individual companies, funding sources, interventions, institutes, and investigators. This information would account for greater transparency and ensure accountability. , Second, RIAT (Restoring Invisible and Abandoned Trials) initiative should be considered. This initiative calls for investigators with unpublished data to signal their intention to publish their findings within a year or else provide public access to their data allowing independent investigators the ability to publish it. Third, investigators publishing trial results on ClinicalTrials.gov website before publishing their findings in a journal should not be subjected to penalties, which can potentially result in non‐publication if a journal refuses to accept the manuscript. Outcomes from these studies should be publicly available without formal publication. Fourth, journal editors should adopt policies to encourage publication of all kinds of clinical trials, including trials that were terminated prematurely. For instance, to curb the pressing issue of non‐publication in HF, a Dead Letter Office was initiated by Journal of American College of Cardiology – Heart Failure (JACC HF) as a repository of clinical trials that had previously remained unpublished. Lastly, although termination of trials is warranted when interim analyses raise safety or efficacy concerns, it is crucial that results of these trials are also available, so that other investigators considering similar studies are aware of the risks associated with similar interventions.

While interpreting our results, there are a few limitations that should be considered. First, we only analysed trials that were listed in ClinicalTrials.gov. Second, we included the trial information as provided by the trial sponsors and investigators. Although registered trial data are verified by ClinicalTrials.gov staff for accuracy, we were unable to verify data for all trials included in our analysis. Third, we only included reasons for discontinuation as listed in ClinicalTrials.gov. In an event that there were multiple reasons for termination but only one listed, we could only record the one reason that was reported. Fourth, although a substantial amount of lag time was given, we cannot exclude the possibility of very late publication of trial results. For instance, the Dead Letter Office of JACC HF has published results of trials that were completed more than a decade ago. Nevertheless, timely publication is important, and such delayed publication is still inconsistent with the investigator responsibility to trial participants. Finally, there were missing data for some variables, including reason for trial discontinuation.

In conclusion, we observed that HF clinical trials are frequently discontinued and may remain unpublished. This raises ethical concerns towards trial participants and the potential risks, inconvenience, and discomfort these individuals expose themselves to, while contributing nothing to the medical literature. Results should explicitly be available for all registered trials, including the ones terminated early or due to safety or toxicity reasons to ensure complete transparency in clinical trial reporting. The publication bias associated with these clinical trials needs to be mitigated, and additional policies involving greater accountability for those responsible for unpublished trials should be implemented to protect those participating in HF trials from having their efforts result in dissemination of no incremental scientific knowledge.

Conflict of interest

S.J.G. is supported by the National Heart Lung and Blood Institute T32 postdoctoral training grant (T32HL069749‐14) and a Heart Failure Society of America/Emergency Medicine Foundation Acute Heart Failure Young Investigator Award funded by Novartis, has received research support from Amgen, Bristol‐Myers Squibb, and Novartis, and serves on an advisory board for Amgen. M.M. declares that he has received research grant support from Amgen and consulting fees from Amgen, Bayer, Fresenius, Novartis, Servier, Vifor, and Windtree Therapeutics. S.D.A. has received research support from Vifor International 38; Abbott Vascular, and fees for consultancy and/or speaking from Astra‐Zeneca, Bayer, Boehringer Ingelheim, Respicardia, Impulse Dynamics, Janssen, Novartis, Servier, and Vifor International. G.F. participated in committees of trials and registries sponsored by BI, Bayer, Medtronic, Servier, Novartis, Vifor and Agmen. G.C.F. reports consulting for Abbott, Amgen, Bayer, Janssen, Medtronic, and Novartis. J.B. is a consultant for Abbott, Amgen, Applied Therapeutics, Astra Zeneca, Bayer, Boehringer Ingelheim, Bristol Myers Squib, CVRx, Janssen, LivaNova, Luitpold, Medtronic, Merck, Novartis, Relypsa, and Vifor. All other authors report no disclosures.