Measuring clinical trial transparency: an empirical analysis of newly approved drugs and large pharmaceutical companies.

OBJECTIVES: To define a series of clinical trial transparency measures and apply them to large pharmaceutical and biotechnology companies and their 2014 FDA-approved drugs.
DESIGN: Cross-sectional descriptive analysis of all clinical trials supporting 2014 Food and Drugs Administration (FDA)-approved new drug applications (NDAs) for novel drugs sponsored by large companies. DATA SOURCES: Data from over 45 sources, including Drugs@FDA.gov, ClinicalTrials.gov, corporate and international registries; PubMed, Google Scholar, EMBASE, corporate press releases, Securities and Exchange Commission (SEC) filings and personal communications with drug manufacturers. OUTCOME MEASURES: Trial registration, results reporting, clinical study report (CSR) synopsis sharing, biomedical journal publication, and FDA Amendments Acts (FDAAA) compliance, analysed on the drug level.
RESULTS: The FDA approved 19 novel new drugs, sponsored by 11 large companies, involving 553 trials, in 2014. We analysed 505 relevant trials. Per drug, a median of 100% (IQR 86%-100%) of trials in patients were registered, 71% (IQR 57%-100%) reported results or shared a CSR synopsis, 80% (70%-100%) were published and 96% (80%-100%) were publicly available in some form by 13 months after FDA approval. Disclosure rates were lower at FDA approval (65%) and improved significantly by 6 months post FDA approval. Per drug, a median of 100% (IQR 75%-100%) of FDAAA-applicable trials were compliant. Half of reviewed drugs had publicly disclosed results for all trials in patients in our sample. One trial was uniquely registered in a corporate registry, and not ClinicalTrials.gov; 0 trials were uniquely registered in international registries.
CONCLUSIONS: Among large pharmaceutical companies and new drugs, clinical trial transparency is high based on several standards, although opportunities for improvement remain. Transparency is markedly higher for trials in patients than among all trials supporting drug approval, including trials in healthy volunteers. Ongoing efforts to publicly track companies' transparency records and recognise exemplary companies may encourage further progress. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

This analysis uniquely evaluates new drugs and large biotechnology and pharmaceutical companies on several different clinical trial transparency standards on the ethics and legal levels—not merely the usual crude measure of whether companies have reported results for the trials they registered on ClinicalTrials.gov. This analysis is limited to drugs approved by the Food and Drugs Administration (FDA) in 2014 for which the new drug application (NDA) holder was a large company. Subsequent and previous publications evaluate more drugs and trial sponsors.

Our review is global; we searched over 39 public trial registries, including international, corporate and patient registries, and we provide an assessment on the completeness of ClinicalTrials.gov and the value of allocating resources to link existing trial databases. A further innovation is rigorously validating study findings and data with NDA holders (pharmaceutical companies).

We created a company ranking to accompany the drug rankings on clinical trial transparency performance. We also analysed three different samples of trials: trials conducted in patients, all trials (including those involving healthy volunteers), and Food and Drug Administration Amendments Act (FDAAA) applicable trials.

This study captures static snapshots of clinical trial transparency for new drugs approved by the FDA in 2014, at the time of FDA approval, 3 months post FDA approval, 6 months post FDA approval and 13 months post FDA approval. Subsequent trial disclosures are not captured.

We do not count abstracts submitted to scientific conferences as ‘publications’ and we included only those trials related to the indication(s) for which the new drug was initially approved, in our legal compliance assessments.

Introduction

Despite its importance in supporting evidence-based patient care, prescription guideline development, formulary decisions, public trust in research and healthcare innovation, transparency around clinical trial results does not always occur.1–7 Despite the promulgation of multiple legal and guidance standards in the 20 years since Congress enacted the first law requiring registration of certain trials,8 transparency practices remain highly variable across research sponsors like universities and drug companies.4 7 9 10 Moreover, studies that measure the transparency of trials, drugs, drug manufactures and research sponsors often use markedly different transparency measures and standards, yielding different findings and progress reports. A research sponsor, trial or drug may look transparent in one study and opaque in another.

Efforts to establish clinical trial transparency standards and improve practices span decades. In 1997, the USA adopted the Food and Drug Administration Modernization Act8 requiring the registration of drug trials for serious or life-threatening conditions. In 2007, the Food and Drug Administration Amendments Act (FDAAA) expanded disclosure requirements to include trials for all types of health conditions and required that select trial results be publicly reported for FDA-regulated products.11 In 2008, the World Medical Association identified trial registration and reporting as an ethical obligation, for all trials, in the Declaration of Helsinki.12 A bevy of other leading scientific and development organisations have also endeavoured to improve research transparency, including the Gates Foundation, WHO and Wellcome Trust.13–20 Most recently, the US Department of Health and Human Services and National Institutes of Health (NIH) released policies and rules to further expand and clarify the types of trials for which reporting is required.21–23 The new NIH policy requires, for the first time, results reporting for all NIH-sponsored trials, including phase I trials conducted in healthy volunteers.24

These efforts have helped foster a culture of transparency in research, but they have also introduced ambiguous, and at times conflicting, standards. To help harmonise standards going forward, assess the current state of transparency, as well as to identify and reform areas where further improvement is needed, we defined a series of concrete transparency measures and applied them to drugs approved by the FDA in 2014 and their large company sponsors.

Our transparency standards include both measures of compliance with US legal requirements—that is, whether companies report what FDAAA requires them to—and two expanded standards applying to broader ranges of trials, one of which includes phase I trials in healthy volunteers, and the other that only looks at trials in patients. This paper is a continuation of an initiative, called The Good Pharma Scorecard, that began with benchmarking the transparency of drugs approved by the FDA in 2012, sponsored by large drug companies, on a significantly narrower set of measures.25–27

This analysis is innovative by uniquely evaluating new drugs and their sponsors on several different standards and measures—not merely the usual crude measure of whether companies have reported results for the trials they registered on ClinicalTrials.gov. Our review is global; we searched 39 public trial registries, including international, corporate and patient registries, and provide an assessment on the completeness of ClinicalTrials.gov. A further innovation is rigorously validating all study measures and data with NDA holders (pharmaceutical companies). Lastly, we added a company ranking to accompany the drug rankings.

Methods

Data

We used data from multiple (over 44) sources (details in the online supplementary appendix 1), including Drugs@FDA.gov, a publicly accessible database containing records of FDA regulatory decisions; 39 trial registries including ClinicalTrials.gov, the Aggregate Analysis of ClinicalTrials.gov database (AACT), individual corporate registries, the WHO’s International Clinical Trials Registry Platform, which aggregates 16 country registries, and the Clinical Study Data Request Repository; journals indexed in PubMed, Google Scholar and EMBASE; and corporate press releases. Additional information was obtained through personal communications with drug manufacturers. All databases were accessed several times in January–August 2016.

Sampled drugs and companies

We examined clinical trials relating to new molecular entities (NMEs), and new combination drugs containing at least one NME component, that were approved by the FDA in 2014.28 We confined our analysis to new drug applications (NDA) that were sponsored by the 20 largest biotechnology and pharmaceutical companies, or their subsidiaries, as measured by 2014 market capitalisation ranking.29 We began with large companies to highlight practices among those with the most resources available to deploy toward satisfying transparency standards that are more comprehensive than those currently imposed by law. Subsidiaries and parent companies were identified by searching corporate websites, press releases and SEC filings and via communications with companies during the data validation process. Future versions of our scorecard will expand the analysis to include all trial sponsors, including small-sized and medium-sized companies and public sponsors.

Sampled clinical trials

A list of every clinical trial included in the NDA was created by reviewing the 2014 FDA approval packages for each of the 19 drugs. Basic characteristics of each trial were extracted (details in the online supplementary appendix 2). We excluded trials terminated without enrolment, expanded access trials, observational studies and trials that were ongoing or not at least 1 year past their primary completion date (PCD) by our study cut-off date of 1 February 2016. PCD was defined as on ClinicalTrials.gov. Observational studies (which constituted 5 of the 553 studies we reviewed) were generally excluded because they were ongoing at the time of our study cut-off date. Additionally, they are not covered under FDAAA legal requirements to report trial results.

From this ‘all-trials’ sample, we then selected two subsamples. The ‘trials-in-patients’ subsample was confined to trials conducted in patients (as opposed to healthy volunteers). The ‘FDAAA trials’ subsample was limited to trials subject to mandatory registration and results reporting under FDAAA. Because our analyses examine both legal compliance with FDAAA and satisfaction of a more aspirational standard, different samples of trials were required.

The legal requirements under FDAAA for reporting trial results apply only to ‘controlled clinical investigations(s), other than a phase I clinical investigation’30 of a drug that is the subject of an approved NDA or for which an NDA would be required in order for the drug to be legally marketed in the USA. The requirements apply only if the trial began after 27 September 2007 or was ongoing as of 26 December 2007. Finally, the trial must meet one of the following conditions: (1) at least one US site, (2) conducted under an FDA investigational new drug application, or (3) involve a drug, biologic or device manufactured in USA or its territories, and exported for research.11 30

Data collection methods

Search terms to match trials in the registries included the trial’s organisational identification number, product name, chemical name(s), number of participants and other characteristics captured from the FDA approval packages. We abstracted all available characteristics on each trial from the registry (details in the online supplementary appendix 2). We matched trials to journal articles using a minimum of three trial characteristics, and searched the registries for links to publications.

For each trial, data were retrieved by at least two research assistants who received training and worked independently (details in the online supplementary appendix 3). Discrepancies between coders’ findings were resolved by consensus. Final datasets and findings were sent to drug companies for validation (details in the online supplementary appendix 4). Feedback from companies was generally incorporated into findings if it could be validated through our public sources. For example, companies in some cases provided a web link to a publication missed in our matching process.

The study did not undergo Institutional Review Board (IRB) review because it was not human subjects research.

Outcome measures

Our first outcome measure examined whether the trials in our samples were registered in any public registry, including corporate and international registries. Second, we determined whether either trial results or a clinical study report (CSR) synopsis were provided in the registry. Third, we determined whether each trial was published in a journal indexed by PubMed, Google Scholar or EMBASE. Fourth, we deemed each trial ‘publicly available’ if it had results reported in a registry, a CSR synopsis provided in a registry or results published in a journal.

Results were considered available if received by a registry or published by 1 February 2016. This date was chosen to provide a generous period of time for reporting results: at least 1 year after FDA approval of the drug plus a 1-month grace period.

We also measured the availability of trial results at the time each drug was approved by the FDA, 3 months after approval and 6 months after approval to track reporting timelines. Additionally, we tracked where trials were registered, reported or had shared CSRs to get a sense of the overall use of corporate, national and international registries and the need to invest in linking multiple databases together.

We applied these measures to two different samples of trials (table 1): (1) all trials, including trials enrolling healthy volunteers and patients, and (2) only trials enrolling patients. The all-trials analysis evaluates companies against the World Medical Association12 recommendation that all trials be registered and have results disclosed to honour ethical obligations to research subjects, both healthy volunteers and patient participants, as described in the Declaration of Helsinki and Belmont Report. The Belmont Report defines research as, ‘an activity designed… to develop or contribute to generalizable knowledge’, which generally requires the dissemination of research results. In contrast, companies generally only commit to disclosing results for trials enrolling patients, not healthy volunteers, as expressed in their trade association codes of conduct. Therefore, we also applied our transparency standards to those trials that enrolled patients in the intent-to-treat population. That analysis excluded patients with renal or hepatic impairment (who did not have the condition being studied) and trials in healthy volunteers.

Table 1

Transparency measures and analysed clinical trial samples

Transparency measure	Samples analysed (from successful NDAs)
	All trials (including those in healthy volunteers)	Trials in patients	Trials subject to FDAAA*
Registered in a public registry by 13 months post FDA approval	X	X
Either trial results or a CSR synopsis provided in a public registry by 13 months post FDA approval	X	X
Published in a journal indexed in PubMed, Google Scholar or EMBASE by 13 months post FDA approval	X	X
Results publicly available in some form (results or CSR synopsis in registry, or journal article) by 13 months post FDA approval	X	X
Compliant with FDAAA—‘trial completion date’ interpretation			X
Compliant with FDAAA—‘approval date’ interpretation			X

*Trials that FDAAA defines as being covered by its results reporting requirements.

CSR, clinical study report; FDA, Food and Drug Administration; FDAAA, Food and Drug Administration Amendments Act; NDA, new drug application.

We also assessed the extent to which trials subject to FDAAA met that statute’s transparency requirements. Setting cut-off points for this analysis was complex because there is a disagreement among companies about what the statute requires. There is broad agreement that FDAAA requires trial registration within 21 days after enrolling the first participant (we gave sponsors a 7-day grace period to account for delayed postings, weekends, holidays and time zones). However, two views exist about when results must be reported, both of which are plausible. One interpretation, which we call the ‘trial completion date’ view, is that results generally must be reported within 12 months after a trial’s PCD, but may be delayed until 30 days after FDA approval if a company files a ‘certificate of delay’ with the NIH.8 The other is that trial results are not due until 30 days after FDA approval of a new drug for an initial use approval. The Final Rule, a regulation released in September 2016 by the NIH and HHS, clarifies that trial results must be reported for both approved and unapproved indications, however, its effective date postdates the trials in our sample.21 We examined compliance with FDAAA among applicable trials using both interpretations and used the ‘approval date’ interpretation in calculating company rankings.

Lastly, we ranked NDA sponsors by their overall clinical trial transparency. For pharmaceutical companies with only one drug approved by the FDA in 2014, we averaged their scores on (1) the trials-in-patients analysis, excluding trials in healthy volunteers, and (2) a FDAAA compliance standard that counted a trial as compliant if it satisfied either interpretation of the reporting requirements. For companies with multiple drugs approved, we pooled the trials from all drugs and then calculated the percentages of trials satisfying each of the two standards. In rare cases, we excluded trials (n=12) from a particular company ranking if the NDA sponsor was not the trial sponsor and therefore not responsible for publicly reporting trial results. If the responsible party was a ranked company, and we could confirm they were indeed the responsible party, we transferred the trials to their denominator and included them in their rankings.

Statistical analysis

Summary statistics (medians and interquartile ranges) were calculated to show how commonly trials for each approved drug and drug company met the transparency measures. All data were collected and analysed in Microsoft Excel V.15.18 (Redmond, Washington, USA).

Validation

Datasets and results were sent to the NDA holders of each drug for validation (details in the online supplementary appendix 4). Nine of 11 evaluated companies participated in both the validation process and a meeting to discuss study methods and findings (details in the online supplementary appendix 5), affording validation for 79% of drugs reviewed. Validated results are presented below.

Results

In 2014, the FDA approved 31 new molecular entities (NMEs) or new combination drugs with at least 1 NME, 19 of which were sponsored by 11 of the 20 largest pharmaceutical or biotechnology companies (figure 1). A total of 553 trials (median of 24 trials per drug) were included in the NDAs.

Figure 1

Drugs included in transparency analysis. BLA, biologic license application; FDA, Food and Drug Administration; NDA, new drug application; NME, new molecular entities.

We analysed 505 of these trials (median of 22 trials per drug), after excluding trials that were not at least 1 year past their PCD by our study cut-off of 1 February 2016, trials terminated without enrolment and expanded-access trials. Trials with unknown phases (n=7) were excluded from the FDAAA subsample. A median of seven trials per drug were conducted in patients. Trials in patients accounted for 233 of 505 trials, but 93% of all trial participants (124 664/133 428).

Transparency scores based on trials in patients

We first report results for the sample of trials in patients. A median of 100% (IQR 86%–100%) of trials in patients per drug were registered (table 2). A median of 71% (IQR 57%–100%) reported results or provided a CSR synopsis, and 80% (IQR 70%–100%) were published. Overall, results for a median of 96% (IQR 80%–100%) of trials, per drug, were publicly available in some form.

Table 2

Transparency of clinical trials in patients and all clinical trials listed in NDAs

			Ethics standard A: trials in patients (excluding healthy volunteer trials)					Ethics standard B: all clinical trials
Drug	NDA sponsor	Short indication*	# Trials analysed	% Registered	% Reported	% Published	% Publicly available‡	# Trials analysed	% Registered	% Reported	% Published	% Publicly available
Xtoro	Alcon/Novartis†	Otitis externa	3	67	67	0	67	4	50	50	0	50
Esbriet	Genentech/Roche †	Pulmonary fibrosis	10	46	40	70	70	17	33	24	47	47
Harvoni	Gilead	Hepatitis C	31	92	61	68	74	60	53	32	37	40
Dalvance	Durata/Allergan	Skin infection	8	50	25	75	75	22	18	9	68	68
Zydelig	Gilead	Leukaemia	5	100	60	60	80	16	58	19	19	25
Sivextro	Cubist/Merck†	Skin infection	7	100	57	71	86	21	90	19	67	71
Zontivity	MerckSharpDohme	Thrombotic Cardiovascular events	7	86	71	71	86	27	26	19	74	78
Xigduo XR	AstraZeneca†	Type 2 diabetes	20	100	70	80	90	25	85	56	64	72
Farxiga	AstraZeneca†	Type 2 diabetes	39	91	49	85	92	58	76	34	78	84
Lynparza	AstraZeneca	Ovarian cancer	24	96	92	75	96	24	96	92	75	96
Zykadia	Novartis	Lung cancer	1	100	100	100	100	5	69	20	20	20
Belsomra	MerckSharpDohme	Insomnia	6	100	100	100	100	37	24	22	27	32
Cerdelga	Genzyme/Sanofi	Gaucher disease	3	100	100	100	100	16	41	19	38	38
Viekira Pak	Abbvie	Hepatitis C	15	100	87	100	100	59	35	22	58	58
Otezla	Celgene	Arthritis; psoriasis	15	93	93	80	100	30	57	47	53	63
Jublia	Dow/Valeant	Onychomycosis	5	60	0	100	100	9	33	0	78	78
Movantik	AstraZeneca	Constipation	6	100	100	67	100	20	85	85	60	85
Invokamet	Janssen/J&J†	Type 2 diabetes	22	100	100	100	100	40	88	63	85	88
Zerbaxa	Cubist/Merck	Urinary and abdominal infections	6	100	100	100	100	15	50	40	93	93
Quartile 1			5	86	57	70	80	16	33	19	37	40
Quartile 3			20	100	100	100	100	37	85	50	75	84
Median			7	100	71	80	96	22	53	24	60	68

*Full FDA approved indications are listed in online supplementary appendix 6.

†Some trials for these drugs were sponsored by a company other than the NDA sponsor, including the below.

Xtoro: Alcon sponsored three trials; two had publicly disclosed results, one trial was not registered. Novartis states that the unregistered trial ‘was conducted before the Novartis position to register all phase I trials in patients became applicable to Alcon. Alcon was purchased by Novartis in April 2011’.

Esbriet: Intermune sponsored two trials; both had publicly disclosed results. Marnac sponsored four trials; two were publicly disclosed. Two trials were investigator initiated and were both publicly available. Genentech merged with Intermune in 2014. Intermune purchased the rights to sell Esbriet in the USA in 2007.

Sivextro: Bayer/Trius sponsored 21 trials, 15 had public results. Bayer and Trius partnered to develop Sivextro (2011). Cubist acquired Trius (2013). Merck acquired Cubist (2014).

Xigduo XR: BMS cosponsored 16 trials with AstraZeneca; 14 had publicly disclosed results.

‡Ranking of drugs.

Farxiga: BMS cosponsored multiple trials with AstraZeneca.

Invokamet: Mitsubishi Tanabe sponsored four trials; all had publicly disclosed results. Misubishi was Johnson & Johnson’s research partner in Japan.

BMS, Bristol Myers Squib; NDA, new drug application.

Ten of 19 drugs (53%) had at least one undisclosed trial conducted in patients. Six drugs (32%) had at least one undisclosed phase II or III trial. At least 2864 patients participated in trials with undisclosed results.

Transparency scores based on all trials in an NDA, including healthy volunteers

Transparency was lower using the standard that all trials in a successful NDA should be publicly available, than it was for the trials-in-patients standard (table 2). A median of 53% (IQR 33%–85%) of all trials per drug were registered, 24% (IQR 19%–50%) reported results or shared a CSR synopsis and 60% (IQR 37%–75%) were published. Overall, a median of 68% (IQR 40%–84%) of trial results per drug were publicly available (reported, shared in a CSR or published).

All 19 drugs had at least one publicly unavailable trial conducted in patients or healthy volunteers. Most of these trials were phase I trials involving healthy volunteers (median of 100% of trials, IQR 72%–100%). At least 7287 patients and healthy volunteers participated in trials with undisclosed results.

Compliance with FDAAA requirements

A median of 25% (IQR 19%–38%) of trials per drug (or four trials per drug) were subject to mandatory disclosures under FDAAA. Applying first the ‘trial completion date’ interpretation of FDAAA’s requirements, a median of 71% (IQR 0%–87%) of these trials per drug were FDAAA compliant (table 3). A median of 100% (IQR 100%–100%) of these trials per drug were registered on time and 71% (IQR 0%–87%) reported results on time. Of the 110 426 participants in trials covered by FDAAA, 66% were in non-compliant trials under this interpretation.

Table 3

Compliance with FDAAA legal disclosure requirements

			‘Trial completion date’ interpretation					‘FDA approval date’ interpretation
Drug	NDA sponsor	Indication	# Trials analysed	% Registered	% Reported	% FDAAA compliant	# Trials analysed	% Registered	% Reported	% FDAAA compliant
Jublia	Dow/Valeant	Onychomycosis	2	100	0	0	2	100	0	0
Dalvance	Durata/Allergan	Skin infection	2	50	0	0	2	50	100	50
Farxiga	AstraZeneca*	Type 2 diabetes	17	100	12	12	17	100	59	59
Harvoni	Gilead	Hepatitis C	27	100	59	59	27	100	70	70
Sivextro	Cubist/Merck*	Skin infection	4	100	75	75	4	100	75	75
Zydelig	Gilead	Leukaemia	4	100	75	75	4	100	75	75
Xigduo XR	Astrazeneca*	Type 2 diabetes	13	100	23	23	13	100	77	77
Otezla	Celgene	Psoriasis	11	100	91	91	11	100	91	91
Viekira Pak	Abbvie	Hepatitis C	13	100	77	77	13	100	92	92
Belsomra	MerckSharpDohme	Insomnia	4	100	0	0	4	100	100	100
Zerbaxa	Cubist/Merck	Urinary and abdominal infections	6	100	0	0	6	100	100	100
Zontivity	MerckSharpDohme	Thrombotic cardiovascular events	3	100	0	0	3	100	100	100
Movantik	AstraZeneca	Constipation	6	100	17	17	6	100	100	100
Lynparza	AstraZeneca	Ovarian cancer	7	100	71	71	7	100	100	100
Esbriet	Genentech/Roche*	Pulmonary fibrosis	4	100	75	75	4	100	100	100
Invokamet	Janssen/J&J*	Type 2 diabetes	15	100	87	87	15	100	100	100
Cerdelga	Genzyme/Sanofi	Gaucher disease	3	100	100	100	3	100	100	100
Xtoro	Alcon/Novartis*	Otitis externa	2	100	100	100	2	100	100	100
Zykadia	Novartis	Lung cancer	1	100	100	100	1	100	100	100
Quartile 1			3	100	0	0	3	100	75	75
Quartile 3			13	100	87	87	13	100	100	100
Median			4	100	71	71	4	100	100	100

Full FDA-approved indications are listed in the online supplementary appendix 6.

*Some trials for this drug were sponsored by a company other than the New Drug Application (NDA) sponsor, including the below.

Farxiga: AZ reports that this drug approved by the FDA on 1 January 2014 and acquired from BMS on 2 January 14. AZ states that ‘the transition of Farxiga studies to AstraZeneca is still in progress. Results were to be posted before the transition to AstraZeneca, which did not occur within the timeline for newly approved products’. AZ has now posted results for the seven FDAAA-applicable studies that did not meet FDAAA requirements. Six out of seven were published in the medical literature on or before 30 days post FDA approval; the other was published after the 30-day cut-off.

Sivextro: Bayer/Trius sponsored four trials, three out of four were FDAAA compliant under the ‘approval date’ interpretation.

Xigduo XR: BMS cosponsored 10 trials with AstraZeneca, 9/10 were FDAAA compliant under the ‘approval date’ interpretation.

Esbriet: Intermune sponsored two trials, both were FDAAA compliant under the ‘approval date’ interpretation.

Invokamet: Mitsubishi Tanabe sponsored two trials, one out of two was FDAAA compliant under the ‘approval date’ interpretation.

Xtoro: Alcon sponsored two trials, both were FDAAA compliant under the ‘approval date’ interpretation.

AZ, AstraZeneca; BMS, Bristol Myers Squib; FDA, Food and Drug Administration; FDAAA, Food and Drug Administration Amendments Act; NDA, New Drug Application

Applying the ‘approval date’ interpretation of the law, a median of 100% (IQR 75%–100%) of trials per drug were FDAAA compliant (table 3). A median of 100% (IQR 100%–100%) were registered on time and 100% (IQR 75%–100%) had results reported on time. A median of 100% (IQR 88%–100%) of participants were in compliant trials.

Most companies (73%) filed at least one certificate of delay for each of their drugs. However, a median of only 28% of FDAAA-applicable trials per drug had certificates of delay filed.

Timing of results reporting

At the time of FDA approval, results for a median of 45% (IQR 20%–52%) of all trials per drug are publicly available in some form (table 4). At 3 months postapproval, 50% (IQR 28%–67%) of all trials were publicly available. This median did not increase at 6 months postapproval, remaining at 50% (IQR 33%–73%).

Table 4

Timing of public availability of trial results for new drugs approved by the FDA in 2014

Trials in patients (excluding healthy volunteer trials) in NDA							All Clinical Trials in NDA
Drug	At FDA approval		3 months postapproval		6 months postapproval		At FDA approval		3 months post-approval		6 months post-approval
	% Publicly available trials	% Participant data available	% Publicly available trials	% Participant data available	% Publicly available trials	% Participant data available	% Publicly available trials	% Participant data available	% Publicly available trials	% Participant data available	% Publicly available trials	% Participant data available
Zydelig	60	65	60	65	60	65	19	48	19	48	19	48
Harvoni	45	64	52	66	61	73	25	60	28	61	33	68
Lynparza	50	46	58	69	67	73	50	46	58	69	67	73
Dalvance	63	76	63	76	63	76	50	71	50	71	50	71
Invokamet	82	87	95	89	95	89	45	83	65	87	73	88
Esbriet	70	92	70	92	70	92	47	79	47	79	47	79
Xigduo XR	65	85	70	93	70	93	52	84	56	92	56	92
Farxiga	74	66	85	95	85	95	71	66	78	94	78	94
Xtoro	67	97	67	97	67	97	50	95	50	95	50	95
Otezla	73	71	93	98	93	98	43	66	60	92	60	92
Sivextro	71	88	71	88	86	99	57	79	67	81	71	90
Zontivity	71	99	86	100	86	100	63	98	74	99	74	99
Belsomra	33	33	100	100	100	100	8	27	24	81	24	81
Cerdelga	33	12	100	100	100	100	25	24	38	56	38	56
Jublia	80	98	80	98	100	100	67	93	67	93	78	95
Movantik	50	57	100	100	100	100	20	48	45	84	45	84
Viekira Pak	60	83	87	88	100	100	15	61	22	65	29	75
Zerbaxa	17	5	100	100	100	100	27	11	93	99	93	99
Zykadia	100	100	100	100	100	100	20	81	20	81	20	81
Quartile 1	50	57	67	88	67	89	20	48	28	69	33	73
Quartile 3	73	92	100	100	100	100	52	83	67	93	73	94
Median	65	76	85	95	86	98	45	66	50	81	50	84

FDA, Food and Drug Administration; NDA, new drug application.

For trials in patients, transparency was achieved more quickly. Results for a median of 65% (IQR 50%–73%) of trials were publicly available at the time of FDA approval. At 3 months postapproval, the median was 85% (IQR 67%–100%), and 86% (IQR 67%–100%) at 6 months.

Location of registration, reporting and CSRs

Almost all registered trials (315 of 316) were registered in ClinicalTrials.gov. No trials were uniquely registered or had summary results reported in international registries. Only one trial was uniquely registered in a corporate registry and not in ClinicalTrials.gov, no summary results were uniquely posted in corporate registries. While ClinicalTrials.gov contained occasional links to CSRs, it was not comprehensive; 41 CSRs were uniquely posted on corporate registries.

Company rankings

Sanofi/Genzyme and Johnson & Johnson/Janssen achieved the highest overall clinical trial transparency scores, tying for first place in the rankings and scoring 100% on the patient and FDAAA trial standards (table 5). AbbVie (96%), Celgene (95%), Merck (93%) and AstraZeneca also scored at or above the industry median. Valeant scored lowest (50%).

Table 5

Companies’ overall clinical trial transparency rankings for drugs approved in 2014*

Rank	Company	Transparency score (%)
#1	Johnson & Johnson/Janssen	100
#1	Sanofi/Genzyme	100
#3	Abbvie	96
#4	Celgene	95
#5	Merck	93
#6	AstraZeneca	91
#7	Roche	90
#8	Novartis	88
#9	Gilead	73
#10	Allergan	63
#11	Valeant	50
Median		91

*Based on the average of companies’ scores for (1) the trials-in-patients analysis and (2) FDAAA compliance measure that counted a trial as compliant if it satisfied either interpretation of the reporting requirements.

FDAAA, Food and Drug Administration Amendments Act.

Discussion

This analysis of all known sources of publicly available information about clinical trials found high levels of transparency among large pharmaceutical companies and newly approved drugs for trials conducted in patients. Per drug, a median of 96% of trials in patients were publicly available in some form, 100% were registered, 71% reported results or shared CSRs and 80% were published, by 13 months after FDA approval. It takes about a year after FDA approval for companies to publicise most (96%) of their trials in patients. At FDA approval, 65% of trials per drug are available and at 3 months after FDA approval, 80%. Compliance with FDAAA requirements was high (median of 100% per drug).

The gap between transparency of results from the all-trials and trials-in-patients samples is striking. The median proportion of trials available per drug was markedly lower in the all-trials sample for trial registration (53% vs 100%), reporting results or CSR summaries (24% vs 71%), publication (60% vs 80%) and overall availability (68% vs 96%). The insomnia drug, Belsomra, is an extreme example of the difference: only 6 of 37 trials supporting the NDA were in patients, and the percentage publicly available was 100% for trials in patients but 20% among all trials (table 2).

Our earlier work examined transparency levels on these measures for trials in a successful NDA for drugs approved in 2012.25 Juxtaposing the two studies’ findings shows improved transparency levels on some standards. The median proportion of trials conducted in patients with publicly available results went up, from 87% in 2012 to 96% in 2014. Additionally, the proportion of drugs with undisclosed phase II or III trials went down, 50% vs 32%. However, there was little change in transparency levels on the all trials metrics (including trials in healthy volunteers). The median proportion of all trials registered per drug was 57% in 2012 vs 53% in 2014; the median for reporting results, 20% vs 24%; the median for publication, 56% vs 60%; and the median for overall availability, 65% vs 68%. The lack of increase in the proportions of trials registered and with results reported, on the all trials level, is surprising because our 2014 methodology newly incorporated more registries and included CSR synopses as satisfying the requirement for posting results. Compliance with FDAAA, measured by the ‘trial completion date’ interpretation, increased from a median of 67% of covered trials per drug in 2012 to 71% in 2014.

There is disagreement about the value of disclosing information for trials in healthy volunteers. Some pharmaceutical companies, pharmaceutical trade associations15 and commentators31 32 have focused on trials in patients for several reasons. Historically, later-phase efficacy trials have been considered to have the greatest public health relevance and salience for prescription guidelines writers and drug formulary committees. Limiting disclosure to trials in patients captures phase I trials for serious diseases, like most cancers, where the relevance of early data to patient care is high. Arguably, any important safety signals that emerge in phase I trials in healthy volunteers resurface in phase II trials, so critical safety information likely does reach the public. Lastly, trials in healthy volunteers are small, seldom controlled and therefore not generally powered to provide statistical significance.

Notwithstanding these arguments, there is value in making information about all trials available. NIH policy now requires it. Many companies already operate on this standard. It would alleviate public concerns about whether useful information is being hidden, and speed decision making based on safety signals. Additionally, disclosing phase I trials may help speed innovation and save money, particularly for small biotechnology companies, by preventing others from travelling down known dead-end pathways or empowering them to design better trials based on the lessons learnt from previous studies.

Whether transparency analyses focus on all trials or trials in patients, there is a need for clearer, more harmonised standards so that progress over time can be gauged and companies receive a consistent message about what is important for them to do. At a minimum, FDA should clarify which of the two interpretations of FDAAA was correct. The Final Rule is helpful in this regard in that it newly requires the reporting of trial results for both approved and unapproved indications.

More broadly, we believe the measures articulated in this study are useful, broadly acceptable and demonstrably workable to implement. In ongoing work, we will supplement them with a measure for patient-level data sharing—the new frontier in clinical trial transparency.

Our study has limitations. First, our decision not to count abstracts submitted to scientific conferences as ‘publications’ may be controversial. Second, some studies may be published after our study cut-off of 13 months post-FDA approval of a drug. Third, we attributed transparency scores, on the drug level, generally to the company that submitted a drug’s NDA although a few trials in the NDA were sponsored by other companies—typically, a company the NDA sponsor acquired (see the online supplementary appendix 7). NDA sponsors presumably had access to data from those trials in order to file the NDA that included them; however, one company reported being unable to edit the ClinicalTrials.gov entry created by the acquired company. This limitation only applied to the drug evaluations, not the company rankings, as these trials were often excluded from the company rankings. Also, it is worth noting that some trials in an NDA are for different indications than the approved indication. The FDA generally evaluates these trials as safety trials for the approved indication. Our analysis was limited to large companies, to drugs approved by the FDA in 2014 and to trials included in the relevant NDAs approved in 2014. Finally, our company rankings are not adjusted for the volume of trials conducted. Some may object that this disadvantages companies with a large number of trials, for whom compliance may be more resource intensive.

Conclusion

Our study shows that clinical trial transparency practices vary according to the standard used to measure them. On the drug level, about half of FDA-approved drugs have publicly disclosed results for all trials in patients who were included in our sample. On the company level, about 18% of large companies fully disclosed all such results and complied with FDAAA disclosure requirements. Per drug, among trials in patients, a median of 100% of trials were registered and 96% had publicly available trial results, in some form. Among large pharmaceutical companies, clinical trial transparency is highl based on many measures, although opportunities for improvement remain.

Momentum for greater clinical trial transparency will grow as we continue to experience its benefits. Legal requirements in FDAAA and NIH rules push the effort forward, along with efforts by other organisations like the Gates Foundation, WHO and Wellcome Trust, but reaching consensus on standards and monitoring and publicising companies’ adherence to emerging standards are also critical. Celebrating progress—and identifying where it is not occurring as quickly as it could—can move the field forward toward a shared vision of transparency and what it can achieve.