Clinical trial transparency: a reassessment of industry compliance with clinical trial registration and reporting requirements in the United States.

OBJECTIVE: To evaluate the accuracy of a 2015 cross-sectional analysis published in the BMJ Open which reported that pharmaceutical industry compliance with clinical trial registration and results reporting requirements under US law was suboptimal and varied widely among companies.
DESIGN: We performed a reassessment of the data reported in Miller et al to evaluate whether statutory compliance analyses and conclusions were valid. DATA SOURCES: Information from the Dryad Digital Repository, ClinicalTrials.gov, Drugs@FDA and direct communications with sponsors. MAIN OUTCOME MEASURES: Compliance with the clinical trial registration and results reporting requirements under the Food and Drug Administration Amendments Act (FDAAA).
RESULTS: Industry compliance with FDAAA disclosure requirements was notably higher than reported by Miller et al. Among trials subject to FDAAA, Miller et al reported that, per drug, a median of 67% (middle 50% range: 0%-100%) of trials fully complied with registration and results reporting requirements. On reanalysis of the data, we found that a median of 100% (middle 50% range: 93%-100%) of clinical trials for a particular drug fully complied with the law. When looking at overall compliance at the trial level, our reassessment yields 94% timely registration and 90% timely results reporting among the 49 eligible trials, and an overall FDAAA compliance rate of 86%.
CONCLUSIONS: The claim by Miller et al that industry compliance is below legal standards is based on an analysis that relies on an incomplete dataset and an interpretation of FDAAA that requires disclosure of study results for drugs that have not yet been approved for any indication. On reanalysis using a different interpretation of FDAAA that focuses on whether results were disclosed within 30 days of drug approval, we found that industry compliance with US statutory disclosure requirements for the 15 reviewed drugs was consistently high. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

This study uniquely analysed compliance with clinical trial disclosure laws in the USA based on expert legal advice regarding the requirements of US law.

This study relies on a more complete dataset than Miller et al. We were able to fill gaps in the datasets used by Miller et al through direct contacts with responsible company personnel overseeing clinical trial registration and reporting activities.

This study is based on a review of the raw data available on the Dryad Digital Repository.

In addition to analysing transparency of clinical trial information across drugs, as done by Miller et al, we also analysed transparency on the trial level, which is consistent with other studies assessing compliance with clinical trial transparency requirements.

We limited our reanalysis to the 15 drugs and 69 clinical investigations identified in Miller et al’s FDAAA compliance assessment.

Although Miller et al assessed transparency according to their interpretation of both legal requirements and ethics standards, we limited our reanalysis to compliance with legal requirements.

Introduction

Meaningful clinical trial transparency can advance medical science, help ensure the integrity of the clinical research enterprise, inform medical decision-making and respect the rights and dignity of research participants.1 2 Compliance with statutory and industry-driven transparency measures will enhance transparency, and published compliance reviews should be accurate, balanced and rigorous.

Numerous stakeholders have played a role in encouraging increased clinical trial transparency, including the International Committee of Medical Journal Editors,3 the WHO,4 the World Medical Association5 and the biopharmaceutical industry itself. The biopharmaceutical industry’s commitment to the ‘timely communication of meaningful results of controlled clinical trials of marketed products or investigational products that are approved for marketing, regardless of outcome’,6 is longstanding. Most recently, in July 2013, the Pharmaceutical Research and Manufacturers of America, a trade organisation representing the innovative pharmaceutical industry, joined with the European Federation of Pharmaceutical Industries and Associations in adopting joint Principles for Responsible Clinical Trial Data Sharing.7 Although the biopharmaceutical industry recognises the need to place some limits on transparency to protect patient privacy, respect the integrity of national regulatory systems and maintain incentives for continued investment in biopharmaceutical research, the Principles reflect the biopharmaceutical sector’s support for responsible data sharing that recognises the importance of these competing societal interests.

In addition to the efforts of stakeholders to increase clinical trial transparency, policy-makers across the globe have sought to require increased disclosure of clinical trial information through state and national laws. In 2007, the US Congress enacted the Food and Drug Administration Amendments Act (FDAAA) to enhance the transparency of clinical trials.8 FDAAA requires sponsors to register ongoing clinical trials and to report the results of completed clinical trials within certain time frames on ClinicalTrials.gov, a publicly accessible government database maintained by the National Library of Medicine (NLM).9 FDAAA’s disclosure requirements apply only to certain types of trials for which data are likely to be more interpretable and clinically relevant. Accordingly, disclosure is required for clinical trials that meet the following criteria:

The trial is controlled 8;

The trial is not a phase I trial 8;

The trial has a US nexus, such as a US trial site, a US Investigational New Drug (IND) application or a US manufacturing site8 10 and

The trial was initiated after 27 September 2007 or was ongoing as of 26 December 2007.8

Moreover, Congress did not initially require the reporting of results for clinical trials of unapproved drugs, although it gave the National Institutes of Health (NIH) authority to do so later through the issuance of final regulations.8

FDAAA imposes specific deadlines on the submission of clinical trial information. If a clinical trial is subject to FDAAA, it must be registered within 21 days after the first patient is enrolled.8 Likewise, results information must be submitted to ClinicalTrials.gov no later than 1 year after the ‘completion date’ of the trial,8 commonly referred to as the ‘primary completion date’ (PCD).11 This 1 year deadline, however, does not apply if the drug is not approved by the PCD because, as noted above, FDAAA does not require submission of results information for unapproved drugs.

FDAAA is complex, and after its passage ‘a spectrum of interpretations’ emerged, particularly with respect to the deadlines for results reporting and the necessity of ‘certificates of delay’ (CODs).1 FDAAA contains two provisions that allow a sponsor to extend an otherwise applicable deadline for results reporting by submitting a COD. First, if a sponsor submits a COD for a trial of a drug that has never been approved for any use, the deadline for submitting results information may be delayed until 30 days after the drug is initially approved by the Food and Drug Administration (FDA).8 Second, if a sponsor submits a COD for a trial studying a new use of a previously-approved drug, the deadline for submitting results information may be delayed until 30 days after approval of the new use, up to a maximum of 2 years from the date of submission of the COD.8

The COD provisions, however, are ambiguous in some cases. For example, is a COD required if a clinical trial is completed before initial approval of an investigational drug? If the drug is not approved for any indication, there arguably is no 1 year deadline to post trial results because FDAAA does not require results posting for trials of unapproved drugs. And if there is no deadline, there arguably is no need to submit a COD to ‘delay’ that deadline. In light of this ambiguity, some companies took the position that a COD was not required for trials of unapproved drugs, but they nevertheless complied with the spirit of FDAAA by submitting results to ClinicalTrials.gov within 30 days after initial approval of the drug.

On 21 September 2016, NIH issued final regulations clarifying many of these legal ambiguities.12 First, as authorised by Congress, the NIH final rule for the first time required companies to submit results information for trials of unapproved drugs. Second, the deadline for such reporting was established as 1 year after PCD. Third, the final rule clarified that this deadline could be extended only if the sponsor submitted a COD, in which case the deadline was 30 days after approval (up to a maximum of 2 years after submission of the COD).

Importantly, these new requirements only apply to clinical trials completed after 18 January 2017.12 For trials completed prior to 18 January 2017, NIH explained that results reporting is not required at all if the PCD occurs before initial approval of the investigational drug.12 NIH reasoned that such trials are considered to be trials of an unapproved drug (even if the drug is later approved), and FDAAA does not require results reporting for trials of unapproved drugs. This also means that CODs are not required for such trials. Table 1 describes the disclosure rules as they existed prior to the new NIH final regulation.

Table 1

FDAAA disclosure requirements

FDAAA disclosure requirements prior to 2017*
Trial type/status	Requirement†
Registration on ClinicalTrials.gov
Controlled trial‡	21 days after first patient enrolment
Non-controlled interventional trial	N/A
Results reporting on ClinicalTrials.gov
Controlled trial of an unapproved drug (ie, before initial approval)	No requirement to post results
Non-controlled, interventional trial of an unapproved drug	N/A
Controlled trial of an approved drug (ie, new use)	1 year after PCD of the trial, unless a COD is submitted, in which case deadline is 30 days after approval of new use (up to a maximum of 2 years from COD submission)
Non-controlled, interventional trial of an approved drug	N/A
Filing a COD
Controlled trial of an unapproved drug (ie, initial approval)	No explicit requirement to file a COD. As stated above, there is no requirement to post results for clinical trials of unapproved drugs, thus filing a COD is unnecessary
Non-controlled interventional trial of an unapproved drug	N/A
Controlled trial of an approved drug (ie, new use)	COD can be filed to extend the otherwise applicable ‘1 year after PCD’ deadline (per above)
Non-controlled, interventional trial of an approved drug	N/A

*The information in this table refers specifically to the statute. The requirements listed do not include the expanded requirements described in the final rule that was published on 21 September 2016 and became effective on 18 January 2017. The trials in this analysis are not subject to the requirements under the final rule.

†These requirements apply to trials completed after 27 September 2007, or were ongoing as of 26 December 2007. Phase I trials and trials without a US nexus are exempt from the disclosure requirements.

‡The statute does not define the term ‘controlled’ but a controlled trial is typically viewed as a trial that includes a control arm (eg, placebo, no treatment and active comparator) or a historical control.

COD, certificate of delay; FDAAA, Food and Drug Administration Amendments Act; PCD, primary completion date.

Since the enactment of FDAAA, numerous analyses have been conducted to assess compliance with its disclosure requirements.13 One of the most recent, a cross-sectional analysis published by Miller et al in the BMJ Open, examined whether clinical trials for drugs approved by FDA in 2012 were registered and had results reported in compliance with FDAAA legal requirements.1 Miller et al reported that pharmaceutical industry compliance with clinical trial registration and reporting requirements was suboptimal and varied widely among companies. The authors also proposed an annual transparency scorecard that audits and ranks all new medicines and vaccines with respect to transparency. The Miller et al analysis was conducted before the issuance of the NIH final rule. Because it is important for this type of ongoing compliance review to be accurate, balanced and rigorous, we performed a reassessment of the methodology used and data reported in Miller et al to evaluate whether the data were accurate and whether and how the results would change using an interpretation of FDAAA that focused on whether results information was publicly disclosed within 30 days of approval without regard to the filing of a COD.

Methods

Data sources

We used the data made available by Miller et al on the Dryad Digital Repository,14 a curated resource that provides access to data underlying scientific publications. We also used information available from ClinicalTrials.gov9 (described above) and from Drugs@FDA, a publicly accessible database maintained by the FDA that contains records regarding drug and biological product approvals.15 The databases were accessed several times between March and August 2016. We also conducted telephone interviews between May and July 2016 with the biopharmaceutical companies identified in Miller et al as sponsors of the clinical trials subject to the original analysis. These interviews were conducted to confirm the accuracy of the data collected from the above sources and to obtain relevant data that were missing from the datasets compiled by Miller et al.

Study samples

Drugs

We restricted our reanalysis to clinical trials of the 15 drugs identified by Miller et al. These were novel drugs approved in 2012 whose clinical trials were sponsored by large biopharmaceutical companies.

Clinical trials subject to FDAAA

We restricted our reanalysis to the clinical trials identified in Miller et al as subject to FDAAA—specifically the 69 trials that comprise table 2 of Miller et al. We were able to identify these trials from the datasets available on the Dryad Digital Repository.

Table 2

Comparison of Miller et al analysis and our reassessment of controlled trials, by drug

Drug	Company	Miller et al				Re-assessment
		Trials subject to FDAAA	Timely registration (%)	Timely reporting (%)	FDAAA compliance (%)	Trials subject to FDAAA	Timely registration (%)	Timely reporting (%)	FDAAA compliance (%)
Elelyso	*Protalix	1	100	0	0	1	100	100	100
Stivarga	Bayer	1	100	0	0	1	100	100	100
Perjeta	Genentech/Roche	2	50	0	0	2	50	0	0
Signifor	Novartis	1	100	0	0	1	100	100	100
Erivedge	Genentech/Roche	2	100	0	0	2	100	100	100
Zioptan	†Merck/Santen	‡6	17	17	17	1	100	100	100
Eliquis	BMS	6	83	33	33	6	83	67	50
Aubagio	Sanofi	7	86	71	§57	7	86	100	86
Zaltrap	Sanofi	6	100	67	67	6	100	83	83
Inlyta	Pfizer	2	100	100	100	2	100	100	100
Stribild	Gilead	3	100	100	100	3	100	100	100
Xeljanz	Pfizer	11	100	100	100	11	100	100	100
Bosulif	Pfizer	1	100	100	100	¶2	100	100	100
MenHibrix	GSK	3	100	100	100	3	100	100	100
Sirturo	Janssen (J&J)	1	100	100	100	1	100	100	100
Median		2	100	67	**57	2	100	100	100
Mid-50% range		(1–6)	(93–100)	(0–100)	(0–100)	(1–4.5)	(100–100)	(100–100)	(93–100)
Overall %			85	62	58		94	90	86

*Protalix was the sponsor on ClinicalTrials.gov for all trials included in this analysis for Elelyso, and all ClinicalTrials.gov-related disclosure activities were the responsibility of Protalix. Pfizer had no responsibility for trial registration or results reporting for Elelyso.

† Miller et al included five controlled trials in their FDAAA analysis despite having collected no data regarding those trials’ completion dates. In our reassessment of the data we found that those five trials were completed prior to the enactment of FDAAA (27 September 2007) and are therefore out of the scope of the analysis. These trials were all sponsored and conducted by Santen rather than Merck.

‡ The 17%t figures were in the Miller et al paper, which suggests six controlled trials instead of the reported seven. This is also consistent with the raw data.

§ The 57% figure is consistent with the raw data in the Miller et al paper, owing to a trial having results reported on time but not registered on time. The reported figure in Miller et al was 71%.

¶ We coded an additional clinical trial as ‘controlled’ because, although it was terminated prior to the controlled portion of the trial, it was originally designed as a controlled clinical investigation. Miller et al coded this trial as non-controlled ‘interventional’.

** The 57% figure is consistent with the raw data in the Miller et al paper, owing to the FDAAA compliance rate for Aubagio being corrected from 71% to 57%. With this change, the median now becomes 57% instead of 67%.

FDAAA, Food and Drug Administration Amendments Act.

We reanalysed these datasets to ensure they included only clinical trials subject to mandatory registration and results reporting requirements under FDAAA. If a clinical trial did not meet one or more of the FDAAA requirements described above, we excluded it from our sample pool; however, we did not exclude any trial unless there was clear evidence that it did not meet one or more FDAAA requirements.

Miller et al created two sample pools of trials subject to FDAAA, one for controlled trials (53) and one for interventional trials (69) (Miller et al incorrectly indicated in their table 2 that there were 54 controlled trials). The interventional sample pool contained the 53 controlled trials plus 16 additional non-controlled trials. As noted above, FDAAA, by its express terms, only applies to ‘controlled’ clinical trials.8 Accordingly, for our primary analysis, we limited the sample pool to the 49 trials subject to FDAAA that were ‘controlled’ (we excluded several trials that were not subject to FDAAA). We nevertheless performed a secondary reanalysis of all 69 interventional trials. After excluding six trials as not subject to FDAAA (for the reasons described below), our interventional sample pool consisted of 63 trials—the 49 trials from our controlled sample pool (since these were also interventional) plus an additional 14 non-controlled, interventional trials.

Main outcome measures

We reviewed whether the trials in our controlled and interventional sample pools met applicable FDAAA deadlines for registration and results reporting. Since FDAAA does not require results reporting for trials of drugs that are not yet approved, we did not apply the ‘1 year after PCD’ deadline—or the related COD requirement—to a clinical trial if that deadline occurred before initial approval of the drug product. In such cases, we instead determined whether the sponsor submitted trial results to ClinicalTrials.gov for public disclosure within 30 days of initial approval. We also applied a 3-day grace period for registration and results reporting, which is consistent with the 3-day grace period applied by Miller et al.14

As in Miller et al., ‘If a trial met FDAAA requirements for both registration and disclosure of results, it was counted as compliant with legal requirements’,1 meaning that each trial could only receive a score of either 0% or 100%.

Validation

Our final datasets for each drug were sent to each party responsible for posting the registration and results information for a given trial (ie, the New Drug Application (NDA) or Biologics License Application (BLA) holder, or the IND sponsor in cases where this was not the same entity as the NDA or BLA holder) for an applicable drug or biological product to verify the accuracy and completeness of our extracted information. We subsequently scheduled teleconferences to discuss and verify the datasets. Where possible, data and input received from companies (response rate was 100%) were verified by public data sources. Where no public data sources were available, we accepted the input from the companies as accurate (eg, Zioptan trial completion dates).

Results

We compare our reanalysis to that of Miller et al for controlled trials in table 2. Among controlled trials, Miller et al reported that a median of 100% (middle 50% range: 93%–100%) of trials for a particular drug met registration requirements, while a median of 67% (middle 50% range: 0%–100%) met results reporting requirements—clinical trial transparency for 6 of the 15 drugs considered were reported to be fully compliant. When we reassessed the data, we found that a median of 100% of trials for a particular drug met both registration and results reporting requirements (middle 50% range: 100%–100% for each). In addition, clinical trials for 11 of the 15 drugs considered were found to be 100% compliant with FDAAA disclosure requirements, and 4 of the 5 drugs reported as 0% compliant in Miller et al were found to be 100% compliant on reanalysis of the data. Compliance scores for 8 of the nine drugs Miller et al reported as having a compliance score of <100% increased on reanalysis. One drug out of the 15 reviewed was found to have a compliance score of 0%. We report the first and third quartiles above, referring to them as the ‘middle 50% range’. Miller et al term this measure the ‘IQR’.

As stated above, we also performed a secondary reassessment of all interventional trials (table 3). Miller et al found that a median of 100% (middle 50% range: 93%–100%) of trials for a particular drug met registration requirements, while 71% (middle 50% range: 0%–100%) met results reporting requirements—5 of the 15 drugs considered were found to be fully compliant. In our reanalysis, we found that a median of 100% of trials for a particular drug met both registration and reporting requirements (middle 50% range: 100%–100% for registration, 95%–100% for results reporting), with 10 of the 15 drugs found to be fully compliant.

Table 3

Comparison of Miller et al analysis and our reassessment of interventional trials, by drug

Drug	Company	Miller et al				Reassessment
		Trials subject to FDAAA	Timely registration (%)	Timely reporting (%)	FDAAA compliance (%)	Trials subject to FDAAA	Timely registration (%)	Timely reporting (%)	FDAAA compliance (%)
Elelyso	*Protalix	3	100	0	0	3	100	33	33
Stivarga	Bayer	2	100	0	0	2	100	100	100
Perjeta	Genentech/Roche	2	50	0	0	2	50	0	0
Signifor	Novartis	2	100	0	0	2	100	100	100
Erivedge	Genentech/Roche	3	100	0	0	3	100	100	100
Zioptan	†Merck/Santen	7	29	14	14	1	100	100	100
Eliquis	BMS	6	83	33	33	6	83	67	50
Aubagio	Sanofi	7	86	71	‡57	7	86	100	86
Zaltrap	Sanofi	9	100	78	78	9	100	89	89
Inlyta	Pfizer	7	100	86	86	7	100	100	100
Stribild	Gilead	3	100	100	100	3	100	100	100
Xeljanz	Pfizer	11	100	100	100	11	100	100	100
Bosulif	Pfizer	2	100	100	100	2	100	100	100
MenHibrix	GSK	3	100	100	100	3	100	100	100
Sirturo	Janssen (J&J)	2	100	100	100	2	100	100	100
Median		3	100	71	§57	3	100	100	100
Mid-50% range		(2–7)	(93–100)	(0–100)	(0–100)	(2–6.5)	(100–100)	(95–100)	(88–100)
Overall %			88	61	58		95	89	86

*Protalix was the sponsor on ClinicalTrials.gov for all trials included in this analysis for Elelyso, and all ClinicalTrials.gov-related disclosure activities were the responsibility of Protalix. Pfizer had no responsibility for trial registration or results reporting for Elelyso.

†Miller et al included six interventional trials in their FDAAA analysis despite having collected no data regarding those trials’ completion dates or US nexus. In our reassessment we found that five of these trials were completed prior to the enactment of FDAAA (27 September 2007) and one trial did not have a US nexus. All six trials therefore are out of the scope of the analysis. All six trials also were sponsored and conducted by Santen rather than Merck.

‡The 57% figure is consistent with the raw data in the Miller et al paper, owing to a trial having results reported on time but not registered on time. The reported figure was 71%.

§The 57% figure is consistent with the raw data in the Miller et al paper, owing to the FDAAA compliance rate for Aubagio being corrected from 71% to 57%. With this change, the median now becomes 57% instead of 71%.

FDAAA, Food and Drug Administration Amendments Act.

In addition to the median compliance percentage across drugs used by Miller et al, we also consider a supplemental metric, the overall compliance rate, computed by dividing the total number of compliant trials by the total number of trials considered. This calculation assigns equal weight to all trials, such that a trial for a drug with few trials does not influence results more than a trial for a drug with many trials. When considering overall compliance (ie, across trials as opposed to across drugs), an analysis based on the raw data tables of Miller et al concludes that of the 53 controlled trials considered, 85% were registered on time and 62% had results reported on time. However, our reanalysis yields 94% timely registration and 90% timely results reporting among the 49 eligible trials, and an overall FDAAA compliance rate of 86%. We found a similar pattern when applying this across trial analysis to the interventional trials. Of the 69 interventional trials that Miller et al considered, 88% were registered on time and 61% had results reported on time based on an analysis of Miller et al’s raw data tables. This contrasts with our findings of an overall timely registration rate of 95% and a timely results reporting rate of 89% among the 63 trials considered in our reanalysis.

We note that the results reported above include some corrected data from Miller et al. For example, Miller et al report a median of 67% FDAAA compliance under ‘FDAAA definition 1: “controlled” trials’ in their table 2. However, the raw data used in the Miller et al paper are consistent with a median FDAAA compliance of 57% rather than 67% due to corrections to the reported Aubagio data. In addition, Miller et al report a median of 71% FDAAA compliance under ‘FDAAA definition 2: “interventional” trials’ in their table 2. However, the raw data used in the Miller et al paper is consistent with a median FDAAA compliance of 57% rather than 71% (again, because of corrections to the reported Aubagio data).

Discussion

Our re-analysis yielded significantly different results than Miller et al. In contrast to Miller et al, we found that industry compliance with legal transparency requirements is consistently high, with a 100% compliance rate for 11 of the 15 drugs considered in our reanalysis, and an overall compliance rate across trials of 86%.

The discrepancies between the original analysis and our reanalysis are mainly due to two issues. First, Miller et al relied on incomplete datasets. As a result, the authors included multiple trials in their analysis that are not subject to FDAAA (we excluded these trials). Second, Miller et al used an interpretation of FDAAA that assigned greater importance to the filing of a COD than the disclosure of clinical trial results on ClinicalTrials.gov and did not consider the approval status of the drugs for which the trials were conducted. As a result, the authors coded multiple trials as untimely where results were publicly disclosed within 30 days of initial FDA approval of the drug (we coded these trials as timely).

Incomplete datasets

With respect to the first issue, the authors included six Zioptan trials in their compliance analysis that are not subject to FDAAA. The authors did not collect or record information about the trial completion dates and US nexus of these trials, as evidenced by a review of the authors’ Zioptan dataset available on the Dryad Digital Repository.14 This information, however, is critical to any compliance analysis because it determines whether or not a clinical trial is subject to FDAAA. In the absence of this critical data, the authors nevertheless treated all six Zioptan trials as subject to FDAAA. We obtained this missing information directly from the original trial sponsor (Santen), who informed us that five trials were completed prior to 26 December 2007 (the FDAAA cut-off date) and the sixth trial was conducted entirely in Russia with no US nexus. Based on this additional information, we determined that all six trials should have been excluded from the analysis because they are not subject to FDAAA.

Interpretation of FDAAA requirements that focuses on COD submission

With respect to the second issue, the authors adopted an interpretation of FDAAA that coded many trials as non-compliant even though they were submitted to ClinicalTrials.gov for public disclosure within 30 days of initial approval of the relevant drug. The authors did this in cases where the trial sponsor did not submit an ‘initial approval’ COD. In other words, Miller et al treated the applicable deadline for these trials as if it were 1 year after PCD even if that date occurred before the drug was approved. We believe there are several problems with this approach.

First, the authors’ methodology does not acknowledge the fact that FDAAA does not require the disclosure of clinical trial results if a drug has not yet been approved.16 For such trials, it is problematic to apply a deadline that occurs prior to product approval, such as the ‘1 year after PCD’ deadline. This is best illustrated by one of the Signifor trials, Study B2202. In that case, the authors coded the trial as non-compliant even though it was publicly disclosed prior to initial approval of the drug product.14 The authors apparently did this because the sponsor submitted the results more than a year after PCD without submitting a COD. We do not believe it is appropriate or accurate to describe a trial as ‘untimely’ or ‘non-compliant’ when the results were publicly reported before such disclosure was legally required.

Second, the authors’ interpretation assigns greater importance to the technical filing of a COD than to the more relevant transparency issue of whether results were publicly disclosed within the expected time frame, that is, 30 days after initial approval. We believe many people would be surprised to learn that most of the trials coded by Miller et al as untimely were, in fact, publicly disclosed within 30 days of initial approval of the drug. They were considered to be non-compliant by Miller et al simply because the sponsor did not submit a non-essential COD.

Given the existence of recognised legal ambiguities prior to 2016 regarding the need to submit a COD, we decided to focus more directly on disclosure. Unlike Miller et al, we thus coded a clinical trial as compliant with FDAAA disclosure requirements if results were publicly disclosed within 30 days of initial approval, regardless of whether a COD had been filed. We believed this approach not only was more consistent with the law but also provided a more meaningful measure of transparency.

Additionally, Miller et al coded at least one study as non-compliant even though it involved an unapproved new use of the drug, and the sponsor had submitted a ‘new use’ COD. This type of certificate extends the deadline for submission of results information up to 2 years. Miller et al coded the study as non-compliant based on the original, non-extended deadline.

The above differences between Miller et al’s analysis and our reanalysis had a significant impact on the study results, excluding or changing the compliance rating of 14 of the 20 controlled trials originally coded as non-compliant (we changed the coding of an additional trial from non-compliant to compliant because the PCD was updated after the BMJ Open article had been published). When the data are reanalysed based on a more complete dataset and a different interpretation of FDAAA, industry compliance with FDAAA disclosure requirements for ‘controlled’ clinical trial results was demonstrated to be notably higher than in the Miller et al analysis. A similar high rate of compliance was demonstrated in our secondary reanalysis of the ‘interventional’ trials.

Limitations

Several limitations deserve mention. Although Miller et al assessed transparency according to both asserted ethics standards and legal requirements, we limited our reanalysis to the latter because, unlike ethics standards, which are not settled, FDAAA legal requirements are enforceable within the United States. We also did not reassess compliance rates strictly in accordance with the new NIH regulations, which were issued after our reanalysis had been completed. As noted above, under those regulations, most of the trials reviewed by Miller et al are technically exempt from results reporting requirements altogether. Instead of applying this new interpretation, we reanalysed the data according to our original plan, which coded a clinical trial as compliant with FDAAA disclosure requirements if results were publicly disclosed within 30 days of initial approval, regardless of whether a COD had been filed. This is consistent with the new rule’s clarification that CODs are not required for such trials.

Conclusion

The biopharmaceutical industry has made great strides in the space of clinical trial data sharing; numerous companies participate in a variety of platforms to make clinical trial data more accessible to qualified researchers. Even so, the extent to which the biopharmaceutical industry is disclosing information regarding clinical trials is a topic of significant public debate. There is a widely-held concern that the industry is not being transparent and is withholding information on the outcomes of their clinical trials.

However, industry compliance with the legal requirement to publicly disclose the results of clinical trials for the 15 reviewed drugs was found to be consistently high for the drugs subject to our reanalysis. The assertion by Miller et al that industry compliance with disclosure requirements is below legal standards is based on an analysis that relies on incomplete datasets and an interpretation of FDAAA disclosure requirements that assigned greater importance to the filing of a COD than the disclosure of clinical trial results on ClinicalTrials.gov and did not consider the approval status of the drugs for which the trials were conducted. On reanalysis, we found that industry compliance with US statutory disclosure requirements for the 15 reviewed drugs was consistently high and considerably better than that reported by Miller et al. While some trials failed to report results within the 30 days, not a single trial subject to our reanalysis failed to report results: every clinical trial in this paper has results posted in the public domain.

There is great value in independent efforts to measure compliance with legal disclosure obligations; however, those efforts must be rigorous, accurate, and balanced. We firmly believe that responsibly enhancing clinical trial transparency for researchers, patients and the public will expand scientific knowledge, foster a collaborative scientific discovery process and support patient care—all with the ultimate goal of improving public health. Moreover, while much hard work remains, sponsors who are actively sharing clinical trial data and disclosing results should be acknowledged.