Compliance with requirement to report results on the EU Clinical Trials Register: cohort study and web resource.

OBJECTIVES: To ascertain compliance rates with the European Commission's requirement that all trials on the EU Clinical Trials Register (EUCTR) post results to the registry within 12 months of completion (final compliance date 21 December 2016); to identify features associated with non-compliance; to rank sponsors by compliance; and to build a tool for live ongoing audit of compliance.
DESIGN: Retrospective cohort study.
SETTING: EUCTR. PARTICIPANTS: 7274 of 11 531 trials listed as completed on EUCTR and where results could be established as due. MAIN OUTCOME MEASURE: Publication of results on EUCTR.
RESULTS: Of 7274 trials where results were due, 49.5% (95% confidence interval 48.4% to 50.7%) reported results. Trials with a commercial sponsor were substantially more likely to post results than those with a non-commercial sponsor (68.1% v 11.0%, adjusted odds ratio 23.2, 95% confidence interval 19.2 to 28.2); as were trials by a sponsor who conducted a large number of trials (77.9% v 18.4%, adjusted odds ratio 18.4, 15.3 to 22.1). More recent trials were more likely to report results (per year odds ratio 1.05, 95% confidence interval 1.03 to 1.07). Extensive evidence was found of errors, omissions, and contradictory entries in EUCTR data that prevented ascertainment of compliance for some trials.
CONCLUSIONS: Compliance with the European Commission requirement for all trials to post results on to the EUCTR within 12 months of completion has been poor, with half of all trials non-compliant. EU registry data commonly contain inconsistencies that might prevent even regulators assessing compliance. Accessible and timely information on the compliance status of each individual trial and sponsor may help to improve reporting rates. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Introduction

The results of clinical trials are used by clinicians, patients, and policy makers to make informed choices about the benefits and safety of interventions. Sharing the methods and results of all trials has therefore long been recognised as an ethical and scientific imperative.1 2 3 More recently, institutions such as the World Health Organization, European Commission, and US Food and Drug Administration have called for the disclosure of results.4 5 6 However, there is extensive and longstanding evidence that the methods and results of completed clinical trials are commonly left unreported. A 2014 systematic review identified 22 cohorts of studies included in trial registries: half were not published in a journal (54.2%, 95% confidence interval 42.0% to 65.9%), 17 that were following up cohorts of trials approved by ethics committees yielded similar rates of unpublished results, and studies with statistically significant results were more likely to be published (odds ratio 2.8, 95% confidence interval 2.2 to 3.5).7 These findings are consistent with a previous review.8

In the US, the FDA Amendments Act 2007 (FDAAA) requires sponsors to post results on to ClinicalTrials.gov itself, rather than be published in a journal, within 12 months of completion for certain categories of trial.6 The two cohort studies published to date on this topic report compliance rates of only one trial in five.9 10 FDAAA, however, has complexities and limitations. Importantly, not all trials on ClinicalTrials.gov are covered by the requirement to report results on to the register—only those meeting certain criteria; and there is no data field on ClinicalTrials.gov to easily identify the subset of trials required to report results. Furthermore, although a list of trials with certificates of exemption from reporting can be obtained and used as a proxy to help identify those trials not covered by the requirement to report results, in practice many sponsors have only requested these certificates when seeking prospective clarity from the regulator on individual trials where exemption may be contentious; therefore, many exempt trials have no such certification. In addition, the “final rule” that gives further detail on which trials are covered and sets out the process for addressing breaches, was not published until 2016; and the first few trials to be legally covered by this rule have only recently become due to report results.11 In addition, the final rule changed the number of trials covered by the act: trials on unapproved products completing before January 2017 are no longer required to post results after the product is approved.

In comparison, the European Commission is moving towards more straightforward transparency rules. Any trial of any medicinal product conducted since 2004 in an EU country has already been required to register on the European Union Clinical Trials Register (EUCTR), which is administered by the European Medicines Agency (EMA). Following the 2012 EC guideline 2012/c302/03, sponsors must ensure that all trials registered on EUCTR since 2004 disclose their results to the EMA within 12 months of trial completion; phase I trials are exempt unless they are denoted as being part of a paediatric investigation plan.12 These trial reports are posted publicly on to the EUCTR within 15 working days of receipt by the EMA and are required to include salient features such as results for all prespecified trial outcomes and statistical analyses, details of “serious” and “non-serious” adverse events, participants’ baseline characteristics, and protocol deviations, as well as discussion of design limitations and caveats.13 Following various delays in the EMA implementing the software platform for results posting, the final date for sponsors’ compliance was 21 December 2016.14 15

We assessed compliance with the EU requirement to post results on to EUCTR for all trials on the registry, explored factors associated with non-compliance, identified the individual trial sponsors that are best at complying, and created a live online service, driven by regular updates of the EUCTR data, to give ongoing and regularly updated performance statistics for compliance.

Methods

Data sources

We downloaded trial records for all trials from the EUCTR database in the week commencing 17 January 2018 using bespoke software produced for OpenTrials.net,16 an open database of publicly accessible documents and data on clinical trials. Fifteen trials were randomly selected, and to ensure the download was correctly accessing data we manually checked variables in the downloaded data against the EUCTR website.

The structure of the data on EUCTR is different to that of other registries: each trial can be conducted in multiple countries, and within each trial, each country where the trial is being conducted has a separate register entry; these are linked by a single common trial identity number, with a country code as suffix. Basic information such as trial phase, completion date, and completion status can be discrepant between countries’ entries for the same trial: while discrepancies for some data fields (such as phase) may be errors, others may reflect true differences in the conduct of a trial between countries. For the analysis we collapsed the protocol data for individual countries into a single entry for each trial based on the unique trial identifier (EudraCT Number). We extracted basic information on each trial, including trial identity number, sponsor name, and sponsor class (commercial, non-commercial). We then generated variables for several features of each trial: earliest country “global end of trial date”, latest country global end of trial date, and trial status (“ongoing”; all countries “complete” or “terminated,” some but not all countries “complete” or “terminated,” or other (suspended or no status given)). Based on the presence of a link to results in the results field in EUCTR, we also created a variable for whether a trial has reported results. No other metric of whether results have been reported on EUCTR exists, and all trials that have reported have a link in this field.

Inclusion and exclusion criteria

Our study population was all trials where results were due under the 2012 guideline. Specifically, this included all trials where all countries’ register entries are marked as complete or terminated. We excluded trials where the latest country completion date was more recent than 19 December 2016 to allow 12 months for results reporting, as per the official trial reporting requirements, and 15 additional working days for EMA to publish submitted results. We also excluded trials marked as completed in all countries but where no global date of the end of the trial was given in any country’s record; although these trials should have a completion date, they do not, and so it cannot be ascertained whether their results are due. We also excluded all phase I trials unless they were part of a paediatric investigation plan.

Explanatory variables

We created variables for a range of features of each trial, selected prospectively on the basis of clinical and methodological interest. If there were discrepancies between country protocols for the same trial on any data element, then we coded that trial as discordant between countries for the variable in question. The following variables were generated: phase (I, II, III, IV, or discordant between countries), whether any country has noted the trial as being part of a paediatric investigation plan, whether the condition being studied is designated as a rare disease, whether the trial is a bioequivalence study, whether the participants were healthy volunteers, whether the trial was terminated (specifically, where all countries records were marked as terminated), whether a trial had multiple sponsors, whether a trial was conducted in multiple countries, and whether the sponsor name was missing or unclear. We attempted to generate variables on other features such as blinding, however structured data on these features was spread between multiple fields, which were often incomplete or inconsistent between countries.

Sponsor names are entered into the EUCTR as free text, and how the same sponsor is identified often varies, such as GSK Ltd and GlaxoSmithKline Limited, or Medical University Vienna and Medizinische Universität Wien. We therefore manually normalised the data in these fields, merging records under a single name. Separately, where possible, we also created an additional sponsor name variable that accounted for acquisitions and mergers among large companies as well as university hospital systems where warranted. We generated a variable containing the number of trials the sponsor of each trial sponsored, and divided this into quarters. The top fourth of this variable therefore contains trials sponsored by organisations that sponsor a large number of trials, such as large pharmaceutical companies, whereas the bottom fourth contains trials sponsored by those who sponsor very few trials, or only one trial ever.

Analysis

We generated descriptive statistics on both the characteristics of trials in the total EUCTR cohort and the study cohort of trials where results were due. We calculated the percentage of trials reported overall, and broken down by completion year, phase, and sponsor class, whether the trial was part of a paediatric investigation plan, whether the condition studied was designated on the register as a rare disease, and whether the trial was a bioequivalence study or conducted in healthy volunteers. The exact method was used to calculate confidence intervals. We constructed a logistic regression model with all these explanatory variables, as they were selected prospectively on the basis of clinical and methodological interest. The binary response variable in our logistic regression analysis was the presence of results for the trial. Lastly, we produced ranked lists of the sponsors with the largest number of reported and unreported trials.

Live data web resource

The EUCTR data underlying this study are updated on a regular basis. We therefore commissioned a software engineer (FI) to develop an interactive online website presenting the overall reporting rate for all due trials, the reporting rates for each sponsor, ranks for these reporting rates, and details of each sponsor’s individual reported and unreported trials. The data underlying this site update regularly following each new download of the EUCTR database: the results and ranks for each individual sponsor are therefore maintained as current, and they change as performance changes. All software underlying this service is shared as open source and available for open code review or for adaptation and re-use.17

Software, data sharing, and reproducibility

All underlying software is freely available online under an open source licence for review and re-use.17 Data were extracted from the database using SQL in PostgreSQL; statistical analyses were conducted using Stata 14. The full downloaded dataset and all analytical code are shared through Github.18

Patient involvement

The development of the overall research question and outcome measures was informed by the AllTrials campaign’s extensive engagement with signatories and supporters, including patient groups. Patients were not formally involved in developing the study design.

Results

Characteristics of included trials

The EUCTR database contained 31 821 trials in total. Overall, 11 345 trials (35.7%) were conducted in more than one country (mean number of countries per trial 2.5, median 1). Three trials were excluded as their final reported completion dates were given incorrectly on EUCTR (completion years 2019 and 2041, both in the future, even though this field is supposed to be retrospectively populated with the actual completion date of the trial; and 2000, before the registry began recording trials), leaving 31 818 trials. We excluded 20 287 trials as their status was not reported as either completed or terminated in all countries. Of the 11 531 trials remaining that were listed as completed or terminated in every country, 3392 (29.4%) were excluded because they had no completion date in any countries’ record. We excluded 540 trials because their completion date was within the past 12 months and results were therefore not yet due, and 325 trials were excluded as they were phase I and not part of a paediatric investigation plan. The final cohort of studies with results due therefore comprised 7274 trials. Figure 1 shows a flow diagram for all trials on EUCTR.

Fig 1

Flow diagram for all trials on the EU Clinical Trials Register (EUCTR). PIP=paediatric investigation plan

Table 1 shows the characteristics of trials in the full EUCTR database and trials in the final cohort with results due. Trial completion dates were evenly spread over the preceding seven years, with fewer trials completing before 2008. Most trials with results due had a commercial sponsor (66.5%). Phase II trials were the most common (44.6%), followed by phase III (32.2%) and phase IV (21.7%). There were few trials on rare diseases (8.0%), bioequivalence (0.1%), or healthy volunteers (9.7%).

Table 1

Characteristics of included trials

Variables	No (%)
	Total cohort (31 818 trials)	Cohort with results due (7274 trials)
Results available	9876 (31.0)	3601 (49.5)
Completion year:
2004	4 (0.0)	3 (0.0)
2005	100 (0.3)	72 (1.0)
2006	475 (1.5)	276 (3.8)
2007	762 (2.4)	444 (6.1)
2008	1121 (3.5)	658 (9.1)
2009	1123 (3.5)	654 (9.0)
2010	1143 (3.6)	699 (9.6)
2011	1204 (3.8)	731 (10.1)
2012	1137 (3.6)	713 (9.8)
2013	1258 (4.0)	737 (10.1)
2014	1251 (3.9)	803 (11.0)
2015	1210 (3.8)	782 (10.8)
2016	1180 (3.7)	702 (9.7)
2017	1015 (3.2)	-
2018	4 (0.0)	-
Missing	18 831 (59.2)	-
Trial status:
All countries, ongoing	12 947 (40.7)	-
All countries, complete or terminated	11 531 (36.2)	7274 (100.0)
Any countries, complete or terminated	5754 (18.1)	-
Other (eg, suspended)	519 (1.6)	-
Blank	1067 (3.4)	-
Phase:
Discordant between countries	398 (1.3)	83 (1.1)
I	1569 (4.9)	23 (0.3)
II	12 191 (38.3)	3247 (44.6)
III	10 282 (32.3)	2340 (32.2)
IV	7378 (23.2)	1581 (21.7)
Sponsor class:
Non-commercial	14 408 (45.3)	2353 (32.4)
Commercial	16 964 (53.3)	4837 (66.5)
Mixed	265 (0.8)	57 (0.8)
Blank	181 (0.6)	27 (0.4)
Part of paediatric investigation plan	1042 (3.3)	107 (1.5)
Condition being studied is rare disease:
No	27 496 (86.4)	6584 (90.5)
Yes	3723 (11.7)	584 (8.0)
Discordant between countries	382 (1.2)	45 (0.6)
Data not available	217 (0.7)	61 (0.8)
Bioequivalence study:
No	31 738 (99.8)	7264 (99.9)
Yes	66 (0.2)	9 (0.1)
Discordant between countries	14 (0.0)	1 (0.0)
Participants are healthy volunteers:
No	29 132 (91.6)	6555 (90.1)
Yes	2638 (8.3)	708 (9.7)
Discordant between countries	48 (0.2)	11 (0.2)
Total No of trials registered for trial’s sponsor:
First quarter (1-10)	8146 (25.6)	1907 (26.2)
Second quarter (11-53)	7971 (25.1)	1677 (23.1)
Third quarter (54-244)	7953 (25.0)	1547 (21.3)
Fourth quarter (274-1260)	7748 (24.4)	2143 (29.5)
No sponsor name given	145 (0.5)	18 (0.3)
Unclear sponsor name given	227 (0.7)	66 (0.9)
All sites terminated	2155 (6.8)	1062 (14.6)
No of countries:
1	20 476 (64.4)	4536 (62.4)
2	2611 (8.2)	941 (12.9)
≥3	8731 (27.4)	1797 (24.7)
Trial has multiple sponsors	982 (3.1)	145 (2.0)

Outcome data

In the main study cohort of 7274 trials where results were due, 3601 reported results (49.5%, 95% confidence interval 48.4% to 50.7%). Table 2 shows the proportion of trials reported overall at each level of each variable for the cohort of trials with results due. Results were reported for 68.1% of due trials with a commercial sponsor (95% confidence interval 66.7% to 69.4%) and 11.0% of trials with a non-commercial sponsor (9.8% to 12.4%). Trials conducted by sponsors with a large number of trials on the register had a higher proportion reported (18.4% (95% confidence interval 16.7% to 20.2%) for the lowest quarter, increasing to 77.9% (76.1% to 79.6%) for the highest quarter); appendix 1 contains a post hoc sensitivity analysis of reporting rates for this variable in smaller categories.

Table 2

Reporting rates in each cohort, by category of trial

Variables	Total trials	Trials with results	% with results (95% CI)
				All due trials	7274	3601	49.5 (48.4 to 50.7)
Completion year:
2004	3	1	33.3 (2.6 to 90.4)
2005	72	44	61.1 (49.4 to 71.7)
2006	276	154	55.8 (49.9 to 61.6)
2007	444	241	54.3 (49.6 to 58.9)
2008	658	326	49.5 (45.7 to 53.4)
2009	654	323	49.4 (45.6 to 53.2)
2010	699	340	48.6 (44.9 to 52.4)
2011	731	366	50.1 (46.4 to 53.7)
2012	713	347	48.7 (45.0 to 52.3)
2013	737	353	47.9 (44.3 to 51.5)
2014	803	378	47.1 (43.6 to 50.5)
2015	782	410	52.4 (48.9 to 55.9)
2016	702	318	45.3 (41.6 to 49.0)
Phase:
Discordant	83	60	72.3 (61.7 to 80.9)
I	23	20	87.0 (65.8 to 95.8)
II	3247	1597	49.2 (47.5 to 50.9)
III	2340	1421	60.7 (58.7 to 62.7)
IV	1581	503	31.8 (29.6 to 34.2)
Sponsor class:
Non-commercial	2353	260	11.0 (9.8 to 12.4)
Commercial	4837	3292	68.1 (66.7 to 69.4)
Mixed	57	45	78.9 (66.4 to 87.7)
Blank	27	4	14.8 (5.6 to 33.9)
Paediatric investigation plan	107	81	75.7 (66.7 to 82.9)
Condition being studied is rare disease:
No	6584	3280	49.8 (48.6 to 51.0)
Yes	584	273	46.7 (42.7 to 50.8)
Discordant	45	33	73.3 (58.5 to 84.3)
Data unavailable	61	15	24.6 (15.3 to 37.0)
Bioequivalence study:
No	7264	3595	49.5 (48.3 to 50.6)
Yes	9	6	66.7 (31.5 to 89.7)
Discordant	1	0	0.0
Participants are healthy volunteers:
No	6555	3226	49.2 (48.0 to 50.4)
Yes	708	364	51.4 (47.7 to 55.1)
Discordant	11	11	100.0
Total No of trials registered for trial’s sponsor:
First quarter (1-10)	1907	351	18.4 (16.7 to 20.2)
Second quarter (11-53)	1677	656	39.1 (36.8 to 41.5)
Third quarter (54-244)	1547	924	59.7 (57.3 to 62.1)
Fourth quarter (274-1260)	2143	1670	77.9 (76.1 to 79.6)
No sponsor name given	18	0	0.0
Unclear sponsor name given	66	2	3.0 (0.8 to 11.4)
All sites terminated	1062	328	30.9 (28.2 to 33.7)
No of countries:
1	4536	1559	34.4 (33.0 to 35.8)
2	941	616	65.5 (62.4 to 68.4)
≥3	1797	1426	79.4 (77.4 to 81.2)
Trial has multiple sponsors	145	76	52.4 (44.3 to 60.4)

Table 3 shows the crude univariable and adjusted multivariable odds ratios for features potentially associated with trial reporting. In the adjusted multivariable analysis, trials with a commercial sponsor were significantly more likely to post results (adjusted odds ratio 23.3, 95% confidence interval 19.2 to 28.2); as were trials by a sponsor who conducted a large number of trials (18.4, 15.3 to 22.1). We note that odds ratios are often high in studies where the outcome is common. In the crude data, trials completing longer ago were more likely to report results, whereas in the adjusted multivariable analysis this relation was reversed and more recent trials were more likely to report results (per year odds ratio 1.05, 95% confidence interval 1.03 to 1.07). Terminated trials were less likely to report results (odds ratio 0.55, 95% confidence interval 0.45 to 0.66). No statistically significant relation was found between the odds of reporting results and trial phase, use of healthy volunteers, rare disease status, giving no clear sponsor name, having multiple sponsors, or being a bioequivalence study. Appendix 1 contains sensitivity analyses treating “completion year” as a categorical rather than a continuous variable, treating “sponsor’s number of trials” as a continuous variable rather than quarters, and retaining only one randomly selected trial for each sponsor.

Table 3

Crude and adjusted odds ratios for factors associated with trial reporting

Variables	Crude odds ratio (95% CI)	P value	Adjusted odds ratio (95% CI)	P value
Completion year:
Increase of one year	0.98 (0.96 to 0.99)	0.005	1.05 (1.03 to 1.07)	<0.001
Phase:
Discordant	1.69 (1.03 to 2.74)	0.04	1.48 (0.75 to 2.89)	0.26
I	4.31 (1.28 to 14.53)	0.02	2.26 (0.50 to 10.28)	0.29
II	0.63 (0.56 to 0.70)	<0.001	1.09 (0.93 to 1.26)	0.29
III	Reference
IV	0.30 (0.26 to 0.34)	<0.001	1.13 (0.92 to 1.38)	0.26
Sponsor class:
Non-commercial	Reference
Commercial	17.17 (14.89 to 19.80)	<0.001	23.25 (19.15 to 28.24)	<0.001
Mixed	30.19 (15.76 to 57.81)	<0.001	15.52 (7.29 to 33.01)	<0.001
Blank	1.40 (0.48 to 4.08)	0.54	2.27 (0.70 to 7.36)	0.17
Part of paediatric investigation plan	3.23 (2.07 to 5.03)	<0.001	1.09 (0.59 to 2.01)	0.79
Condition being studied is rare disease:
No	Reference
Yes	0.88 (0.75 to 1.05)	0.15	0.96 (0.76 to 1.22)	0.74
Discordant	2.77 (1.43 to 5.37)	0.003	0.78 (0.34 to 1.77)	0.55
Data not available	0.33 (0.18 to 0.59)	<0.001	0.62 (0.26 to 1.45)	0.27
Bioequivalence study	2.04 (0.51 to 8.17)	0.31	5.16 (0.89 to 29.82)	0.07
Participants are healthy volunteers	1.09 (0.93 to 1.28)	0.27	1.29 (1.03 to 1.62)	0.02
Total No of trials registered for trial’s sponsor:
First quarter (1-10)	Reference
Second quarter (11-53)	2.84 (2.44 to 3.31)	<0.001	5.99 (4.98 to 7.20)	<0.001
Third quarter (54-244)	6.57 (5.63 to 7.67)	<0.001	19.17 (15.54 to 23.64)	<0.001
Fourth quarter (274-1260)	15.67 (13.43 to 18.29)	<0.001	18.38 (15.31 to 22.06)	<0.001
All sites terminated	0.40 (0.35 to 0.46)	<0.001	0.55 (0.45 to 0.66)	<0.001
Unclear sponsor name given	0.03 (0.01 to 0.13)	<0.001	1.65 (0.36 to 7.47)	0.52
No of countries:
1	Reference
2	3.63 (3.13 to 4.21)	<0.001	1.31 (1.08 to 1.60)	0.01
≥3	7.34 (6.45 to 8.36)	<0.001	1.89 (1.58 to 2.26)	<0.001
Trial has multiple sponsors	1.13 (0.81 to 1.56)	0.48	1.63 (1.03 to 2.58)	0.04

Errors, omissions, and inconsistencies in EUCTR data

We found that omissions and inconsistencies were common in EUCTR data, and so we present an additional analysis of these issues. While the date for global end of the trial is expected to be consistent across different country’s register entries for the same trial, 1890 trials (5.9% of the total EUCTR database) had discrepant such dates between countries: the median difference between the earliest and latest completion date among trials with a discrepancy was 48 days (interquartile range 13-133). Data for the completion date were commonly missing: in 11 531 trials every country was reported as “completed” or “terminated”—all these trials should also report a completion date; 3392 trials (29.4%) did not.

Trials with no completion date could not be included in our analysis as it could not be ascertained whether their results were due. We are, however, able to explore the possible impact of trials with missing completion dates on the overall results reporting rates. Among the 7786 eligible trials that were reported as completed in every country and correctly gave a completion date, 7024 (90.2%) completed more than 12 months ago and therefore had results due, and 722 of the 3270 (22.1%) otherwise eligible trials with missing completion dates, reported results. It is therefore likely that many of the trials with missing dates were due to report results, but failed to do so.

We found related inconsistencies for trials with discrepancies on trial status between countries. In the full EUCTR database only some (but not all) countries were marked as completed or terminated for 5754 trials and therefore could not be included in our cohort of trials with results due, as some trial sites may have been genuinely ongoing. However, 4846 of these trials (84.2%) had at least one global end of the trial date, which should only be available when all countries for that trial have completed; this strongly suggests that many of these trials have inconsistent data on the register. It is not possible to ascertain whether these trials have results due; however, 3643 of such trials (63.3%) had reported results.

Sponsor rankings

Tables 4 and 5 present ranked lists for major sponsors with the highest and lowest proportion of reported trials. Only sponsors with more than 50 trials in total on the register are included. For these tables we do not attribute trials to sponsors by company acquisition or merger, only by sponsor name (at our online audit tool, trials that may be attributable to a sponsor due to acquisition or merger are listed, but separately). As expected from the crude reporting rates and the results of the multivariable analysis, the sponsors with the highest proportion of trials reported are overwhelmingly pharmaceutical companies, whereas the sponsors with the lowest reporting rates are universities.

The live data tool was successfully delivered. Figure 2 shows a screenshot of an arbitrarily selected sponsor’s page, showing sponsor’s summary results, followed by the list of their individual trials, and reporting status for each trial. The data on the site update every month, and all current data can be viewed online (EU.TrialsTracker.net). The names of any new sponsor are manually matched against the existing list of sponsors as appropriate on a monthly basis. Over time—for example, through company acquisition or merger—one current sponsor may become responsible for previous trials from another listed sponsor: this is reflected at the bottom of each sponsors page, where suggested additional sponsors are listed for review by users.

Fig 2

Screenshot of single sponsor page on EU.TrialsTracker.net

Discussion

Compliance with the European Commission requirement for trial results to be reported to the EU Clinical Trials Register (EUCTR) is poor: only half (49.5%) of 7274 due trials have reported results. Trials with commercial sponsors were substantially more likely to post results, as were trials by a sponsor who conducted a large number of trials. Unexpectedly, we also found extensive evidence of omissions and contradictory data in EUCTR—notably, that 29.4% of trials marked as completed gave no completion date, which prevents ascertainment of compliance with reporting requirements. We also found evidence that completed trials were mislabelled on EUCTR as ongoing in some countries.

Strengths and weaknesses of this study

To our knowledge this is the first study of compliance with European Commission requirements on trials transparency, covering all trials of medicinal products conducted over a 12 year period in a territory of 500 million people, for the second largest trials registry in the world. Compliance rates with reporting requirements can be arguably ascertained more accurately for EUCTR than for ClinicalTrials.gov, as the inclusive nature of European provisions for transparency means that all trials on the EU register are required to report results within 12 months of completion; whereas the FDA Amendments Act 2007 (FDAAA) permits various exemptions from reporting requirements that cannot reliably be extracted automatically across a large volume of past trials from trial metadata on ClinicalTrials.gov.

The European Commission guideline requires the results of all trials to be reported in structured form on to the register itself. Ascertainment of the outcome—a results report on EUCTR—was therefore accurate and complete. It is possible that some trials that did not report results to EUCTR did report results elsewhere—for example, in a conference presentation, an academic journal article, as part of a meta-analysis after data were requested by systematic reviewers, or in the grey literature. Such publications do not meet the reporting requirements of the European Commission guideline and are therefore outside the scope of our study, as with previous studies on compliance with FDAAA requirements to report results on to ClinicalTrials.gov.9 10 We conducted a manual search of academic journals and grey literature for a random sample of 100 trials unreported on EUCTR, as requested during peer review for this paper (see appendix 2): 46 had results in a journal publication and five in the grey literature. Ascertainment of results publication by manual searches in academic journals and other sources for a complete cohort of trials on a register is time consuming, cannot be done with perfect accuracy, and cannot be repeated on a regular cycle of audit such as our online audit tool, where the data are updated on a monthly basis. This reflects an important advantage of rules requiring trial results to be reported directly on to a register rather than elsewhere. In addition, results reported in standardised formats to a registry may be more reliable than journal publication: both FDAAA and EU rules require complete reporting of prespecified outcomes, analyses, and adverse events; whereas reporting quality is highly variable in journal publications.19 Furthermore, two large cohort studies of 202 and 110 trials have now reported that structured results reports posted on ClinicalTrials.gov present more complete data on both results and adverse events than do traditional journal publications.20 21

We were able to identify the large cohort of trials where results were definitely due; however, omissions and inconsistencies in EUCTR data presented challenges for assessing compliance with reporting requirements for an additional subset of trials, where it was only possible to ascertain that the data on the register were flawed: 29.4% of trials listed as entirely completed in EUCTR gave no completion date, even though one is required, which made it impossible to assess whether results were due for these trials. Overall, reporting rates were worse in this subset of trials than in the cohort with consistent data; we may therefore have over-estimated compliance.

Findings in context

To our knowledge this is the first study of compliance with European Commission requirements on trial reporting. Our findings are consistent with those summarised in the most current systematic review on publication rates from 2014, which included 39 cohorts and found journal publication rates of 46.2% (95% confidence interval 40.2% to 52.4%) for trials approved by ethics committees and 54.2% (42.0% to 65.9%) for trials on trial registers; and with a previous review from 2010.7 8 Two cohort studies in 2012 and 2015 have assessed compliance with FDAAA and found compliance rates of only one trial in five.9 10 However, these estimates may be inaccurate owing to the challenges in ascertaining whether trials are exempt from FDAAA reporting requirements; and they relate to compliance with FDAAA 2007 before changes made in the 2016 Final Rule on the Act. Various further studies have assessed reporting rates on ClinicalTrials.gov for a cohort of studies without formally assessing compliance with the 12 month reporting requirements of FDAAA, and found similar proportions reported.22 23 24

Policy implications

We have found strong evidence that the European Commission guideline, requiring all trials’ results to be reported on EUCTR within 12 months of completion, is commonly being breached. Sponsors doing fewer trials, and non-commercial sponsors such as universities, have particularly low reporting rates: they may be more likely to be unaware of their obligations or lack administrative procedures to flag breaches and support compliance among their researchers. They may also lack clear lines of responsibility: in law, the sponsor is responsible for reporting the trial on to the register; in reality, it falls to the principal investigator or administrative staff. This may be particularly problematic for the smaller cohort of trials newly required (since 2016) to report results on to the register, but that completed in 2005 and have been left unreported; as staff may have moved jobs or even retired. We encourage EU universities to prioritise clarifying these issues for their staff and investing in basic internal audits and administrative work to ensure that results are reported on time. It is possible that enforcement notices or penalties would improve compliance and raise awareness of the obligation to report all trial results: these may become commonplace with the enactment of the new 2014 European Clinical Trials Regulation, which will come into force by 2022. In the absence of formal legal sanctions, public accountability and audit have valuable roles. The presence of a public ranking of sponsors’ reporting performance may encourage organisations to prioritise results reporting in general. In addition, the online resource we have produced also makes it easy for sponsors to identify individual trials from their organisations which have not yet reported results to EUCTR; it therefore offers practical support for sponsors wanting to improve compliance.

Although poor reporting rates in some sectors is a source of concern, the extremely high rate of compliance among commercial sponsors conducting a large number of trials is positive: it shows that, with an unambiguous requirement for all trials to report results, near perfect compliance can practically be delivered. In addition, since transparency requirements are relatively new, compliance may improve over time: we will assess this in future research and through routine monthly updates on the accompanying website (EU.TrialsTracker.net). We are concerned by extensive omissions and contradictory data in trial register entries on EUCTR. In some cases these errors were critical and made it impossible to ascertain the compliance status of a trial. If EUCTR is the only source of data to regulators, then it does not contain the information needed for them to establish whether all trials are compliant with European Commission guidelines on transparency. While sponsors are responsible for entering correct data, omissions and inconsistencies could be monitored and addressed by the European Medicines Agency, by running the same checks on its EUCTR database that we have run for this analysis.

Future research

Typically, publication bias research is retrospective—published long after a cohort of trials have completed—and presents only a single static estimate of overall performance for a population of trials. Static retrospective analyses such as these may not be the most effective use of analytic resources on registry and reporting data, which could be an important source of feedback to improve reporting in individual organisations. Quality improvement work through audit typically aims to identify good performers, learn from their successes, and help those with poor performance to improve. To be effective, audit should give timely, relevant, and actionable data, be repeated, and ideally be ongoing.25 These principles can be readily applied to clinical trials reporting, as we have done in this paper and the associated live data tool online (EU.TrialsTracker.net). From the launch of the associated online tool, using feedback from end users such as policy makers and the research community, we aim to learn how best to implement live feedback on reporting rates and information on individual unreported trials, for maximum usability and positive impact.

Conclusions

Compliance with the European Commission guideline, which aims to ensure that all trials report results within 12 months of completion, has been poor. Half of all due trials have not yet reported results. However, sponsors conducting a large number of trials, and pharmaceutical companies, show higher rates of compliance. We hope that accessible and timely information on the compliance status of each individual trial and sponsor will help to improve reporting rates.

Numerous cohort studies have shown that the results of clinical trials are routinely left unreported

2007 legislation that was intended to fix this problem in the US has been widely ignored

Recent EU rules require all trials conducted in Europe on medicinal products to report results directly on to the EU Clinical Trials Register (EUCTR) within 12 months of trial completion; but compliance has never been assessed

Compliance with the EU rules has been poor overall, compliance among pharmaceutical companies has been good, and universities have performed poorly

A live online searchable web resource was created, showing the reporting status of every individual trial conducted in Europe and overall performance rankings for every sponsor

This openly accessible data is updated every month

Table 4

Sponsors with highest proportion of trials reported

Sponsor	Total trials on EUCTR	Due trials	Due trials with results	% reported
Gilead Sciences	213	31	31	100.0
Chiesi Farmaceutici	94	37	37	100.0
CSL Behring	72	25	25	100.0
Alcon	71	20	20	100.0
Genentech	63	18	18	100.0
Vertex Pharmaceuticals	62	19	19	100.0
Daiichi Sankyo	62	12	12	100.0
Almirall	53	37	37	100.0
Ferring Pharmaceuticals	53	19	19	100.0
Sanofi	573	111	110	99.1
Bayer	274	72	71	98.6
Johnson and Johnson	424	108	106	98.1
Novo Nordisk	202	52	51	98.1
Servier Laboratories	134	48	47	97.9
Novartis Vaccines	142	44	43	97.7
Abbvie	179	33	32	97.0
H Lundbeck	76	29	28	96.6
Astrazeneca	520	141	136	96.5
Otsuka	58	27	26	96.3
Amgen	244	51	49	96.1
Pfizer	744	168	161	95.8
Takeda	172	47	45	95.7
Astellas	137	23	22	95.7
Bristol-Myers Squibb	314	36	34	94.4
Eisai	113	13	12	92.3
Boehringer Ingelheim	340	90	83	92.2
Biogen	103	35	32	91.4
Merck	662	164	146	89.0
GlaxoSmithKline	1060	293	260	88.7
Ipsen	74	25	22	88.0
Merck	149	33	29	87.9
Novartis	1260	473	415	87.7
UCB	180	40	35	87.5
Celgene	107	8	7	87.5
Roche	596	115	100	87.0
Abbott	109	57	47	82.5
University of Dundee	69	61	50	82.0
Actelion Pharmaceuticals	82	14	11	78.6
University of Oxford	102	26	20	76.9
Shire	98	17	13	76.5
Galderma R&D	54	24	18	75.0
Teva	81	25	18	72.0
EORTC	88	14	10	71.4
Menarini Group	64	23	16	69.6
Allergan	115	46	30	65.2
Pierre Fabre	117	18	11	61.1
Baxter	61	28	16	57.1
University of Leeds	57	14	7	50.0

EUCTR=EU Clinical Trials Register.

Table 5

Sponsors with highest proportion of trials unreported

Sponsor	Total trials on EUCTR	Due trials with results	Due trials	% reported
Hospitals of Paris	194	0	7	0.0
Karolinska Institutet	189	0	21	0.0
Radboud University	178	0	3	0.0
Charité-Universitätsmedizin Berlin	177	0	63	0.0
Erasmus University	161	0	3	0.0
University of Amsterdam	153	0	4	0.0
Agostino Gemelli University Polyclinic	142	0	11	0.0
Ghent University	126	0	19	0.0
VU University Medical Centre	126	0	3	0.0
Utrecht University	122	0	6	0.0
AOU di Bologna, Policlinico S.Orsola-Malpighi	120	0	1	0.0
Helsinki University	101	0	12	0.0
Université libre de Bruxelles	85	0	3	0.0
Vita-Salute San Raffaele University	83	0	5	0.0
Hospices Civils de Lyon	81	0	3	0.0
Heidelberg University	75	0	17	0.0
University of Oslo	72	0	1	0.0
University of Munich (Ludwig-Maximilians)	71	0	26	0.0
Maastricht University	61	0	2	0.0
Fundació Clínic per a la Recerca Biomèdica	60	0	1	0.0
University of Cologne	57	0	18	0.0
Gothenburg University	56	0	6	0.0
Uppsala University/Uppsala County Council	55	0	6	0.0
Manchester University NHS Foundation Trust	54	0	13	0.0
European Institute of Oncology	54	0	1	0.0
Blaise Pascal University	53	0	4	0.0
Hospital de la Santa Creu i Sant Pau	53	0	3	0.0
Odense University Hospital	88	1	27	3.7
Technical University of Munich	61	1	27	3.7
Medical University of Graz	105	2	53	3.8
Medical University of Vienna	354	8	166	4.8
University of Nottingham	58	1	17	5.9
Belfast Health and Social Care Trust	50	1	16	6.2
Copenhagen University and Hospitals	395	9	133	6.8
NHS Greater Glasgow and Clyde	52	1	13	7.7
KU Leuven	192	1	8	12.5
Guy's and St Thomas' NHS Foundation Trust	64	1	8	12.5
University of Birmingham	83	2	13	15.4
Aarhus University	134	7	41	17.1
King’s College London	90	2	11	18.2
Innsbruck Medical University	59	3	15	20.0
No sponsor name given	184	6	29	20.7
Imperial College London	118	5	19	26.3
Newcastle upon Tyne NHS Foundation Trust	59	4	14	28.6
The Royal Marsden NHS Foundation Trust	51	2	5	40.0
University College London	113	9	20	45.0
Eli Lilly	375	41	86	47.7
University of Leeds	57	7	14	50.0

EUCTR=EU Clinical Trials Register.