Sharing of clinical trial data and results reporting practices among large pharmaceutical companies: cross sectional descriptive study and pilot of a tool to improve company practices.

OBJECTIVES: To develop and pilot a tool to measure and improve pharmaceutical companies' clinical trial data sharing policies and practices.
DESIGN: Cross sectional descriptive analysis.
SETTING: Large pharmaceutical companies with novel drugs approved by the US Food and Drug Administration in 2015. DATA SOURCES: Data sharing measures were adapted from 10 prominent data sharing guidelines from expert bodies and refined through a multi-stakeholder deliberative process engaging patients, industry, academics, regulators, and others. Data sharing practices and policies were assessed using data from ClinicalTrials.gov, Drugs@FDA, corporate websites, data sharing platforms and registries (eg, the Yale Open Data Access (YODA) Project and Clinical Study Data Request (CSDR)), and personal communication with drug companies. MAIN OUTCOME MEASURES: Company level, multicomponent measure of accessibility of participant level clinical trial data (eg, analysis ready dataset and metadata); drug and trial level measures of registration, results reporting, and publication; company level overall transparency rankings; and feasibility of the measures and ranking tool to improve company data sharing policies and practices.
RESULTS: Only 25% of large pharmaceutical companies fully met the data sharing measure. The median company data sharing score was 63% (interquartile range 58-85%). Given feedback and a chance to improve their policies to meet this measure, three companies made amendments, raising the percentage of companies in full compliance to 33% and the median company data sharing score to 80% (73-100%). The most common reasons companies did not initially satisfy the data sharing measure were failure to share data by the specified deadline (75%) and failure to report the number and outcome of their data requests. Across new drug applications, a median of 100% (interquartile range 91-100%) of trials in patients were registered, 65% (36-96%) reported results, 45% (30-84%) were published, and 95% (69-100%) were publicly available in some form by six months after FDA drug approval. When examining results on the drug level, less than half (42%) of reviewed drugs had results for all their new drug applications trials in patients publicly available in some form by six months after FDA approval.
CONCLUSIONS: It was feasible to develop a tool to measure data sharing policies and practices among large companies and have an impact in improving company practices. Among large companies, 25% made participant level trial data accessible to external investigators for new drug approvals in accordance with the current study's measures; this proportion improved to 33% after applying the ranking tool. Other measures of trial transparency were higher. Some companies, however, have substantial room for improvement on transparency and data sharing of clinical trials. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Introduction

Public expectations for transparency in the conduct and reporting of clinical trials continue to evolve. In the late 1990s, US law required only that clinical trials relating to life threatening conditions be registered. In 2007, the Food and Drug Administration Amendments Act (FDAAA) expanded registration requirements to trials for all conditions and mandated the posting of results for many phase II and phase III trials for FDA approved drugs. A decade later the Department of Health and Human Services’ Final Rule expanded trial registration and results reporting requirements to still more types of trials, including those for unapproved drug indications and phase I trials funded by the National Institutes of Health.1

Today, the transparency discussion has shifted to new terrain: sharing of patient level clinical trial data. Initiatives by the European Medicines Agency, research funders, medical journal editors, pharmaceutical companies and trade associations, the Institute of Medicine (now the National Academies of Sciences, Engineering, and Medicine), and others have heightened expectations that data sharing be a routine part of clinical trial research.2 3 An abundance of data sharing policies and guidelines and several online platforms have supported this shift.4

Evaluating and tracking progress on the implementation of data sharing policies and practices among pharmaceutical companies is, however, difficult. Existing guidelines for what should be shared, how, and when vary widely and are often vague. As might be expected, given this lack of standardization and concreteness, an analysis of 42 pharmaceutical companies’ transparency policies as of early 2016 found considerable heterogeneity in companies’ commitments.5

As part of a larger project called the Good Pharma Scorecard, we developed a harmonized, practical set of measures and a tool for assessing sharing of participant level clinical trial data by research sponsors and applied them to measure policies and practices among large pharmaceutical companies with drugs newly approved by the FDA in 2015. We also evaluated the feasibility of the tool (a ranking system) in improving companies’ practices. We further report companies’ performance on other measures of clinical trial transparency, such as trial registration and publication.

Methods

Development of data sharing measures

Review of existing data sharing guidelines

To develop the data sharing measures, we first reviewed and characterized 10 prominent data sharing guidelines, produced by the Institute of Medicine,6 Biotechnology Industry Organization,7 European Medicines Agency,8 Pharmaceutical Research and Manufacturers of America (PhRMA) and European Federation of Pharmaceutical Industries and Associations (EFPIA),9 World Health Organization,10 International Committee of Medical Journal Editors,11 National Institutes of Health,12 New England Journal of Medicine,13 Association of American Medical Colleges,14 and United Kingdom’s Medical Research Council.15 Characteristics extracted from the guidelines included which data the guidelines stated should be shared, the types of trials covered or excluded, and the timeline for sharing. At least two researchers blinded to one another’s work and trained on variable extraction independently reviewed each guideline. Reviewers noted conflicts across guidelines and vague language (eg, recommendations to share data within a “reasonable amount of time”).

Following this review, a decision was made to adhere closely to the Institute of Medicine guidelines unless there was a compelling reason to depart from them on certain measures, as they proved the most detailed and reflected both in-depth deliberation by national experts and multi-stakeholder consultation. Although we did not explicitly prespecify principles in our project, several guided our decision making—that is, that the data sharing standards must require companies to provide all the information necessary to achieve the potential benefits of data sharing, they must be clear and objectively measurable, and they should not be unreasonably burdensome on any stakeholder.

Translating guidelines into measures and patient, public, and multi-stakeholder involvement

Because the Institute of Medicine recommendations are guidelines, not measures, we had to create methods for assessing their implementation. This included identifying data sources for our assessment and clarifying ambiguous language. After translating the guidelines into draft measures, we engaged a multi-stakeholder group for review and feedback on the measures and our Scorecard/ranking concept. This group included 10 non-industry experts on data sharing (academics, regulators, medical journal editors, and trial repository experts), representatives from 11 large pharmaceutical companies, and 12 patient representatives. Companies were invited if they had a novel drug approved by the FDA between 2012 and 2015. We identified patient groups based on the relevance and responsiveness of our work to theirs. This involved applying two selection criteria—they had to have an interest in clinical trial data sharing or in conditions treated by our cohort of ranked drugs from 2012 through present. When producing the invitations, we made efforts to invite patients from organizations known to be independent from industry and provided financial support to patient participants as a way of ensuring that funding was not a barrier to participation. Appendix section 1 lists participant names and organizations.

The research team discussed each comment received and documented a decision on what revisions, if any, should be made in response. Comments from companies, along with our documented decision on each comment, were collated and will be shared publicly on the Bioethics International website (https://bioethicsinternational.org/good-pharma-scorecard/scorecard-methodology/).

The revised measures were then piloted on a sample of drugs to ensure scoring feasibility and evaluate their feasibility in improving company practices. Box 1 summarizes the final measures, with full text and a flow chart provided in Appendix sections 2 and 7. The elements assessed included whether companies registered all data sharing applicable trials so that interested parties could learn about them and request data, whether they publicly reported the number and outcome of data requests, and whether their policies provided access to analysis ready datasets and clinical study reports for applicable trials, explained how data could be requested, and shared data by our deadline. A clinical study report is a “written description of a study of any therapeutic, prophylactic, or diagnostic agent conducted in human subjects, in which the clinical and statistical description, presentations, and analysis are fully integrated into a single report.”16

Covered trials

Included—phase II and phase III trials in a successful New Drug Application

Excluded—phase I trials, expanded access trials, trials terminated without enrollment, trials for unapproved indications, and (if requested) trials with high risk of reidentification

Compliance deadline

Six months after drug approval by the FDA, six months after drug approval by the European Medicines Agency (if requested), or 18 months after the trial completion date, whichever is latest

Required elements (each 20% of data sharing score)

Applicable trials are registered, enabling data requesters to locate them

Company’s policy provides access to both analysis ready datasets and either clinical study reports or all the following: statistical analysis plan, study protocol, dataset codebook, and synopsis of clinical study report

Company’s policy explains how data can be requested

Company reports annually the number of data requests received and the outcome of each (granted or rejected)

Company’s policy specifies data will be made available by compliance deadline

At the end of the study, we will convene our annual multi-stakeholder meeting which includes patients, regulators, academics, healthcare professionals, ethicists, and industry to disseminate results, in keeping with our methods from the past several years. We will invite the patients who participated in past events as well as new groups. We also plan to engage the news and social media about the results of this research project.

Trials analyzed for data sharing measures

We applied the new data sharing measures to phase II and phase III clinical trials in new drug applications for novel drugs approved by the FDA in 2015 that were sponsored by the 20 largest biopharmaceutical companies (based on 2015 market capitalizations).17 Novel new drugs are defined as new molecular entities and new combination drugs containing at least one new molecular entity component. We confined our analysis to large companies because they sponsor most of the novel drugs approved by the FDA.

Phase I trials were excluded from our data sharing analysis primarily because of a lack of consensus on the value of reporting basic summary results for these trials that examine small numbers of healthy volunteers, let alone on the investment of resources to protect patient privacy to make these individual patient data available. Phase IV trials were not included because they are completed after the FDA approves a novel new drug application and were therefore out of our sample frame (which is focused on the trials that support a new approval). We specified that trials with a high risk of reidentifying individual participants could be excluded upon request, but sponsors made no such request.

Data collection for data sharing measures

In June 2017 we abstracted data sharing policies from company websites and trial repositories. We assessed the data sharing policies and practices from June 2017 through January 2018 (we confirmed policies did not change on company websites in January 2018). Phase II and phase III trials conducted to gain FDA approval of each drug were identified from FDA drug approval packages on Drugs@FDA. This included the FDA summary; medical, pharmacology, clinical pharmacology, and biopharmaceutics reviews; and all other review documents. During this period, we also assessed the registration of data sharing applicable trials in ClinicalTrials.gov and reports on the number of data requests received and how each data request was handled (ie, granted or rejected), with data gathered from data sharing repositories (such as clinicalstudydatarequest.com and yoda.yale.edu) and corporate websites and repositories.

In February 2018, we sent to companies the raw data underpinning our analyses, not rankings. The companies had a 30 day window in which to amend their policies and practices to meet our measures and to request correction of any errors; error corrections were adopted if confirmable through public data sources. In the rare case a new drug application holder stated it was not the responsible party for a trial and did not have control over data, we reassigned responsibility to another company if the other company confirmed responsibility in writing.

In April-May 2018, we gathered and analyzed again company policies and practices to capture any changes after the 30 day amendment window. At least two research assistants who received training and worked independently assessed the company policies. Discrepancies between their evaluations were resolved by consensus of the authors.

Additional measures of clinical trial transparency

Previously, the Good Pharma Scorecard project18 developed and published a suite of other measures of clinical trial transparency and applied them to drugs approved by the FDA in 2012 and 2014.19 20 Those measures related to clinical trial registration, reporting of trial results in a public registry, publication of results in the medical literature, and adherence to the transparency requirements of FDAAA. Using our previously published methodology (described in Appendix section 3), we applied those measures to our new sample of drugs approved by the FDA in 2015.

Following our previously published methods, the transparency measures were applied to three samples of trials from the 2015 FDA approved new drug applications, described in table 1. The “all trials” registration sample consisted of every clinical trial in each new drug application for novel drugs approved in 2015, sponsored by a large company. Although including trials in healthy volunteers in transparency requirements is controversial, we provide this analysis because the National Institutes of Health policy and some companies now require the disclosure of such trials, and these trials generate useful scientific information. For results reporting only, the all trials sample excluded trials terminated without enrollment, expanded access trials, observational studies, and trials that were ongoing or less than one year past their primary completion date by the study assessment cut-off date. Observational studies were excluded because they were generally ongoing as of our study cut-off date. From the all trials sample we selected two subsamples: one comprising trials conducted in patients (including phase I trials), as opposed to healthy volunteers; and a subsample of FDAAA trials, which consisted of trials that appear to be subject to the registration and results reporting requirements in the FDAAA Final Rule.

Table 1

Summary of all transparency measures, clinical trial samples, and compliance deadlines, on drug level

Measure	Samples to which transparency measures were applied and deadlines for compliance
	All trials in new drug application (includes trials in healthy volunteers)	Trials in patients (intent-to-treat population only; excludes trials in healthy volunteers and trials for other indications)	FDAAA trials (generally non-phase I trials with US site or by US based manufacturer)	Data sharing applicable trials (generally phase II and III trials)*
Data sharing	Not applicable	Not applicable	Not applicable	6 months after FDA and European Medicine Agency’s approval or 18 months after trial completion date, whichever is later
Trial registration	6 months after FDA approval of drug	6 months after FDA approval of indication	21 days after trial start date listed on ClinicalTrials.gov	6 months after FDA and European Medicine Agency’s approval or 18 months after trial completion date, whichever is later
Results reporting	6 months after FDA approval of drug†	6 months after FDA approval of indication†	30 days after FDA approval of indication	Not applicable
Publication	6 months after FDA approval of drug	6 months after FDA approval of indication	Not applicable	Not applicable

FDA=Food and Drug Administration.

Excludes phase I, expanded access, terminated trials without enrollment, trials for unapproved indications, and (if requested) trials with high re-identification risk.

Can include linking to a clinical study report synopsis in registry.

We made one change to the methods applied in our previous reports—in the drug and company rankings, we shortened the cut-off date for assessing whether results were published from 13 to six months after FDA approval. The time was shortened because payers and others making formulary decisions often review the medical literature for new drugs earlier than 13 months post-FDA approval of a drug.21 We continue to report the public availability of trial information at the time of FDA approval and at 3, 6, and 12 months after approval so that our results can be compared year after year; however, the drug and company rankings for 2015 are based on trial reporting and publication at six months.

Company rankings

Firstly, we ranked companies on their data sharing policies and practices by calculating an overall data sharing score for each company based on averaging their scores from the five constituent elements of our data sharing measure (box 1). For four elements, a score of 100% or 0% was assigned depending on whether a company’s policy or practice did or did not meet the requirement, respectively. Because one objective of the project was to improve data sharing, companies were given 30 days to improve their policies to satisfy the data sharing measures. We present both the initial and the final scores.

Next, we ranked companies on their overall clinical trial transparency, by averaging companies’ scores on three items: the data sharing measures applied to data sharing applicable trials, the other transparency measures applied to trials in patients, and the other transparency measures applied to FDAAA trials (details of calculations presented in table 2). If companies had multiple new drug applications approved in a reporting year, trials were pooled and aggregated, then categorized into our three trial samples for scoring. The approval of one combination drug (Tresiba) and one new molecular entity (Ryzodeg) relied on the same trial data, so we treated them as a single drug.

Table 2

Summary of measures included in overall company transparency scores

Trial samples	Measures	% of trial sample score	% of company score
Data sharing applicable trials	Registration by deadline	6.7	33.3
	Policy provides access to analysis ready dataset and clinical study report	6.7
	Policy explains how data might be requested	6.7
	Company reports number and outcome of data requests	6.7
	Policy specifies data will be shared by deadline	6.7
Trials in patients	Registration of trial and results publicly available by 6 months after primary completion date (reported or published)	33.3	33.3
Food and Drug Administration Amendments Act trials	Registration by 21 days of trial start date and results reported by 30 days after approval by FDA of indication studied	33.3	33.3
Total		100	100

Statistical analysis

Summary statistics (medians and interquartile ranges) were calculated to show how commonly trials for each approved drug met the transparency measures. All data were recorded and analyzed in Microsoft Excel V.15.11 (Redmond, WA).

Results

Establishing the data sharing measures

After analyzing existing policies and engaging with our multi-stakeholder advisory team, a decision was made that our data sharing measures should adhere closely to the US Institute of Medicine’s data sharing guidelines unless there was a compelling reason to depart from them on certain measures, as they proved the most detailed and reflected both in-depth deliberation by national experts and multi-stakeholder consultation. We departed from the Institute of Medicine guidelines in three key ways (see table 3 for a comparison of our measures to the Institute of Medicine and PhRMA/EFPIA guidelines). Firstly, we closed a loophole allowing companies to share data only after publication in a medical journal, which permitted companies to evade data sharing requirements by not publishing trials.22 Secondly, we added a requirement that all data sharing applicable trials be registered in ClinicalTrials.gov, so that interested parties can easily learn about the existence of trials and request data for their purposes. Thirdly, we added a requirement that companies report annually the number of data requests received and the decision made upon each request.

Table 3

Results of review of 10 key prominent data sharing guidelines and how the Good Pharma Scorecard data sharing measures compared with reviewed guidelines

Standards	Good Pharma Scorecard	Institute of Medicine report	PhRMA & EFPIA guidelines
When?	Six months after FDA approval, six months after marketing approval by European Medicines Agency, or 18 months after trial completion date, whichever is latest	30 days after approval, 18 months after trial completion date, or six months after publication, whichever is latest	Within a reasonable period after drug approval
Included trials	Phase II or III trials in a successful new drug application	Clinical trials initiated after 14 January 2015	Patient trials in United States and European Union drug submissions approved after 1 January 2014
Excluded trials	• Phase I trials      • Expanded access trials      • Trials terminated without enrollment      • Trials for indications other than those approved by FDA      • Trials with a high risk of re-identification (if requested)	Trials with a high risk of re-identification	Trials with a high risk of re-identification
Which data?	Trial is registered      Company’s data sharing policy provides access to: analysis ready dataset and CSR or all of the following:                  • Statistical analysis plan                  • Study protocol                  • Dataset codebook                  • CSR synopsis      Company’s policy explains how data can be requested      Company reports annually the number of data requests it received and how they are handled (approved/rejected)	Analysis ready dataset      All the following metadata:        • CSR        • Statistical analysis plan        • Study protocol        • Analytic code	Analysis ready dataset     All the following metadata:        • CSR        • Study protocol

FDA=Food and Drug Administration; CSR=clinical study report; PhRMA=Pharmaceutical Research and Manufacturers of America; EFPIA= European Federation of Pharmaceutical Industries and Associations.

Sample characteristics

In 2015, 12 of the 20 largest pharmaceutical companies (60%) had novel new drugs approved by the FDA. Collectively, they sponsored 56% (19/34) of the new drug applications for novel drugs approved in 2015, which were based on a total of 674 clinical trials. We analyzed 628 of these trials, a median of 25 (interquartile range 18-49) trials for each new drug application, after excluding trials that were not at least one year past the completion date, expanded access trials, and trials terminated without enrollment (table 4). These 628 trials reported data on more than 154 000 participants, 92% of whom were patients and 8% of whom were healthy volunteers.

Table 4

Companies’ adherence to transparency measures relating to trial registration, results reporting, and publication. Values are number with event/No in sample (percentage) unless stated otherwise

Drug	Company	Trials in patients					All trials
		Registered	Reported	Published	Publicly available		Registered	Reported	Published	Publicly available
Avycaz	Cerexa/Allergan	9/9 (100)	0/8 (0)	2/8 (25)	2/8 (25)		19/26 (73)	3/25 (12)	13/25 (52)	14/25 (56)
Viberzi	Forest/Allergan	3/3 (100)	0/3 (0)	1/3 (33)	1/3 (33)		3/14 (21)	0/14 (0)	2/14 (14)	2/14 (14)
Vraylar	Forest/Allergan	21/23 (91)	0/22 (0)	9/22 (41)	9/22 (41)		22/38 (58)	0/36 (0)	9/36 (25)	9/36 (25)
Corlanor	Amgen	12/36 (33)	9/35 (26)	13/35 (37)	16/35 (46)		12/72 (17)	9 /71 (13)	21/71 (30)	24/71 (34)
Kybella	Kythera/Allergan	15/16 (94)	8/13 (62)	2/13 (15)	8/13 (62)		23/26 (88)	8/22 (36)	4/22 (18)	10/22 (45)
Addyi	Sprout/Valeant	12/17 (71)	11/17 (65)	7/17 (41)	13/17 (76)		14/54 (26)	12/54 (22)	7/54 (13)	14/54 (26)
Yondelis	Janssen/Johnson & Johnson	10/16 (63)	6/13 (46)	11/13 (85)	11/13 (85)		12/28 (43)	8/24 (33)	15/24 (63)	15/24 (63)
Zurampic	Ardea/AstraZeneca	10/11 (91)	6/9 (67)	1/9 (11)	7/9 (78)		18/41 (44)	6/39 (15)	5/39 (13)	11/39 (28)
Genvoya	Gilead	18/20 (90)	15/18 (83)	14/18 (78)	17/18 (94)		18/33 (55)	15/31 (48)	14/31 (45)	17/31 (55)
Farydak	Novartis	21/22 (95)	18/21 (86)	18/21 (86)	20/21 (95)		21/22 (95)	18/21 (86)	18/21 (86)	20/21 (95)
Daklinza	BMS	26/26 (100)	14/24 (58)	20/24 (83)	23/24 (96)		30/54 (56)	17/52 (33)	29/52 (56)	32/52 (62)
Ibrance	Pfizer	7/7 (100)	5/5 (100)	4/5 (80)	5/5 (100)		24/24 (100)	5/20 (25)	4/20 (20)	5/20 (25)
Alecensa	Roche	5/5 (100)	1/2 (50)	2/2 (100)	2/2 (100)		10/10 (100)	1/7 (14)	3/7 (43)	3/7 (43)
Cotellic	Genentech/ Roche	3/3 (100)	0/1 (0)	1/1 (100)	1/1 (100)		9/10 (90)	1/8 (13)	4/8 (50)	4/8 (50)
Entresto	Novartis	24/24 (100)	22/22 (100)	10/22 (45)	22/22 (100)		25/47 (53)	23/45 (51)	14/45 (31)	27/45 (60)
Odomzo	Novartis	15/15 (100)	10/11 (91)	3/11 (27)	11/11 (100)		15/19 (79)	10/15 (67)	4/15 (27)	12/15 (80)
Bridion	Merck	30/36 (83)	36/36 (100)	32/36 (89)	36/36 (100)		33/54 (61)	39/54 (72)	45/54 (83)	52/54 (96)
Tagrisso	AstraZeneca	12/12 (100)	5/5 (100)	1/5 (20)	5/5 (100)		15 /15 (100)	8/8 (100)	2/8 (25)	8/8 (100)
Tresiba/ Ryzodeg	Novo Nordisk	83/83 (100)	78/78 (100)	53/78 (68)	78/78 (100)		87/87 (100)	82/82 (100)	53/82 (65)	82/82 (100)
Total (median) No of trials analyzed		384 (16)	343 (13)	343 (13)	343 (13)		674 (28)	628 (25)	628 (25)	628 (25)
Median (interquartile range) (%)		100 (91-100)	65 (36-96)	45 (30-84)	95 (69-100)		61 (49-93)	33 (14-59)	31 (23-54)	55 (31-72)

In 2017, Addyi was reacquired by Sprout Pharmaceuticals. In 2016, Odomzo was acquired by Sun Pharmaceuticals. Amgen is the US new drug application holder and has rights to commercialize Corlanor in the US, through a collaboration with Les Laboratoires Servier. Corlanor is the subject of a licensing agreement between Amgen Servier, executed in June 2013. All studies included in the Corlanor US new drug application were sponsored by Les Laboratoires Servier. Additionally, 21 trials submitted in the new drug application were completed before 2005. Three trials supporting approval of Yondelis were conducted by PharmaMar, and another two studies were investigator initiated. All five of these studies were publicly available. One trial in Alecensa was conducted by Chugai Pharmaceuticals. This trial was publicly available. Two trials in Avycaz were sponsored by AstraZeneca; Pfizer is now responsible to submit results. Each of these trials is publicly available. Allergan acquired Avycaz, Kybella, Viberzi, and Vraylar before approval by the Food and Drug Administration and notes that results submission is in process for trials without results on ClinicalTrials.gov for each of these drugs.

Data sharing

One quarter of companies fully met our data sharing measure (table 5). The median overall data sharing score among companies was 63% (interquartile range 58-85%). The most common reason companies did not satisfy the data sharing measure before the 30 day amendment window (75%) was failure to share trial data by the specified deadline. Several companies’ policies did not commit to a deadline for sharing data, whereas others committed to sharing data only after publication in a medical journal. The next most common problem was not reporting the number and outcome of data requests (six of 12 companies) and failure to register all data sharing trials so interested parties could learn about and request data (five of 12 companies). Two companies did not have data sharing policies (table 5).

Table 5

Companies’ adherence to data sharing measures.* Values are percentages unless stated otherwise

Company (new drug application sponsor)	Before 30 day amendment window								After 30 day amendment window
	No of applicable trials in new drug application	Policy provides access to analysis-ready dataset and CSR	Policy explains how data may be requested	Company publicly reports No, and outcome of data requests	Policy specifies data will be shared by deadline†	% of covered trials registered	Data sharing score		Policy provides access to analysis-ready dataset and CSR	Policy explains how data may be requested	Company publicly reports No and outcome of data requests	Policy specifies data will be shared by deadline‡	% of covered trials registered	Data sharing score
Valeant	17	0	0	0	0	71	14		0	0	0	0	71	14
Allergan	45	100	100	0	0	96	59		100	100	0	0	96	59
Amgen‡	35	100	100	100	0	31	66		100	100	100	0	31	66
Pfizer	0	100	100	100	0	NA	75		100	100	100	0	NA	75
AstraZeneca	10	100	100	0	0	90	58		100	100	100	0	90	78
Merck§	29	100	100	0	0	83	57		100	100	100	0	100	80
BMS	13	100	100	0	0	100	60		100	100	100	0	100	80
Gilead	10	0	0	0	0	100	20		100	100	0	100	100	80
Janssen/J&J	5	100	100	100	100	100	100		100	100	100	100	100	100
Novartis	6	100	100	100	0	100	80		100	100	100	100	100	100
Novo Nordisk	46	100	100	100	100	100	100		100	100	100	100	100	100
Roche	1	100	100	100	100	100	100		100	100	100	100	100	100
Median (interquartile range)	12 (6-32)	100 (100-100)	100 (100-100)	50 (0-100)	0 (0-25)	100 (86-100)	63 (58-85)		100 (100-100)	100 (100-100)	100 (75-100)	0 (0-100)	100 (93-100)	80 (73-100)
% of companies fully adopting measures		83	83	50	25	50	25		92	92	75	42	58	33

CSR=clinical study report.

As measured after 30 day window in which companies could improve their data sharing policies to satisfy the current study’s data sharing measures and using trial completion date (rather than primary completion date) to establish the deadline for providing data access.

Deadline refers to cut-off point set forth in this project’s data sharing measure: six months after marketing approval by the Food and Drug Administration and European Medicines Agency or 18 months after the trial’s completion date, whichever is later.

Amgen certifies all 35 trials were sponsored by Les Laboratories Servier, which developed the drug. Amgen is the new drug application holder for and has rights to commercialize Corlanor in the US, through a collaboration with Servier. Additionally, 21 of the 35 trials were completed before 1 January 2005.

Six trials were removed from the “after 30 day amendment window” denominator because Merck certified that those trials’ data are under the control of another company, Organon.

At the end of the 30 day window in which companies could amend their policies to meet our measure, the number of companies meeting our measure increased from 25% to 33%, and the median overall data sharing score for the 12 companies increased from 63% to 80% (interquartile range 73-100%) (table 5). Three companies changed their policies. AstraZeneca added a new provision to report annually the number of received data requests and the outcome of each. Novartis added timelines for data sharing where previously none were specified. Gilead substantially expanded its data sharing policy (eg, by adding timelines for data sharing), although we could not confirm whether this was in direct response to our preliminary scoring. In our sensitivity analysis, results were similar whether completion date or primary completion date was used as the benchmark for adhering to the measure (see Appendix section 4).

A few companies’ data sharing commitments exceeded the standards measured. Novo Nordisk, for example, provides access to trial data sooner than our standard required. Some companies also commit to sharing data for additional types of trials, such as phase I and phase IV trials.

Additional transparency measures

When we examined trials conducted in patients, a median of 100% (interquartile range 91-100%) of patient trials per drug were registered, 65% (36-96%) reported results or provided a clinical study report (CSR) summary, and 45% (30-84%) were published (table 4). Overall, results for a median of 95% (69-100%) of trials in patients were publicly available in some form (reported, shared in a CSR, or published) within six months of FDA approval of the drug (table 4).

Trials became increasingly more available over time, with 60% (interquartile range 25-76%) of patient trials available at FDA approval, 76% (46-96%) 30 days later, 80% (46-97%) three months later, and 100% (77-100%) 12 months later (see Appendix section 5). A median of four trials for each new drug application were FDAAA applicable trials, with a median of 100% (83-100%) of these trials per drug meeting our FDAAA trial measures (see Appendix section 6).

When we examined results on the drug level, less than half (42%) of reviewed drugs had results for all their new drug application trials in patients publicly available in some form by six months after FDA approval. Of drugs with FDAAA applicable trials (17), 35% of these drugs did not fully meet the FDAAA applicable measures for timely registration and results reporting.

When we examined all trials in successful new drug applications, including those in healthy volunteers, a median of 61% (49-93%) of trials per drug were registered, 33% (14-59%) reported results or shared a CSR summary, and 31% (23-54%) were published by six months post-FDA approval of the drug (table 4). Overall, results for a median of 55% (31-72%) of all trials were publicly available in some form. Only 11% of reviewed drugs had public results (reported, published, or CSR summary) for all new drug application trials by six months post-FDA approval. However, public availability of trial results for the sample comprising all trials also improved over time, with 29% (interquartile range 15-56%) available at FDA approval, 41% (24-62%) 30 days later, 50% (26-63%) three months later, 55% (28-80%) six months later, and 63% (29-96%) 12 months later (see Appendix section 5).

Company transparency rankings

The median overall transparency score among the 12 companies was 92% (interquartile range 78-95%) (table 6). Two companies, Roche and Novo Nordisk, tied for the top ranking in overall transparency; each with scores of 100%. Roche, Novo Nordisk, and Janssen/Johnson & Johnson all achieved scores of 100% on the data sharing measure.

Table 6

Company rankings on overall clinical trial transparency for novel drugs approved in 2015

Rank No	Company (new drug application sponsor)	Patient trials score (%)	Data sharing score (%)	FDAAA trials score (%)	Overall company score (%)
1	Roche	100	100	100	100
1	NovoNordisk	100	100	100	100
3	Novartis	98	100	100	99
4	Merck	100	80	100	93
5	BMS	96	80	100	92
5	Pfizer	100	75	100	92
7	Gilead	94	80	100	91
8	Janssen/Johnson & Johnson	85	100	86	90
9	AstraZeneca	86	78	86	83
10	Valeant	76	14	100	63
11	Amgen	46	66	Not applicable	56
12	Allergan	43	59	40	47
Median (interquartile range) score		95 (83-100)	80 (73-100)	100 (93-100)	92 (78-95)

Amgen received approval from the Food and Drug Administration for its drug Corlanor based on trials largely completed before FDAAA (FDA Amendments Act) took effect. Therefore, its company score and ranking are based solely on public availability of trials conducted in patients (it does not include a FDAAA score). Additionally, Amgen is the US new drug application sponsor of the drug, but it did not sponsor any of the trials in the new drug application. Corlanor is the subject of a licensing agreement between Amgen and Les Laboratoires Servier, executed in June 2013, for the US registration and commercialization of the drug.

Discussion

Tracking and incentivizing progress in the journey towards routine sharing of participant level clinical trial data requires harmonized, practical measures that can be applied to any research sponsor. In this study, we developed such a data sharing measure through a rigorous process, demonstrated its feasibility, and assessed current practices for data sharing among large pharmaceutical companies. We also studied the feasibility of using our data sharing measures and tool (a ranking process) to improve company policies and practices. Additionally, we reported on how adherence to transparency standards relating to trial registration and results publication has changed over time.

Application of our newly developed data sharing measure to the 12 large pharmaceutical companies with drugs approved in 2015 showed moderate initial adherence (median score of 63%; one quarter of companies achieved perfect scores), which increased after companies were offered one month to improve their policies (median final score 80%; one third of companies with perfect scores). Most (83%) companies we studied had data sharing policies, all of which now provide access to analysis ready datasets and CSRs and explain how data can be requested.

However, one quarter of companies’ policies still do not report on how the company deals with requests, and 58% do not commit to furnishing data by a reasonable or defined time point. These are important omissions. Documenting that most data requests are granted, as some companies have done,23 is an important accountability mechanism that shows companies’ practices adhere to their policies. Committing to timely provision of data and metadata also helps ensure data sharing policies result in dissemination on a useful timeline.

Comparison with our other studies on data sharing and trial transparency

Juxtaposing our results concerning rates of trial registration and results reporting to our earlier analyses of trials in new drug applications approved in 2012 and 2014 reveals improvement over time in key measures.19 20 The median proportion of trials in patients with publicly available results at 12 months after FDA approval increased from 87% for 2012 drugs to 100% for 2015 drugs. On the other hand, improvements in transparency were not observed for the sample of all trials, which includes trials in healthy volunteers. The median proportion of such trials with publicly available results (in any form) was lower for 2015 (63%) than for 2012 (65%). These findings generally accord with reports conducted by other colleagues on results reporting in the European Union.24

Our results reveal some persistent heterogeneity among large pharmaceutical companies in their commitments to data sharing, confirming results from other recent studies,5 25 26 one of which examined whether companies had data sharing policies but imposed no standards for what constituted an acceptable policy nor measured adherence to policies. In our analysis, companies’ data sharing scores ranged from 14% to 100% and overall transparency scores from 47% to 100%. Three companies were in the bottom tercile of the overall transparency rankings in both 2014 and 2015; two of these have not adopted data sharing policies. In contrast, three other companies showed willingness to strengthen their data sharing policies after receiving feedback from our project, suggesting that public ranking and benchmarking can be helpful in moving at least some companies toward greater transparency.

Comparison with other studies

These finding are in keeping with other studies showing that ranking, rating, and benchmarking are associated with improved quality and firm performance. Evaluating 598 firms subject to environmental ratings, Chatterji and Toffel found low scoring firms substantially changed practices in response to poor ratings; responsive action was particularly prevalent in heavily regulated industries and when “faced (with) less costly opportunities to improve.”27 28

Many examples exist of successful grading and rating systems using reputational incentives to improve firm behaviors. Restaurant grades, for instance, have catalyzed restaurants to substantially improve cleanliness, which in turn has reduced the number of hospital admissions for foodborne illnesses.29 Because the legitimacy and even the survival of institutions are often threatened when negative information is disclosed about the operations of firms, companies are incentivized to pay attention to ratings and improve their performance upon receiving poor scores. The knowledge that other companies are performing better, along with opportunities to learn and refine beliefs about previously unobserved quality, further helps catalyze change.

Conclusions and policy implications

In the case of the pharmaceutical industry, external stakeholders such as patients/carers, clinicians, and investors can further speed change by demanding that low scoring companies commit to reform as a condition of partnering with or supporting them.30 Empowering these groups to be effective levers for change is one reason we report aggregate scores on the company level. More detailed information about companies’ scores can be found in the appendix of this paper and more easily understandable information on the Bioethics International website.

Five years ago, few companies had policies to share participant level trial data. Data sharing gained traction around 2013 with leading efforts by GSK,31 Yale’s YODA Project with Johnson & Johnson,32 33 and Project DataSphere.34 In 2014, Duke and Bristol-Myers Squibb partnered to form the DCRI-BMS Data Sharing Initiative (SOAR).35 Other companies are now voluntarily coming along, some quicker than others.

Clearly, major shifts are occurring in data sharing. Although evidence documenting tangible benefits of data sharing has not emerged, optimism about the potential clinical and scientific benefits is considerable.6 For these benefits to be quickly and fully realized, however, we need a way to benchmark progress—and by doing so, further encourage it.

The Good Pharma Scorecard, along with efforts by others,36 37 provides a means of monitoring progress and identifying areas where research sponsors’ transparency and data sharing practices can improve. These measures should help solidify consensus about the standards that companies must meet. This, in turn, may stimulate companies to improve transparency practices by reducing uncertainty about what is expected and frustration from trying to satisfy conflicting standards. Developed through a detailed process including review of leading transparency guidelines and multi-stakeholder consultation, the Good Pharma Scorecard measures represent standards that set a high ethical bar for companies but are both operationally feasible and not unduly burdensome to meet, as shown when one quarter of companies achieved perfect scores.

Limitations of this study

Our study has several limitations. Firstly, at the drug level, we attribute transparency scores to the company that sponsored a drug’s new drug application, although a few trials were sponsored by other companies (eg, companies the sponsor acquired). We report details of such trials in the results tables and reassigned trials to other companies where both agreed that was appropriate.

Secondly, our analysis was limited to large companies with drugs approved in the US in 2015. A recent analysis of company policies suggests commitments might differ between large and smaller companies.5 Further, some companies that appeared in our 2012 and 2014 rankings do not appear for 2015, because they did not have a novel drug approval, complicating longitudinal comparisons.

Thirdly, our study focused on companies’ data sharing, reporting, and publication policies and practices; it was not feasible to review the quality of disseminated data, the ease with which data could actually be obtained, or criteria used by each company to evaluate data requests. Fourthly, our company rankings are not adjusted for the volume of trials conducted. For some companies, the number of trials relating to drugs approved in 2015 was small.

Finally, the company rankings weight different disclosure methods differently (see table 3), which may be controversial. Trial registration, for instance, is weighted more heavily than whether a company reports the number of data requests received. We did this to keep evaluations easy to interpret; each evaluated sample (eg, trials in patients, data sharing applicable trials) accounts for one third of the company score, regardless of the number of parts the measure comprises. Because trial registration is essential for achieving all benefits of trial transparency, including recruiting participants into trials, it is not unreasonable to weight it more heavily than other factors in determining company rankings.

Conclusion

We found it feasible to develop a tool to measure data sharing policies and practices among large companies and have an impact in improving company practices. Using this measurement and ranking approach, we found some noteworthy efforts among large companies to share participant level trial data and a willingness by some of the companies to improve their policies when needed. Though these efforts are laudable, many companies still need to substantially improve their data sharing policies and practices. Providing companies with a consistent, fair, achievable set of measures is important to encourage and track further progress toward routine data sharing.

Numerous initiatives have called for sharing deidentified, participant level data to be a routine part of clinical trial research and have developed policies, guidelines, and online platforms to support data sharing

Existing guidelines for what should be shared, how, and when vary widely and are often vague

A previous study of pharmaceutical companies’ transparency policies found considerable heterogeneity in companies’ commitments

A tool was developed to assess data sharing policies and practices among pharmaceutical companies and show its feasibility and utility in improving transparency and data sharing practices among companies

Using the developed measures and tools (including a company ranking system), 25% of large pharmaceutical companies fully met the data sharing standard—the proportion increased to 33% when companies were given an opportunity to improve their policies and practices

Despite noteworthy commitments by some companies to share participant level trial data and a willingness by others to improve their policies, many companies still have substantial room for improvement