Evidence-informed recommendations to reduce dissemination bias in clinical research: conclusions from the OPEN (Overcome failure to Publish nEgative fiNdings) project based on an international consensus meeting.

BACKGROUND: Dissemination bias in clinical research severely impedes informed decision-making not only for healthcare professionals and patients, but also for funders, research ethics committees, regulatory bodies and other stakeholder groups that make health-related decisions. Decisions based on incomplete and biased evidence cannot only harm people, but may also have huge financial implications by wasting resources on ineffective or harmful diagnostic and therapeutic measures, and unnecessary research. Owing to involvement of multiple stakeholders, it remains easy for any single group to assign responsibility for resolving the problem to others.
OBJECTIVE: To develop evidence-informed general and targeted recommendations addressing the various stakeholders involved in knowledge generation and dissemination to help overcome the problem of dissemination bias on the basis of previously collated evidence.
METHODS: Based on findings from systematic reviews, document analyses and surveys, we developed general and targeted draft recommendations. During a 2-day workshop in summer 2013, these draft recommendations were discussed with external experts and key stakeholders, and refined following a rigorous and transparent methodological approach.
RESULTS: Four general, overarching recommendations applicable to all or most stakeholder groups were formulated, addressing (1) awareness raising, (2) implementation of targeted recommendations, (3) trial registration and results posting, and (4) systematic approaches to evidence synthesis. These general recommendations are complemented and specified by 47 targeted recommendations tailored towards funding agencies, pharmaceutical and device companies, research institutions, researchers (systematic reviewers and trialists), research ethics committees, trial registries, journal editors and publishers, regulatory agencies, benefit (health technology) assessment institutions and legislators.
CONCLUSIONS: Despite various recent examples of dissemination bias and several initiatives to reduce it, the problem of dissemination bias has not been resolved. Tailored recommendations based on a comprehensive approach will hopefully help increase transparency in biomedical research by overcoming the failure to disseminate negative findings. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Comprehensive set of evidence-informed recommendations aimed at reducing dissemination bias based on work conducted as part of European Union FP7 project (Overcome failure to Publish nEgative fiNdings (OPEN) project; http://www.open-project.eu).

General and specific recommendations, both targeted at various key stakeholder groups involved in the knowledge generation and translation process.

Use of a rigorous and transparent methodology to develop recommendations including at consensus meeting including external experts and stakeholders.

Iteration of some recommendations that are not new but have so far not been broadly implemented.

Introduction

In order to make informed decisions, healthcare practitioners, consumers, public health professionals, policymakers, and health and research funding bodies rely on evidence from clinical research. The generation of evidence is made possible because patients participate in clinical trials, and accept any research-related risks and burdens. To inform medical decision-making, demonstrate respect to trial participants and maintain public trust in clinical research, it is important that such evidence is made available in an easily accessible and unbiased way.1 However, many research findings are either not published at all (an estimated 50%),2 or only selectively, that is, with a bias towards specific aspects or presenting only partial information. Healthcare professionals and policymakers are therefore frequently unable to make decisions based on the entire relevant research evidence.3 This problem has been called ‘publication bias’,4 or—as more recently suggested—‘dissemination bias’, to reflect the multiple facets of this problem.5 6 Dissemination bias has recently received attention due to initiatives such as the AllTrials campaign (http://www.alltrials.net7) and the RIAT proposal,8 the discussion of oseltamivir in the context of avian influenza,9 and announcements from companies, for example, GlaxoSmithKline, to grant access to patient-level data from all company trials.10 However, general awareness of this problem and, even more importantly, the necessity to address and resolve it, has presumably not yet been fully recognised by many stakeholders.

Therefore, the European Commission called for projects to investigate the extent and impact of dissemination bias, and to develop recommendations to overcome incomplete or selective access to trial results. The European Union has committed funds to two projects, Overcome failure to Publish nEgative fiNdings (OPEN; http://www.open-project.eu)11 and UNCOVER (http://www.ait.ac.at/uncover), supported within the Seventh Framework Program. The two projects used different methodologies but aimed to address similar problems, namely, the non-publication—or in a broader sense the non-dissemination—of the complete data of all clinical trials.

We present the results of the 24-month OPEN project that ran from November 2011 to October 2013. As an interdisciplinary initiative, it brought together academics and stakeholders from across Europe, with the aim of developing evidence-informed recommendations and strategies for overcoming the failure to publish negative research findings.

One activity of OPEN was to conduct a series of systematic reviews to assess the occurrence of non-publication of research findings and the resulting dissemination bias. These reviews addressed aspects such as existing terminology to describe problems of publication and related biases,5 available methods to detect and measure dissemination bias,12 and the extent,13 and impact,14 of the problem of non-publication of research findings.

Another focus of OPEN was to describe current practices by various players involved in knowledge generation and knowledge translation to provide insights about how to avoid or reduce bias due to non-publication of research findings, and to identify ways to change practices that contribute to the problem. This has been addressed by assessing and evaluating the policies and procedures in place for preventing dissemination bias by the main parties involved in approving, funding, conducting, publishing, disseminating and assessing clinical research. Different work packages of OPEN have surveyed representatives of funding agencies, the pharmaceutical industry, research ethics committees, research institutions, researchers,15 trial registers,16 medical journals,17 regulatory agencies and benefit (health technology) assessment agencies such as NICE in the UK (http://www.nice.org.uk) and IQWiG in Germany (http://www.iqwig.de).

Findings from the OPEN work packages informed a 2-day recommendations workshop in May 2013, attended by the OPEN project partners, researchers from the UNCOVER project and selected key stakeholders from across the world. The workshop aimed at developing and refining a set of general and targeted recommendations designed to specifically consider the roles that the respective stakeholder groups should play in reducing the incomplete dissemination of research findings.

This article first describes the process used to reach a consensus on the recommendations, followed by a discussion of the general recommendations, which are aimed at the major stakeholders in the knowledge translation process. It then presents 47 concrete recommendations complementing the general recommendations aimed at the specific target groups, namely, funding agencies, pharmaceutical and device companies, research institutions, researchers (systematic reviewers and trialists), research ethics committees, trial registries, journal editors and publishers, regulatory agencies, benefit assessment institutions, and legislators.

Methods

Based on the data gathered in each OPEN work package (ie, conducted surveys, qualitative studies, and systematic reviews; see online supplementary figure), each of these groups formulated 4–7 recommendations on how to reduce dissemination bias tailored to their respective key group in the knowledge translation process. These recommendations were compiled, ordered, and harmonised to form draft recommendations used as the basis for discussion at the OPEN workshop, including further international key stakeholders in clinical research. The focus of these recommendations was on non-dissemination of full trials rather than the selective reporting of outcomes2 and/or analyses.18 This workshop was held in Freiburg, Germany, on 23–24 May 2013, in order to reach a consensus and to finalise the recommendations. The participants were the members of the OPEN Consortium (see online supplementary appendix 1), members of the OPEN advisory board (see online supplementary appendix 2), and 11 stakeholders from relevant key groups (see online supplementary appendix 3). Workshop participants discussed the draft recommendations in a structured and transparent manner, and decided on consistent definitions to be used in the phrasing of its recommendations (eg, what is meant by ‘clinical trial’; see box 1).

Clinical trial:

A clinical trial is any research study that prospectively assigns human participants or groups of humans to one or more health-related interventions to evaluate the effects on health outcomes. Clinical trials may also be referred to as interventional trials. Interventions include but are not restricted to drugs, cells and other biological products, surgical procedures, radiological procedures, devices, behavioural treatments, process-of-care changes, preventive care, etc. This definition includes phase I–IV trials.39

The recommendations were also discussed in small groups. To structure the discussion and to facilitate the documentation of the judgements underlying the recommendations, a decision table was used for rating each individual recommendation (figure 1). The decision tables required each group to assess five criteria for each recommendation, and to grade it as a ‘strong recommendation’ or only a ‘recommendation’, based on their judgements. The following five explicit criteria, which are conceptually based on the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach for developing recommendations,19 20 were assessed for each recommendation:

Figure 1

Example of framework worksheet used for discussion of recommendations.

Confidence in effectiveness—whether the group was confident that if this recommendation were implemented it would effectively reduce dissemination bias.

Balance of benefits and downsides—whether the group thought that the benefits of this recommendation would clearly outweigh its potential downsides.

Likelihood of opposition—whether the group thought that there would be no or only minor opposition to the implementation of this recommendation by relevant key groups.

Resource use—whether the group thought that implementing this recommendation would require only limited resources (both direct financial costs and indirect resources).

Implementability and feasibility—whether the group thought that this recommendation could be easily implemented (within a reasonable timeframe) and easily sustained.

For all five criteria, the groups were asked to provide a ‘yes or no’ answer and an explanation for their judgement. Regarding the classification as either ‘strong recommendation’ or ‘recommendation’, we decided not to use a strict rule (eg, ≥3 saying yes results in strong recommendation, <2 saying yes results in recommendation), but rather apply the following rule of thumb: if a recommendation was rated for each or most of the criteria as ‘yes’, the recommendation was rated as a ‘strong recommendation’. If only one or a few judgements strongly favoured a recommendation, it was classed simply as a ‘recommendation’. The rationale for this flexible approach lies in the fact that the weight each of these criteria bears on the final classification can vary depending on the recommendation at hand and/or on the extent to which a certain criterion speaks for or against a recommendation. In summary, a judgement based on integration of transparent assessments for these criteria seemed more appropriate than following a rigid and inflexible rule.20

The outcomes of all small group deliberations were then discussed in the open plenary sessions until a consensus on all recommendations was reached. The OPEN Consortium and the other meeting participants then endorsed the full set of recommendations.

Following our workshop, the list of endorsed recommendations was circulated and refined by members of the OPEN Consortium. As a last step, the final recommendations document was sent to all participants of our recommendations workshop for final technical revisions and approval.

Results

In this report, we first present four general overarching recommendations—aimed at major stakeholders in the knowledge generation and translation process—followed by the main results from the OPEN project, that is, 47 targeted recommendations addressing 11 key stakeholder groups.

General recommendations

Each general recommendation is followed by an explanation of the rationale and reasons why it was rated as ‘strong’ or not based on our decision table (see online supplementary file 1 for detailed decision tables).

All stakeholders should raise awareness about dissemination bias and measures to reduce it (strong recommendation).

Awareness of the problem of dissemination bias is a prerequisite for change. However, some of the responses to our surveys indicated that many people still deem this problem not relevant.17 It needs to be emphasised among all stakeholder groups that dissemination bias may have serious consequences on the health of people and the trust in research.

We decided to make this a strong recommendation because we think that the benefits of this recommendation clearly outweigh its potential drawbacks and that only limited resources are necessary for its implementation (as has recently been shown by the AllTrials campaign7). Potential drawbacks of this strong recommendation are that it might not be very effective in the short term and that it might face some opposition. But the Consortium deems the recommendation to be highly effective in the long run, and recognises that any opposition is likely to be due to passivity on the part of the key stakeholder groups and the limited resources available for actively engaging in raising awareness, and emphasising its relevance. Another important aspect that was identified is the need to extend awareness-raising activities to patients and their representatives. All patients participating in clinical trials should be made aware of the fact that their participation only contributes to scientific progress if the results are published, and that this is often not the case.

2. All stakeholders should disseminate and facilitate the implementation of targeted OPEN recommendations as outlined in table 1 (strong recommendation).

Table 1

Targeted recommendations by key stakeholder group

Number	Recommendation
Funding agencies
1.1	Funding agencies should include a statement on dissemination bias and the requirement for the dissemination of research results in all calls for proposals (strong recommendation)
1.2	Funding agencies should include the requirement for grantees to provide a dissemination plan for funded projects in all calls for proposals (strong recommendation)
1.3	Funding agencies should include the requirement for grantees to explicitly declare that the results of funded research will be disseminated, regardless of the nature of findings, in all funding contracts (strong recommendation)
1.4	Funding agencies should implement measures to ensure that the evaluation process of funded projects does not end with the project's final report, but instead is followed up until all agreed data have been disseminated (recommendation)
1.5	Funding agencies should consider providing incentives for researchers who disseminate their results, or, alternatively, withhold a part of the funding until a project's results are adequately disseminated (recommendation)
1.6	Funding agencies should create a publicly accessible database of all grants awarded and on how their results were disseminated in order to keep an accurate record of funded projects and publication outcomes (recommendation)
Pharmaceutical and device companies
2.1	Pharmaceutical and medical device companies should make their policies concerning the dissemination of methods and results of clinical trials publicly accessible (strong recommendation)
2.2	Pharmaceutical and devices companies should register all clinical trials in a public registry before the recruitment of the first participant (strong recommendation)
2.3	Pharmaceutical and devices companies should make their trial protocols+amendments (as submitted to RECs) available on the publication/dissemination of results (strong recommendation)
2.4	Pharmaceutical and devices companies should publish/disseminate complete summary results (aggregate data) of all trials conducted and provide access to their CSRs (for clinical trials) on request (recommendation)
Research institutions
3.1	Research institutions should provide guidance and training about the implications of and possible measures for avoiding dissemination bias (strong recommendation)
3.2	Research institutions should not accept any funding that includes clauses that prevent the dissemination of data (strong recommendation)
3.3	Research institutions should mandate the dissemination of complete summary results of all clinical trials (strong recommendation)
Researchers I: systematic reviewers
4.1	Researchers conducting SRs, MAs and NMAs should follow the best practices25 38 for performing SRs (especially those practices concerning the search for trials and the assessment of the impact of dissemination bias26) (strong recommendation)
4.2	Systematic reviewers should make SR protocols and the results of SRs informing clinical care publicly available (strong recommendation)
Researchers II: trialists
5.1	Trialists should register every trial they plan to conduct before the recruitment of the first participant (strong recommendation)
5.2	Trialists should disseminate complete summary results (as soon as possible, but no later than 12 months) from all clinical trials they conduct, that is, through journal publications and results/posting of results in registers (strong recommendation)
5.3	Trialists should make trial protocols publicly available both within the register where the trial is registered and as appendix/supporting material with the journal publication (strong recommendation)
RECs
6.1	RECs should require the registration of all clinical trials before the recruitment of the first participant (strong recommendation)
6.2	RECs should require that applicants commit to making complete summary results publicly available (recommendation)
6.3	RECs should encourage applicants to share anonymised individual patient-level data on request (strong recommendation)
6.4	RECs should require that applicants provide annual reports describing the dissemination of their study results (strong recommendation)
Trial registries
7.1	Trial registries should enable the reporting of aggregate summary results (strong recommendation)
7.2	Trial registries should enable and encourage the registration of observational human studies (recommendation)
7.3	Trial registries outside English-speaking countries should facilitate the registration process for non-English-speaking trialists (strong recommendation)
7.4	Initiatives should be developed that enable non-English speakers to use the information contained in trial registers (recommendation)
7.5	Trial registries should enable and encourage the inclusion of links to publications and other permanent data sources (eg, PubMed, other bibliographical databases, data repositories) in trial registry entries (strong recommendation)
7.6	All trial registries should comply with the WHO International Standards for Clinical Trials Registries (strong recommendation)
Journal editors/publishers
8.1	Journal editors and publishers should remove all barriers to publishing negative or inconclusive studies and consider studies for publication regardless of the direction of their findings or their sources of funding (strong recommendation)
8.2	All journals should make trial registration a requirement for publication (strong recommendation)
8.3	Journals should check all submitted manuscripts against study protocols and/or trial registry entries to detect selective reporting (strong recommendation)
8.4	Journal editors should publish editorials and commentaries about the problem of dissemination bias and the benefits of trial registration (recommendation)
8.5	Journals should check for redundant publication of results by using text-matching software and asking peer reviewers about papers reporting the same findings (strong recommendation)
Regulatory agencies
9.1	Regulation of pharmaceutical products should be extended to cover other therapeutic and diagnostic agents, such as medical devices and biologicals (strong recommendation)
9.2	Responsible authorities (such as the EMA for drugs) should mandate that all clinical trials in humans falling under their remit are registered in an EU database that is publicly accessible (strong recommendation)
9.3	Responsible authorities (such as EMA for drugs) should mandate that, on a trial's registration in an EU database, the full protocol approved by the REC, including the potential protocol amendments, is submitted and made publicly available as a searchable document (strong recommendation)
9.4	Responsible authorities (such as EMA for drugs) should mandate that the full report, including all results (eg, CSR) of a trial, is made available in the same registry that the trial was registered in, in a timely fashion (ie,1 year after trial completion or inactivity) for all trials registered in the EU database (strong recommendation)
9.5	Responsible authorities (such as EMA for drugs) should ensure that trial sponsors failing to comply with such result submission requirements are sanctioned (strong recommendation)
Benefit assessment institutions
10.1	Benefit assessment institutions should make their methods and processes of benefit assessment publicly available, in order to achieve better transparency and understanding (strong recommendation)
10.2	Benefit assessment institutions should aim for a higher degree of collaboration between institutions to facilitate the detection of further (unpublished) data and to foster data sharing (strong recommendation)
10.3	Benefit assessment institutions should use the full evidence base available for an intervention for their assessments (strong recommendation)
10.4	Benefit assessment institutions should specify their course of action if they find that the evidence base for an assessment is deemed incomplete (eg, no adequate proof of benefit based on incomplete data set) (strong recommendation)
10.5	Benefit assessment institutions should request from legislators the following items which will allow the consideration of all study results (disclosure of full protocols and full CSRs): A. A legal obligation for manufacturers to submit all requested evidence B. Public access to EMA databases C. Public access to protocols and full study reports (strong recommendation)
Legislators
11.1	Legislators should make prospective registration of clinical trials in humans mandatory (strong recommendation)
11.2	Legislators should ensure that all data related to the health of patients and the public are NOT commercially confidential/proprietary information (strong recommendation)
11.3	Legislators should institute a legal obligation for manufacturers to submit all data and other required information for the formal decision-making process (strong recommendation)
11.4	Legislators should ensure that the raw data (anonymised individual patient data) are made publicly available for all clinical trials (registered in the EU database) (strong recommendation)

CSR, clinical study report; EMA, European Medicines Agency; EU, European Union; MA, meta-analysis; NMA, network meta-analysis; REC, research ethics committee; SR, systematic review.

In order for our recommendations to have an impact, they need to be disseminated to as many representatives of the key stakeholder groups as possible. Another prerequisite to change is that the stakeholders make a collective effort. The problem of dissemination bias needs to be addressed through concerted activities that include several or all stakeholders in order to prevent responsibilities from being shifted from one stakeholder to the other (our research showed that many groups believe it to be ‘somebody else's problem’).

This recommendation was rated a strong recommendation. Its drawbacks are the likelihood of some opposition, and the need for substantial resources for comprehensive dissemination and implementation. While the dissemination of the recommendations alone will not effectively reduce dissemination bias, their full or even partial implementation should help reduce dissemination bias. In order to be fully effective, tailored dissemination and implementation strategies will be necessary.

3. All stakeholders should promote trial registration and posting of results, and support initiatives that facilitate searches across multiple trial registries (strong recommendation).

Clinical trial registers are important sources for identifying completed and ongoing trials. Efforts to promote prospective registration of trials are, therefore, crucial. The usefulness of registers, however, depends on the completeness and comprehensiveness of the included information.21 Accordingly, adequate quality control measures need to be implemented. It also became evident that metasearching, the possibility of searching across different registries through a single interface, is key to their usefulness. This is highly relevant considering existing language barriers and a lack of awareness of the existence of national registries among users, which might result in relevant registers not being included in their searches. A platform offering a metasearch for trials across multiple registries would help in identifying trials that would otherwise have been missed.

This recommendation was considered to have long-term, rather than immediate, impact. Together with the favourable balance of benefits and drawbacks, the knowledge of how limited resources could hinder good implementation led us to make this a strong recommendation. As a result of the US FDA Amendments Act, Section 801, passed in 2007, ClinicalTrials.gov has required (since September 2008) the posting of results of aggregate data from applicable trials by researchers within 12 months of completion.22 Once more registries adopt this approach and once platforms allowing metasearching are available, clinical trial registers will become essential tools to reduce dissemination bias.

4. All stakeholders should support activities to systematically, with rigorous methodology, synthesise information from studies (strong recommendation).

The work of identifying all studies that address a particular question undertaken by authors of systematic reviews helps detect dissemination bias. These efforts include systematic literature searches across electronic databases, the search of trial registries23 and grey literature as well as contacting experts in the field. Also helpful in detecting dissemination bias are, for example, the exploration of selective reporting of outcomes as part of the detailed risk of bias evaluation within systematic reviews,24 and the graphical (eg, funnel plots) and statistical assessment of potentially selective publication of whole studies (eg, Egger's test).25 26 Finally, users of systematic reviews will be made aware of the potential impact of dissemination bias on the interpretation of the findings (if these are clearly described by the review authors).

It was noted during the discussion that the availability of systematic reviews in languages other than English needs to be increased to achieve wider dissemination and use of these crucial resources for evidence-based decision-making.27 Broad acceptance and use of systematic reviews will also help to increase awareness about the problem of dissemination bias, as suggested in recommendation #1.

Specific recommendations targeted at key stakeholder groups

The 47 specific recommendations targeted at funding agencies, pharmaceutical and device industry, research institutions, researchers (systematic reviewers and trialists), research ethics committees, trial registries, journal editors and publishers, regulatory agencies, benefit assessment institutions and legislators are presented in table 1.

While they focus on broader concepts such as transparency (trial registration, access to study protocols and results data), education and training, the tackling of language barriers, and regulatory and legislative activities to counteract selective dissemination, they do provide clear guidance on activities that could and should be undertaken by the respective key stakeholder groups. It is obvious that some of these recommendations are also relevant for other stakeholder groups; however, we decided to list them primarily among those key groups where the largest impact can be expected. More detailed rationales as well as related assessments based on our decision tables for these targeted recommendations will be discussed more extensively in forthcoming articles and will be made available through the OPEN website (http://www.open-project.eu).

Discussion

The OPEN project included a broad range of key stakeholders across all relevant areas of the knowledge generation and knowledge translation process. The research that was undertaken within the separate work packages not only shed light on current measures in place to limit dissemination bias, but also gathered the views of key stakeholders on the possible barriers to transparency in medical research. Thus, based on the evidence collated throughout the first phase of OPEN, a draft set of preliminary recommendations for each stakeholder group was developed. These were then discussed with the OPEN Consortium, the advisory board of OPEN and with selected external key stakeholders to capture the different perspectives.

Instead of solely describing aspects of the general problem of dissemination bias, in addition to the general recommendations, we developed several targeted and concrete recommendations that specifically address each key stakeholder group and thus, as a whole, offer a holistic approach to address the problem. Further, due to the discussions held on topics such as possible opposition, resources required and implementability, we were able to go beyond generic recommendations. The detailed rationales and explanations for these recommendations will be made available soon in separate articles and through the OPEN website (http://www.open-project.eu). By means of this dissemination strategy, we hope to expedite the widespread dissemination and uptake of OPEN recommendations, by the relevant key groups and institutions, and to thus initiate the implementation of more adequate mechanisms to reduce and prevent dissemination bias in the long run.

Although we developed separate sets of recommendations for industry and public funding agencies, it is important to highlight that we agreed that the same ethical considerations apply to all research funders. After all, the interests of patients and the public in general should always override commercial or other interests.28 During workshop discussions it was noted that the tendency of for-profit companies and also (to some extent) other stakeholder groups such as medical licensing bodies, research institutions or policymakers and legislators, to place the protection of commercial interest over the public interest in access to data from clinical trials, does not seem to be proportional.29 30 Despite the fact that companies are investing large sums of money to develop new treatments, the group agreed that intellectual and commercial property rights are overall less important than basic human rights with regard to information and health.

As already raised, it is obvious that some of our recommendations are relevant for more than one stakeholder group. The recommendations for benefit assessment institutions, for example, are also applicable for regulators; however, they are specifically for benefit assessment agencies since work conducted within OPEN revealed that cooperation between agencies, and the transparency of methodology used, could particularly be improved within this key group. Similarly, recommendations with respect to data sharing, for example, for research ethics committees and legislators, are again also relevant for other key groups. It needs to be kept in mind, though, that data sharing was not the focus of the OPEN project.

We hope that the results of our project will complement and support ongoing and future activities aiming for more transparency in clinical trial results dissemination. As outlined in the OPEN paper on our conceptual approach to the problem of selective dissemination and resulting dissemination bias (Bassler et al, submitted for publication), the OPEN project is unique in that it aimed to determine which stakeholder groups should be held responsible, and for identifying what they can do to reduce and eliminate this multifaceted and multidimensional problem. In addition, all stakeholders who are willing to address the problem can now be provided with clear and concrete recommendations for action.

While the OPEN project aims mainly at supporting changes that will result in greater transparency in the knowledge generation and knowledge translation process in the future, it is clear that transparency and free access to data also need to be established for past trials of current treatments. As recently raised by Doshi et al8 as well as by the AllTrials campaign (http://www.alltrials.net), the complete and unbiased reporting of findings from past trials is equally important for securing access to the full evidence. The broader issue of waste in research, first introduced by Chalmers and Glasziou,31 has recently been explored in a series of articles including—among others—waste due to inaccessible research,32 and waste due to incomplete or unusable research reports.33

The recommendations from the OPEN project coincide with a number of important decisions at the European Medicines Agency (EMA) and the European Parliament, namely, the finalisation and implementation of the EMA's data sharing policy. EMA and the European Parliament, which approved the new European clinical trials regulation on 2 April 2014, have received praise for their leadership on facilitating access to clinical study reports.34 However, the recently released draft terms for use of these reports included several important restrictions, such as strict confidentiality, access to registered users and on-screen viewing only.35 36 Certainly, this would impede the widespread and most effective use of these important data sources. Criticism was voiced by academia, to which EMA responded with a press release announcing more user-friendly amendments.37 As of now, it is unclear what the final policy will look like.

We hope that the findings from the OPEN project will contribute to these decisions and implementation processes, emphasising once more the needs of academia and the public for unrestricted access to an unbiased body of evidence of complete results of all clinical trials.