Nonregistration, discontinuation, and nonpublication of randomized trials: A repeated metaresearch analysis.

BACKGROUND: We previously found that 25% of 1,017 randomized clinical trials (RCTs) approved between 2000 and 2003 were discontinued prematurely, and 44% remained unpublished at a median of 12 years follow-up. We aimed to assess a decade later (1) whether rates of completion and publication have increased; (2) the extent to which nonpublished RCTs can be identified in trial registries; and (3) the association between reporting quality of protocols and premature discontinuation or nonpublication of RCTs. METHODS AND
FINDINGS: We included 326 RCT protocols approved in 2012 by research ethics committees in Switzerland, the United Kingdom, Germany, and Canada in this metaresearch study. Pilot, feasibility, and phase 1 studies were excluded. We extracted trial characteristics from each study protocol and systematically searched for corresponding trial registration (if not reported in the protocol) and full text publications until February 2022. For trial registrations, we searched the (i) World Health Organization: International Clinical Trial Registry Platform (ICTRP); (ii) US National Library of Medicine (ClinicalTrials.gov); (iii) European Union Drug Regulating Authorities Clinical Trials Database (EUCTR); (iv) ISRCTN registry; and (v) Google. For full text publications, we searched PubMed, Google Scholar, and Scopus. We recorded whether RCTs were registered, discontinued (including reason for discontinuation), and published. The reporting quality of RCT protocols was assessed with the 33-item SPIRIT checklist. We used multivariable logistic regression to examine the association between the independent variables protocol reporting quality, planned sample size, type of control (placebo versus other), reporting of any recruitment projection, single-center versus multicenter trials, and industry versus investigator sponsoring, with the 2 dependent variables: (1) publication of RCT results; and (2) trial discontinuation due to poor recruitment. Of the 326 included trials, 19 (6%) were unregistered. Ninety-eight trials (30%) were discontinued prematurely, most often due to poor recruitment (37%; 36/98). One in 5 trials (21%; 70/326) remained unpublished at 10 years follow-up, and 21% of unpublished trials (15/70) were unregistered. Twenty-three of 147 investigator-sponsored trials (16%) reported their results in a trial registry in contrast to 150 of 179 industry-sponsored trials (84%). The median proportion of reported SPIRIT items in included RCT protocols was 69% (interquartile range 61% to 77%). We found no variables associated with trial discontinuation; however, lower reporting quality of trial protocols was associated with nonpublication (odds ratio, 0.71 for each 10% increment in the proportion of SPIRIT items met; 95% confidence interval, 0.55 to 0.92; p = 0.009). Study limitations include that the moderate sample size may have limited the ability of our regression models to identify significant associations.
CONCLUSIONS: We have observed that rates of premature trial discontinuation have not changed in the past decade. Nonpublication of RCTs has declined but remains common; 21% of unpublished trials could not be identified in registries. Only 16% of investigator-sponsored trials reported results in a trial registry. Higher reporting quality of RCT protocols was associated with publication of results. Further efforts from all stakeholders are needed to improve efficiency and transparency of clinical research.

Introduction

Rigorously planned and conducted randomized clinical trials (RCTs) are critical to inform the effectiveness and safety of healthcare interventions [1,2]. Clinical trial registries were implemented in the early 2000s to avoid unnecessary duplication of research and to estimate and deter publication bias. In 2005, the International Committee of Journal Editors proclaimed prospective trial registration (i.e., registration before enrolling the first participant) as a requirement for publication [3,4]. Shortly thereafter, laws in different regions (e.g., European Union and North America) required trials to be registered [5,6] and making results available [7,8]. Several studies have explored the proportion of published RCTs that are registered [9-12]; however, such investigations used published RCTs—and not RCT protocols—as the denominator and thus were unable to assess publication bias.

Trial discontinuation and nonpublication can constitute substantial research waste [13]. For example, if an RCT is discontinued due to slow participant recruitment before the planned sample size is reached, the trial is typically not sufficiently powered to answer the primary research question. The data, however, can still be useful in meta-analyses. Hence, it is crucial that all RCT results, including discontinued trials, are made available so that evidence is not lost and resource waste is minimized. We conducted an international metaresearch study of 1,017 RCT protocols approved between 2000 and 2003 that found 1 in 4 trials was prematurely discontinued, primarily due to poor recruitment [14]. Only 59% of approved trials were published at a median follow-up of 12 years, and premature discontinuation was associated with a lower likelihood of publication [14]. We acquired a new sample of RCT protocols, approved in 2012 by the same research ethics committees (RECs) plus an REC from the United Kingdom, to explore if trial completion and publication rates have changed. In addition, we aimed to investigate to what extent nonpublished RCTs can be found in trial registries, and the association between reporting quality of RCT protocols and trial discontinuation due to poor recruitment or nonpublication of results [15].

Methods

The present study is an associated project of the Adherence to SPIrit REcommendations (ASPIRE) study [15]. The ASPIRE study group is an international collaboration of researchers with a mandate to evaluate the completeness of RCT protocols before and after publication of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) statement [16]. Further ASPIRE substudies examine the use of patient-reported outcomes in RCT protocols, the reporting quality of RCT protocols with regulated versus nonregulated interventions, the planning of subgroup analyses in RCT protocols, and the use of routinely collected data in RCTs [15].

Study sample

The rationale and protocol for this study has been published [15]. In brief, we acquired 360 RCT protocols that were approved in 2012 by RECs located in Switzerland (Basel, Bellinzona, Bern, Geneva, Lausanne, St. Gallen, and Thurgau), the UK (the Bristol office of the UK National Research Ethics Service responsible for 19 RECs in the UK), Germany (Freiburg), and Canada (Hamilton; see for details of participating RECs). We included RCTs in which participants were randomly assigned to different interventions (or an intervention and control group) to evaluate effects on health outcomes. We included all eligible RCT protocols that were available at participating RECs with the exception of Freiburg (Germany), Hamilton (Canada), and Zürich (Switzerland), where a random sample was selected (see study protocol for more details [15]). Studies labeled as pilot, feasibility, or phase 1 studies were excluded [15]. We also excluded duplicate protocols, and protocols for trials that were ongoing, had not started at the time of data collection, or were terminated but did not recruit any patients.

Reviewers determined, independently and in duplicate (for over 75% of included protocols), the reporting quality of all eligible RCT protocols by assessing the proportion of SPIRIT checklist items met [15-17].

Data collection

Reviewers determined whether each RCT was registered, prematurely discontinued (including reasons for discontinuation), and if trial results were published in a peer-reviewed journal or trial registry. In detail, we assessed if trials were registered by reviewing REC files and through a systematic search of the following registries and databases between March and September 2019: (1) the World Health Organization: International Clinical Trial Registry Platform (ICTRP) database; (2) US National Library of Medicine (ClinicalTrials.gov); (3) European Union Clinical Trial Registry (EUCTR); and (4) the ISRCTN registry. We also used the Google search engine to identify registration details. We classified an RCT as unregistered if we were not able to find any record or registration through our search. We used the following strategies to identify trials: (i) searching trial registration numbers (if reported in the protocol); (ii) full titles; (iii) short titles; (iii) study acronyms; and (iv) searching for the study population and intervention (with or without specifying the control group).

We extracted the trial status (i.e., completed, early discontinuation and why, or unclear), planned and achieved sample size, availability of study results, and reported links to full text publications. In February 2022, we rechecked the trial status and availability of study results for all included RCTs. We designated the trial status as unclear when an RCT was labeled as ongoing in the registry but the status had not been updated in the previous 2 years, unless the planned completion date was after February 2022. We contacted RECs and surveyed principal investigators for clarification when trial status was unclear (see and for details). We classified RCTs as prematurely discontinued if they were specified as such in a trial registry, publication, or communication with a REC or trial author, or if the achieved sample size was <90% of the prespecified target sample size in the approved study protocol [14]. We conducted a systematic search of the following 3 electronic databases for full text publications corresponding to RCT protocols: PubMed, Google Scholar, and Scopus (see for search strategies). For all corresponding full text publications identified, we extracted the planned and achieved sample size and, if applicable, the reason for premature trial discontinuation. All searches and data extraction were conducted in duplicate, and disagreements were resolved by discussion.

Analysis

Trial registration, publication, completion, and reasons for discontinuation are reported as frequencies and percentages with 95% confidence intervals (CIs), stratified by sponsorship (industry versus investigator) and country of study approval. We conducted univariable and multivariable logistic regression analyses with the following factors as dependent variables: (i) publication in a peer-reviewed journal; (ii) premature trial discontinuation due to poor recruitment; and (iii) discontinuation due to preventable reasons (considering discontinuation due to futility, benefit, harm, and external evidence as not preventable [18]; not prespecified in protocol paper [15]). These variables were selected because all RCTs should be published and discontinuation due to recruitment problems or preventable reasons should ideally be avoided (while other reasons for discontinuation, e.g., due to benefit or harm, might be in the interest of patients). We examined the following 7 independent variables in our models, hypothesizing that they might be associated with lower rates of discontinuation and nonpublication: (1) greater proportion of SPIRIT items reported in the protocol; (2) larger target sample size; (3) use of an active comparator versus placebo; (4) multicenter versus single-center study; (5) reporting of any recruitment projection versus not reporting; (6) industry- versus investigator-sponsored trials; and (7) support from a Clinical Trial Unit (CTU) or a Contract Research Organization (CRO) versus no support. The first variable was prespecified in our study protocol [15]. The other 6 variables were selected post hoc following our previous conducted analysis of trial protocols approved between 2000 and 2003 to facilitate comparison [14]. Variable 7 could only be included in a separate analysis in which we excluded UK protocols, as we were not able to extract from these protocols whether support from a CTU or CRO was provided. For all regression models, we calculated unadjusted and adjusted odds ratios (ORs) with 95% CIs. All analyses were conducted using Stata version 16.1 with p < 0.05 (2-sided) as the threshold for statistical significance (see for statistical code).

Registration and protocol

Since PROSPERO does not allow the registration of systematic metaresearch studies that do not assess the effect of an intervention on a health outcome, we did not register this study. The protocol is published and publicly available [15].

Patient and public involvement and reporting

Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research. This study is reported as per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline [19,20].

Results

Of the 360 potentially eligible RCT protocols approved by our study RECs in 2012, 14 were ongoing, 15 never started, and 5 were duplicate submissions, resulting in a total of 326 protocols for analysis (). Included RCT protocols had a median planned sample size of 250 participants (interquartile range [IQR] 100 to 600) and a median proportion of 69% SPIRIT items reported (IQR 61% to 77%). Approximately half (55%) were industry sponsored (179/326), the majority were multicenter studies (82%; 266/326), employed a parallel group study design (91%; 296/326), and assessed the effect of a drug (64%; 207/326; ). Most included RCT protocols were approved in Switzerland (51%; 165/326), 27% in the UK (89/326), 11% in Germany (37/326) and Canada (35/326). Baseline characteristics stratified by countries are presented in . Characteristics of included trials were similar compared to our previous study of RCT protocols approved between 2000 and 2003 [14].

Of 326 RCTs, 94% (307/326) were registered (84% prospectively; 274/326, 10% retrospectively; 33/326), and 6% (19/326) of trials were unregistered (for 11 of the 19 unregistered RCTs completion status remained unclear; Table 2). Retrospective registration, meaning registration after recruitment of first patient, was more common in investigator-sponsored trials (15%; 22/147) than industry-sponsored trials (6%; 10/179). Approximately half (53%; 173/326) of all RCTs reported their results in a trial registry (industry-sponsored 84% [150/179] vs. 16% [23/147] investigator-sponsored), and 79% (256/326) of trials were published in a peer-reviewed journal at 10 years follow-up. Of the 70 RCTs that were not published in a peer-reviewed journal, 15 (21%) were unregistered. These 15 RCTs had a median planned sample size of 80 (IQR, 30–150) and were mostly single-center (67%; 10/15) investigator-sponsored (80%; 12/15) trials. Among the 256 published trials, only 4 were unregistered (2%). Results for 42 RCTs (13%) were neither published nor reported in a trial registry. The results of investigator-sponsored trials were more likely to be unavailable than industry-sponsored trials (24%; 35/147 vs. 4%; 7/179). Among unpublished trials, 40% (28/70) made their results available through a clinical trial registry (industry-sponsored 79% [26/33] vs. 5% [2/37] investigator-sponsored; ).

Table 2

Registration, completion, and publication status of RCTs approved by research ethics committees in 2012.

	Industry-sponsored RCTs (n = 179) N (%, 95% CI)	Investigator-sponsored RCTs (n = 147) N (%, 95% CI)	All RCTs (n = 326) N (%, 95% CI)
Registration status
Registered	175 (97.8%, 94.4%–99.4%)	132 (89.8%, 83.7%–94.2%)	307 (94.2%, 91.0%–96.5%)
Prospectively registered	164 (91.6%, 86.6%–95.2%)	110 (74.8%, 67.0%–81.6%)	274 (84.0%, 79.6%–87.9%)
Retrospectively registered	10 (5.6%, 2.7%–10.0%)	22 (15.0%, 9.6%–21.8%)	33 (10.1%, 7.1%–13.9%)
Not registered	4 (2.2%, 0.6%–5.6%)	15 (10.2%, 5.8%–16.3%)	19 (5.9%, 3.5%–9.0%)
Completion status
Completed	119 (66.5%, 59.1%–73.3%)	84 (57.1%, 48.7%–65.3%)	203 (62.3%, 56.8%–67.6%)
Discontinued	57 (31.8%, 25.1%–39.2%)	41 (27.9%, 20.8%–35.9%)	98 (30.1%, 25.1%–35.4%)
Unclear	3 (1.7%, 0.3%–4.8%)	22 (15.0%, 9.6%–21.8%)	25 (7.7%, 5.0%–11.1%)
Results availability
Peer-reviewed publication	146 (81.6%, 75.1%–87.0%)	110 (74.8%, 67.0%–81.6%)	256 (78.5%, 73.7%–82.8%)
In clinical trial registry	150 (83.8%, 77.6%–88.9%)	23 (15.7%, 10.2%–22.5%)	173 (53.1%, 47.5%–58.6%)
As peer-reviewed publication and in clinical trial register	124 (69.3%, 62.0%–75.9%)	21 (14.3%, 9.1%–21.0%)	145 (44.5%, 39.0%–50.1%)
Results not available (neither as publication nor in clinical trial register)	7 (3.9%, 1.6%–7.9%)	35 (23.8%, 17.2%–31.5%)	42 (12.9%, 9.4%–17.0%)
Neither registered nor published	3 (1.7%, 0.3%–4.8%)	12 (8.2%, 4.3%–13.8%)	15 (4.6%, 2.6%–7.5%)
Not published in journal but registered a	30/33 (90.9%, 75.7%–98.1%)	25/37 (67.6%, 50.2%–82.0%)	55/70 (78.6%, 67.1%–87.5%)
Not published in journal but results available in registry a	26/33 (78.8%, 61.1%–91.0%)	2/37 (5.4%, 0.7%–18.2%)	28/70 (40.0%, 28.5%–52.4%)

aOnly a subsample of 70 unpublished trials considered.

CI, confidence interval; RCT, randomized clinical trial.

Flow chart.

ASPIRE, Adherence to SPIrit REcommendations; RCT, randomized clinical trial.

Approximately 1 in 3 RCTs were prematurely discontinued (30%; 98/326; Tables and S2). From the discontinued RCTs, 67% (66/98) were available as a peer-reviewed publication (). Compared to discontinued trials, completed RCTs were more likely to be published (OR 7.08; 95% CI: 3.35 to 15.52; p < 0.001) and to make their results available through a journal or trial registry (OR 6.06; 95% CI: 1.94 to 22.24; p < 0.001; .

The main reason for RCT discontinuation was poor recruitment (37%; 36/98) followed by stopping for futility (16%; 16/98; ). The proportion of prematurely discontinued RCTs did not change compared to protocols approved a decade earlier (28% of protocols approved in 2000 to 2003 [14] versus 30% of those approved in 2012). Data comparing results from this study with the cohort from 2000 to 2003 [14] are presented in . Switzerland had more unregistered trials (7.9%; 13/165), compared to the other countries (UK 4.5%, 4/89; Germany 2.7%, 1/37; Canada 2.9%, 1/35); otherwise, results were comparable in terms of discontinuation and nonpublication among the 4 included countries (). Multivariable analyses indicated that RCTs with more complete protocol reporting according to SPIRIT guidelines [19,20] were less likely to be unpublished (OR 0.71 [in increments of 10% proportion of adherence]; 95% CI: 0.55 to 0.92; p = 0.009; ). None of the assessed characteristics were found to be associated with discontinuation due to poor recruitment or discontinuation due to preventable reasons. A separate analysis excluding the sample from the UK found no association of CTU/CRO support with trial discontinuation or publication (see ).

Discussion

Our study found that most RCTs with ethics approval in 2012 (94%) were registered; however, 1 in 10 were registered retrospectively. Further, when restricted to unpublished RCTs, only 4 out of 5 trials were registered. RCT protocols with higher reporting quality, as indicated by greater adherence to SPIRIT guidelines [19,20], were more likely to be published in a peer-reviewed journal. Only 1 in 6 investigator-sponsored trials made results available in a trial registry. Approximately 1 in 3 RCTs were discontinued before the original planned sample size was reached, and 1 in 5 trials remained unpublished at 10 years follow-up. The results of discontinued RCTs were less likely to be published compared to completed trials. Reporting quality of trial protocols was not associated with premature discontinuation. In comparison to investigator-sponsored trials, industry-sponsored RCTs tend to perform better in prospectively registering trials, avoiding discontinuation due to poor recruitment, and making results available in trial registries.

Comparison with other studies

A systematic review and meta-analysis published in 2018 found that in different medical specialties, 2% to 79% of RCTs were not registered [21]. The study authors highlighted that the proportion of registered trials increased over time [21]. However, they only considered published RCTs, whereas we had access to the trial protocols and were also able to explore if unpublished results were made available in a registry. Overall, 6% of the RCTs from our sample were not registered. When separately assessing published and nonpublished RCTs, these proportions were 2% and 21%, respectively, indicating that nonregistration is more common among nonpublished RCTs. The proportion of prospectively registered RCTs was 84%. Other studies show a wide range of lower proportions (24% to 72%) [9,22], depending on medical specialties, time frame assessed, and journals considered for selecting included RCTs. We can only speculate why we found a higher rate of prospectively registered RCTs. Reasons might be that there was a general improvement over the last years and that the included countries from which we selected RECs might be more stringent in enforcing registration prior to patient recruitment.

Compared to a decade earlier [14], the proportion of prematurely discontinued RCTs did not change. Further, discontinued RCTs were more likely to remain unpublished in both our prior [14] and current study, and the most common reason for discontinuation remained poor recruitment. Publication rates between our cohorts showed improvement, with 59% of approved trials appearing in a peer-reviewed journal in our prior study [14] and 79% in our current analysis. When assessing the availability of results either as a publication or in clinical trial registry, 87% of study results were available (not assessed for RCTs approved in 2000 to 2003 [14]). In addition, both studies found that industry-sponsored trials published their results more frequently and were associated with lower rates of discontinued trials [14]. A 2018 systematic review concluded that industry-sponsored trial publications were more comprehensively reported than investigator-sponsored [23]. In line with these results, we found that the reporting quality in RCT study protocols approved in 2012 was better for industry-sponsored trials compared to investigator-sponsored RCTs. A systematic review by Schmucker and colleagues [24] revealed that 2 previous studies also assessed the publication rate of ethically approved RCT protocols. Both found low publication rates for protocols that were approved between 1988 and 1998 (i.e., 52%; 233/451; approved 1988 to 1998 in Switzerland [25]; 37%; 102/274 approved 1994 to 1995 in Denmark [26]). These findings are in line with our repeated metaresearch analysis, indicating that the publication rate has improved over the last decades.

Strengths and limitations

Strengths of our study include full access to the protocols of all trials approved by the collaborating RECs during the study period. We recruited reviewers with training in healthcare methodology to complete all data abstraction and considered only a limited number of variables in our regression models to reduce the chance of spurious associations. Further, we sampled trial protocols from the same RECs in both 2000 to 2003 and 2012 (with the exception of the added UK REC in 2012), which provides greater confidence in the shifts we observed regarding increased rates of RCT discontinuation and improved rates of publication over the past decade.

Our study has the following limitations: First, our sample size was modest, which may have limited the ability of our regression models to identify significant associations between protocol features and discontinuation or nonpublication. Second, of the 19 RCTs we classified as unregistered, the completion status of 11 was unclear. Thus, it is possible that some of those RCTs were never started and therefore the proportion of 6% unregistered RCTs may be smaller. Third, the SPIRIT checklist was created as a reporting guideline and not as a measurement tool for reporting quality [27]. However, we carefully operationalized the SPIRIT checklist and conducted various sensitivity analyses before using reporting quality estimates for the present study [15,17]. Fourth, we used trial protocols approved by RECs in Switzerland, the UK, Germany, and Canada, and the generalizability of our findings to RCT protocols approved by other RECs in these or other countries is uncertain. Fifth, regulatory aspects might have changed in some countries since 2012 (e.g., registration has been mandatory by law since 2014 in Switzerland, which had the highest proportion of unregistered trials [28,29]), hence it is possible that registration rates are higher nowadays.

Implications

Our study revealed encouraging results in terms of registration rates and making trial results available, but further efforts are still needed. Meerpohl and colleagues have developed 47 recommendations targeted at a variety of stakeholders [30]. Among others, they strongly recommend that legislators make trial registration mandatory, funding agencies request dissemination of all funded projects, and that RECs require trial registration before the recruitment of the first patient and request annual reports describing the dissemination of study results. Furthermore, publishing journals should remove barriers to publish negative or inconclusive results (e.g., from discontinued trials) and trial investigators should consequently make results available in trial registries [30]. Currently few investigator-sponsored RCTs make their results available in trial registries. The advantage of results posted in a trial registry over results reported in a published article may be the avoidance of spin [31]. Future research should address the current hurdles that exist among investigators to share study results in trial registries and how sharing results in registries could be promoted.

Other areas, such as discontinuation of RCTs due to preventable reasons and retrospective registration, need to be addressed too. Future research should assess if the rate of discontinued trials due to poor recruitment can be reduced with pilot or feasibility studies [18]. As stated by clinical trial registry representatives, trial registration and prospective trial registration should be enforced by publishing journals (including checking if trial registration exists) [32]. In case a trial was not prospectively registered, authors should at least explain in the published article why this was not done.

Conclusions

In our sample of RCTs approved by RECs from 4 countries, almost all were registered; however, 1 in 10 trials was registered retrospectively, which could result in methods being altered by study findings (e.g., changing the primary outcome [33]). Furthermore, 1 in 5 unpublished trials were not registered, and only 1 in 6 investigator-sponsored trials made results available in a trial registry. Higher reporting quality of trial protocols was positively associated with peer-reviewed publication, but not with prevention of trial discontinuation, highlighting the importance of feasibility assessments before embarking on a definitive trial. Despite a decade of efforts, premature trial discontinuation and nonpublication of RCTs remain common and comprise important targets to reduce waste in research.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist.

(DOCX)

Click here for additional data file.

Information on all participating research ethics committees.

(DOCX)

Click here for additional data file.

Procedure to receive more information about the trial status from ethical committees or by contacting principal investigators by sending them a survey through ethical committees.

(DOCX)

Click here for additional data file.

Survey to receive more information from investigators about the fate of their trial.

(DOCX)

Click here for additional data file.

Search strategy to identify corresponding full text publications.

(DOCX)

Click here for additional data file.

Code for analysis in Stata.

(DOCX)

Click here for additional data file.

Baseline characteristics of included randomized controlled trials stratified by country of ethical approval.

(DOCX)

Click here for additional data file.

Registration, completion, and publication status of randomized controlled trials approved in 2012 stratified by country of ethical approval.

(DOCX)

Click here for additional data file.

Association between completion of a randomized controlled trial and making the results available.

(DOCX)

Click here for additional data file.

Nonpublication and discontinuation in protocols approved by ethical committees in 2012 compared to protocols approved between 2000 and 2003.

(DOCX)

Click here for additional data file.

Factors associated with (a) publishing main results in a peer-reviewed journal; (b) discontinuation of trials due to poor recruitment; and (c) discontinuation of trials due to preventable reasons. UK samples excluded because variable “CTU/CRO support” was not assessed.

(DOCX)

Click here for additional data file.

13 Dec 2021

Dear Dr Speich,

Thank you for submitting your manuscript entitled "Non-registration, discontinuation, and non-publication of randomized trials: A repeated meta-research analysis" for consideration by PLOS Medicine.

I am writing to let you know that we would like to send your submission out for external peer review.

However, before we can send your manuscript to reviewers, we need you to complete your submission by providing the metadata that is required for full assessment. To this end, please login to Editorial Manager where you will find the paper in the 'Submissions Needing Revisions' folder on your homepage. Please click 'Revise Submission' from the Action Links and complete all additional questions in the submission questionnaire.

Please re-submit your manuscript within two working days, i.e. by Dec 15 2021 11:59PM.

Login to Editorial Manager here: https://www.editorialmanager.com/pmedicine

Once your full submission is complete, your paper will undergo a series of checks in preparation for peer review. Once your manuscript has passed all checks it will be sent out for review.

Feel free to email us at plosmedicine@plos.org if you have any queries relating to your submission.

Kind regards,

Callam Davidson

Associate Editor

PLOS Medicine

12 Jan 2022

Dear Dr. Speich,

Thank you very much for submitting your manuscript "Non-registration, discontinuation, and non-publication of randomized trials: A repeated meta-research analysis" (PMEDICINE-D-21-05025R1) for consideration at PLOS Medicine.

Your paper was evaluated by an associate editor and discussed among all the editors here. It was also discussed with an academic editor with relevant expertise, and sent to independent reviewers, including a statistical reviewer. The reviews are appended at the bottom of this email and any accompanying reviewer attachments can be seen via the link below:

[LINK]

In light of these reviews, I am afraid that we will not be able to accept the manuscript for publication in the journal in its current form, but we would like to consider a revised version that addresses the reviewers' and editors' comments. Obviously we cannot make any decision about publication until we have seen the revised manuscript and your response, and we plan to seek re-review by one or more of the reviewers.

In revising the manuscript for further consideration, your revisions should address the specific points made by each reviewer and the editors. Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments, the changes you have made in the manuscript, and include either an excerpt of the revised text or the location (eg: page and line number) where each change can be found. Please submit a clean version of the paper as the main article file; a version with changes marked should be uploaded as a marked up manuscript.

In addition, we request that you upload any figures associated with your paper as individual TIF or EPS files with 300dpi resolution at resubmission; please read our figure guidelines for more information on our requirements: http://journals.plos.org/plosmedicine/s/figures. While revising your submission, please upload your figure files to the PACE digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at PLOSMedicine@plos.org.

We hope to receive your revised manuscript by Feb 02 2022 11:59PM. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

We ask every co-author listed on the manuscript to fill in a contributing author statement, making sure to declare all competing interests. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. If new competing interests are declared later in the revision process, this may also hold up the submission. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT. You can see our competing interests policy here: http://journals.plos.org/plosmedicine/s/competing-interests.

Please use the following link to submit the revised manuscript:

https://www.editorialmanager.com/pmedicine/

Your article can be found in the "Submissions Needing Revision" folder.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

We look forward to receiving your revised manuscript.

Sincerely,

Callam Davidson,

Associate Editor

PLOS Medicine

plosmedicine.org

-----------------------------------------------------------

Comments from the academic editor:

I would suggest that the authors (1) better describe the sample so as to help understand if it is generalizable and (2) dig into the 'research waste' idea a bit more, since a major limitation of the study is any details on the RCTs themselves to understand how many patients accrued, what the flaws in the study were, etc. If no patients accrued, then where is the 'waste'?

Requests from the editors:

Please report your SR/MA according to the PRISMA guidelines provided at the EQUATOR site.

http://www.equator-network.org/reporting-guidelines/prisma/

Please provide the completed PRISMA checklist. I appreciate that the nature of the study means some items may be non-applicable.

When completing the checklist, please use section and paragraph numbers, rather than page numbers.

Please add the following statement, or similar, to the Methods: "This study is reported as per the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline (S1 Checklist)."

If possible, please update your search end date from March 2021 to the present time.

The Data Availability Statement (DAS) requires revision. Please also include an appropriate contact (web or email address) for inquiries (this cannot be a study author).

Abstract Methods and Findings:

* Please include the study design.

* Please include the key databases searched.

* Please include relevant p-values when quantifying your results.

* In the last sentence of the Abstract Methods and Findings section, please describe the main limitation(s) of the study's methodology.

Abstract Conclusions:

* Please address the study implications without overreaching what can be concluded from the data; the phrase "In this study, we observed ..." may be useful.

Please remove the Funding and Competing Interests sections from the main text, this information is captured as metadata from the submission form.

At this stage, we ask that you include a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract. Please see our author guidelines for more information: https://journals.plos.org/plosmedicine/s/revising-your-manuscript#loc-author-summary

Please define the abbreviations in Figure 1.

Please include the relevant p values for the results presented on lines 262 and 267.

Line 296: Please do not present new data in the Discussion, relocate this to the Results.

Please expand your Discussion to include implications and next steps for research, clinical practice, and/or public policy (this should follow your Strengths and Limitations section and precede your Conclusions).

Please relocate the Patient and Public Involvement section to the Methods.

Reference #5: Please remove the COI information from this reference.

Supplementary materials: Please update your supplementary references to follow our reference guidelines (https://journals.plos.org/plosmedicine/s/submission-guidelines#loc-references)

Comments from the reviewers:

Reviewer #1: I thank the editor to having the opportunity to review this manuscript.

The authors report the fate (with regards to completion, quality, and publication) of 327 randomized controlled trials (RCTs) approved by research ethics committees in 4 countries in 2012. The assessment was conducted in 2021.

This work is of high interest since there are very few studies that have access to the protocols approved by RECs ─most similar analysis only have access to the published (on articles or posted on registries) results. The results reported are also of interest since the same authors conducted a similar analysis with RCTs approved in 2003-2004 allowing to compare the outcome of both cohorts. The manuscript is very easy to follow, well written and focused. I would stress the highly useful data reported regarding reasons for discontinuation, which is hardly found in other reports.

This manuscript should affect public health policy with regards to the low fulfilment of posting trial results of investigator-initiated trials on trial registries. RECs should be involved on this.

I have no major concerns on how the data was presented and analyzed, except for the following: In Acknowledgement, one can read the RECs involved. However, it is not clear how the UK NHSHRA provided the 89 RCTs included in this analysis. I would bet that there were many more than 89 RCTs approved in the UK in 2012; then, which were the criteria to choose those 89 RCTs? This is a very relevant information that readers should know and that should be included in Methods.

In addition, I would like to stress that mentioning that investigator-initiated trials usually fail on having trial results posted on the registries should be included in both the abstract and conclusion. This should be one of the main messages of this report.

Below I mentioned a few suggestions that, I believe, would be of interest to many readers.

Minor comments:

Seems to me that this research includes RCTs on medicines only (Industry-sponsored may also include trials on devices, but I believe that the research was conducted on RCTs assessing medicines only). That the research was conducted on medicines RCTs must be stated in the title, and in other appropriate sections of the manuscript.

Abstract

The abstract should report the number (and percentage) of RCTs by trial phase (2, 3 or 4) and the number (and percentage) of RCTs that were not published but of which the results are posted on the registries, and on RCTs that were published and posted the results on the registries. (This latter is relevant, since many RCTs could be published in subscription-only journals, so access to the results is limited to a minority of interested parties).

Reporting the clinical development phases (2, 3 and 4) of the RCTs assessed is relevant, since the clinical usefulness of phase 3 and 4 RCTs is much higher than of phase 2 trials.

Introduction

In addition to the ICJME requirement of trial registration, authors should mention that both the EU and North American regulations (Canada, USA) requested registration of trials on medicines. I do not know whether the Swiss regulation requested registration as of 2012, but is likely it did.

Methods

Authors should report the number of research ethics committees by country involved in this research.

Lines 210-211.- Authors should inform who, among the authors, were the 'pairs of reviewers'.

Among those RCTs that were registered in more than one registry, how many posted the results in all registries?

Analysis

(General comment: Since authors had complete data for all RCTs, I do believe it is inappropriate to present confidence intervals as if these were point estimates randomly sampled from a larger population. Do authors agree with this?)

Line 221.- Are the pre-specified independent variables in their models based on the previous analysis by the same authors or by published reports of other investigators?

Lines 225-228.- What was the reason supporting the inclusion of a post-hoc analysis? That this analysis was done excluding UK RCTs (as reported in line 264) should be included here. I wonder how relevant this post-hoc analysis is.

Line 230.- p>0.05 should be p<0.05 (?)

Results

Lines 240-241.- The number of RCTs approved by REC and clinical development phases (2, 3 or 4) should be included in Table 1. Although of less relevance, I would also suggest reporting whether the trials were conducted in hospitals and/or primary care centers ─ this info might be difficult to know in some trials.

Line 250.- 4 published trials were not registered. It would be of interest to know whether the journals, at the time of publication, were followers of the ICMJE requirements or not as per the info included in their websites.

Line 264.- 'A separate analysis...' should read as follows: 'A separate post-hoc analysis…'

Lines 267-268: I would suggest deleting 'compared to discontinued trials' that is mentioned at the beginning of the sentence (line 266)

Discussion

Line 279.- I do believe that posting the results on the registry is even more relevant than publishing them (see above). Hence, I strongly suggest mentioning that industry-sponsored trials tend to perform better in this regard as well. Maybe the emphasis should be on the bad performance of investigator-sponsored trials, rather than in the better performance of industry-sponsored trials.

Lines 319-321.- I do not see the need to add this 4th limitation…since only 6% of RCTs were not registered (!). In addition, mentioning it could mislead readers as, I do believe, that in 2012 all 4 countries where the RCTs were conducted were requesting prospective registration of trials ─as well as the ICMJE requirements, mentioned in the Introduction.

Line 326.- One or two of the many references that support the statement 'e.g., changing the primary outcome' should be included here.

Line 326.- The statement '1 in 5 unpublished trials were not registered' has not enough relevance to be mentioned here (conclusions), since 94% of trials were registered (!).

Table 2.- In the EU, to have a RCT authorized by the regulatory agency of the country involved, an EudraCT number must have been given before study start. Hence, I would expect that the 20 RCTs that were not registered and those 34 that were retrospectively registered should have been approved by non-EU RECs. Am I correct? I do believe that the country (ies) in which these 54 RCTs were approved should be included as a footnote. This is a relevant piece of information.

Table 3.- An additional column reporting 'results available in a peer reviewed publication + clinical trial register' would be of great interest. Although regulations request to post clinical trial results on the registries, it is well known ─and the results of this research show─ that many sponsors publish the results in articles but forget to post them on the registry.

Reviewer #2: See attachment

Michael Dewey

Reviewer #3: Thank you for the invitation to review this rigorous and interesting research article. In this manuscript, Speich et al. focus on an important issue - rates of completion and publication of clinical trials. As the authors note, trial discontinuation and non-publication contribute to substantial research waste. Over the past two decades, clinical trial registries have been implemented to improve transparency and journal policies related to trial registration have been strengthened. This evaluation builds upon a previous evaluation by similar authors, which found that one-quarter of 1017 RCTs approved between 2000-2003 were discontinued prematurely; 44% remained unpublished at a median of 12 years follow-up. In the current evaluation, which included 327 RCTs protocols approved in 2012 by research ethics committees in Switzerland, UK, Germany, and Canada, the authors found that 31% were discontinued prematurely; 24% remained unpunished at 9 years follow-up. Overall, the authors conclude the premature trial discontinuation has not improved in the past decade. Although non-publication of RCTs has improved, opportunities exist for improvement. Overall, this is an important manuscript, which builds upon a prior, seminal evaluation. The findings remain relevant to a broad clinical audience.

Here are a number of suggestions for the authors and editors to consider:

First, as the authors note, the sample size is rather small. In particular, the sample is nearly a third of the size of the previous evaluation. In the published protocol, the authors note that they identified 450 protocols approved in 2012 and 402 protocols approved in 2016. Is there a reason that the 2016 protocols were not considered? In particular, the authors note that they "propose to compare RCT cohorts from 2012 and 2016 with RCTs approved 2000-2003…". If the authors have these data collected, a larger sample size would also strengthen the logistic regression-based analyses conducted, which may have been underpowered.

Second, the authors may want to consider updating the discussion section to include a brief section with potential implications. Currently, the authors recap the main findings (paragraph 1), provide a comparison with other studies (paragraphs 2 and 3), and present strengths and limitations. However, opportunities exist to mention several implications and/or recommendations, given that there has been little improvement over the past two decades.

Third, it appears as if new data are reported in the Discussion section that are not included in the Results section (e.g., Supplementary Table 5). In particular, the authors present a table comparing study-protocols approved in 2012 with study-protocols approved in 2000-2003 in the Discussion section (data from the previous publication). As the authors note in the protocol, one of their objectives was to "compare the RCT cohorts from 2012 and 2016 with RCTs approved 2000-2003". However, these comparisons are not described in the Method or reported in the Results of the manuscript. To improve clarity, the authors could consider updating the Results section to include information from the formal comparison between the two samples, especially if they hope to discuss these data in the Discussion section.

Lastly, some of the results from the reported multivariable analyses may be difficult to interpret. In particular, those focused on odds ratios based on increments of 10% proportion of adherence. Why were increments of 10% proportion adherence selected? ("guidelines were less likely to not published their results in a peer-reviewed journal (OR 0.73 (in increments of 10% proportion adherence); 95%CI: 0.57-0.93)). This feel rather arbitrary, especially given that the SPIRIT checklist has 33 items.

Specific comments:

Page 5, Abstract:

Line 139: Overall, the abstract is very clear and concise. The odds ratio (0.73 for each 10% increment in the proportion of SPIRIT items met) was the only part that may need some refining (please see above).

Line 141: Perhaps the authors could rephrase "Non-publication of RCTs has improved" to increase clarity. For instance, the proportion of RCTs that remain unpublished has decreased. I think the "non-publication" and has "improved" makes it challenging to identify whether it is good or bad.

Introduction:

Line 150: The authors note that "A number of studies have explored the proportion of published RCTs that are registered (5-8); however, such investigations use published RCTs - and not RCT protocols - as the denominator and thus unable to assess publication bias." However, it is worth noting that previous studies have also looked at the proportion of registered RCTs that are published (e.g., https://www.bmj.com/content/344/bmj.d7292 ; https://jamanetwork.com/journals/jama/fullarticle/1840223 ; https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1000144 ; etc. [disclosure: I am not a co-author on any of these manuscripts]. These studies assess publication bias, so it is worth mentioning these efforts and the corresponding results.

Methods:

Line 166: It was great to see the authors transparently report that their analysis of discontinuation due to preventable reasons was not prespecified. What prompted the additional analysis? Perhaps, the authors could include a short paragraph (supplementary materials?) outlining any changes from the published protocol?

Line 191: I may have missed this, but how were the trials matched if they did not have the same title, abbreviation, etc?

Results:

Line 227+: How did these characteristics compare to the previous sample? Do the authors have information about therapeutic areas?

I may have missed this in the Results, but did the authors report the following in the text:

1. How many trials were prematurely discontinued were not published?

2. The proportion of trials that remained unpublished at 9 years follow-up? This shows up in the Discussion, but I did not see it in the Results text.

3. The proportion of prospectively registered RCTs (83%). This shows up in the discussion but I did not see it in the Results text.

Discussion:

The authors not that "of the 20 RCTs we classified as unregistered, the status of 11 was unclear". Was this reported in the Results text?

Figures/Tables:

Table 3: Perhaps the authors could stratify the rows in the table, since all the rows before the last one add up to 100%.

Reviewer #4: Many thanks for the opportunity to review this manuscript. Overall, this seems to be a very methodologically solid piece of work, meets the stated aims, and aligns with the published protocol. I also commend the authors on being clear about deviations throughout their manuscript. My comments are generally around minor clarifications and elaborations.

Abstract:

Page 6, lines 135-136: "We recorded whether RCTs were registered, discontinued (including reason for discontinuation), or published." Based on my understanding, this should say "and published" not "or published."

Introduction:

I'd like to see a little bit more discussion of the overall ASPIRE project and the role of this study within the larger project. No need for excruciating detail, as I know you clearly reference the overall project protocol, but the context is useful for readers to understand where this fits into the larger project and why the work was undertaken.

Methods:

Page 8 Line 189-193: Worth noting the ICTRP is not actually a registry but a meta-registry of entries from other registries (including the other ones you looked at). Also, I assume you actually used the EUCTR and not the back-end EudraCT system (which do not overlap perfectly as some things on EudraCT are not made public on the EUCTR).

Why did you include the additional post-hoc independent variable? Seems like some reasoning or justification should be provided.

Did you check for/account for duplicate registration?

Can the authors please note their data sharing plans in the manuscript? While some information from the RECs may not be able to be shared, it seems to me that the overall dataset, or at the very least any analytic code, should be able to be shared. I strongly encourage the authors to do so or explain why they are not sharing their code and data.

Results:

One cut of the results I think is clearly missing is how many results were unique to a registry, especially among discontinued trials. Since publication in a peer reviewed journal might at times be difficult for a prematurely ended study, this is where registries really have the ability to shine as a dissemination mechanism. It seems you have the ability to offer some data here and I think it would be valuable to present.

Since only 30.3% of your sample actually reported a recruitment projection in the protocol (per Table 1), how did you then use this as an independent variable in your regressions? Seems like you would have had a lot of missing data to contend with and don't describe this anywhere. Perhaps I'm misunderstanding?

Discussion:

The context of this paper in relation to existing work is very sparse. You cite a bit about registration and then compare your findings to your previous findings. I would like to see a bit more depth in terms of delving into the existing literature related to your findings, for instance on reporting of industry vs. non-industry findings. In addition, I don't see a citation of Schmucker et al. 2014 anywhere (https://pubmed.ncbi.nlm.nih.gov/25536072/) which seems a very important citation as it is, to my knowledge, the most recent systematic review that includes an analysis of studies of eventual publication of trials approved by RECs.

I'd also like to see some more engagement with the implications of these findings in the Discussion. The whole section feels a bit empty. I'm left, as a reader, with little indication as to what these findings mean for the larger clinical trial landscape and what changes the authors might like to see or recommend or reflect on what has been done. The most we get is a single sentence suggesting more feasibility studies.

Another potential limitation is your use of the SPIRIT checklist in a way that assumes equivalence of the points. Additionally, there have been objections raised to using checklists as indicators of quality as the authors have done so here (https://onlinelibrary.wiley.com/doi/full/10.1002/hsr2.165).

Minor Comments:

Page 8 Line 181: "Pairs of reviewers determined, independently and in duplicate" redundant to say pairs worked in duplicate.

Just to make the authors aware there are still some track changes in the supplement.

Any attachments provided with reviews can be seen via the following link:

[LINK]

Submitted filename: speich.pdf

Click here for additional data file.

24 Feb 2022

Submitted filename: p-b-p response 2022-02-23.docx

Click here for additional data file.

24 Mar 2022

Dear Dr. Speich,

Thank you very much for re-submitting your manuscript "Non-registration, discontinuation, and non-publication of randomized trials: A repeated meta-research analysis" (PMEDICINE-D-21-05025R2) for review by PLOS Medicine.

I have discussed the paper with my colleagues and the academic editor and it was also seen again by three reviewers. I am pleased to say that provided the remaining editorial and production issues are dealt with we are planning to accept the paper for publication in the journal.

The remaining issues that need to be addressed are listed at the end of this email. Any accompanying reviewer attachments can be seen via the link below. Please take these into account before resubmitting your manuscript:

[LINK]

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

In revising the manuscript for further consideration here, please ensure you address the specific points made by each reviewer and the editors. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments and the changes you have made in the manuscript. Please submit a clean version of the paper as the main article file. A version with changes marked must also be uploaded as a marked up manuscript file.

Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. If you haven't already, we ask that you provide a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract.

We hope to receive your revised manuscript within 1 week. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

We ask every co-author listed on the manuscript to fill in a contributing author statement. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT.

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript.

Please note, when your manuscript is accepted, an uncorrected proof of your manuscript will be published online ahead of the final version, unless you've already opted out via the online submission form. If, for any reason, you do not want an earlier version of your manuscript published online or are unsure if you have already indicated as such, please let the journal staff know immediately at plosmedicine@plos.org.

If you have any questions in the meantime, please contact me or the journal staff on plosmedicine@plos.org.

We look forward to receiving the revised manuscript by Mar 31 2022 11:59PM.

Sincerely,

Callam Davidson,

Associate Editor

PLOS Medicine

plosmedicine.org

------------------------------------------------------------

Requests from Editors:

Data Availability: Thank you for providing more details in your statement. I think it is important (if possible) to have the full details of the participating ethics committees in the manuscript – could these be provided in the Supplementary Materials? This item could be cited at around line 214.

Line 137: Please begin this sentence ‘Study limitations include’ or similar.

Please reorder you Appendices items such that they appear in sequential order throughout the text (Appendices 2 and 3 currently are cited before Appendix 1).

Please cite Appendix 4 (code) in the Methods.

Appendix Table 4: I can’t see the relevant flag for footnote c?

Please relocate the ‘Registration and Protocol’ section to the Methods. The ‘Availability of Data’ and ‘Authors’ Contributions’ can be removed as these are captured as metadata via the Submission Form.

Comments from Reviewers:

Reviewer #1: I think the authors have appropriately addressed my suggestions or have provided plausible/correct answers

Reviewer #2: The authors have addressed all my points especially in the toning down of the treatment of discontinued trials.

Michael Dewey

Reviewer #4: Many thanks for the opportunity to review a revised version of this manuscript. The authors have adequately addressed all of my points. I present some very minor points related to clarifications or copy-edits below but shouldn't need to see a revised version of the entire manuscript as these should mostly be easily addressed.

Line 180-181: Those requirements also often included requirements not just to register but to report results to the registry.

Line 188: "For example, if a RCT is discontinued due to slow participant recruitment before the planned sample size is reached, the data typically do not allow to answer the primary research question." - I think the last part of the sentence might be missing a word? "…the data typically do not allow to answer…" doesn't make sense.

Lines 189-190: "Hence, it is crucial that all RCT results (also from discontinued trials) are made available so that evidence is not lost and no resource waste minimized." - I think the word "no" in "no resource waste minimized" should be deleted?

Line 232: "...European Union Clinical Trial Registry (EudraCT)," change acronym to EUCTR.

Lines 232-233: "(4) the International Standard Randomised Controlled Trial Number (ISRCTN) registry." - While that is what ISRCTN used to stand for, technically it doesn't stand for anything anymore. The name of the registry is just the ISRCTN. (See: https://www.isrctn.com/page/about)

Line 313-315: "Compared to discontinued trials, completed RCTs were more likely to be published (OR 7.08; 95%CI: 3.35-15.52; p<0.001) and making their results available (OR 6.06; 95%CI: 1.94-22.24; p<0.001 Appendix Table 3)." - Should read "and make their results available."

Lines 323-326: "Multivariable analyses indicated that RCTs with more complete protocol reporting according to SPIRIT [16, 17] guidelines were less likely to not publish their results in a peer-reviewed journal (OR 0.71 [in increments of 10% proportion of adherence]; 95%CI: 0.55-0.92; p=0.009; 326 Table 4)." - This is an awkward sentence construction with the double negative ("...less likely to not publish…"). Wouldn't it be accurate to say "RCTs with more complete protocol reporting according to SPIRIT were more likely to publish their results"?

Lines 345-346: When discussion the context of unregistered RCTs, this review might be worth considering/including (perhaps in addition whatever individual studies you consider most notable): https://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-018-1168-6

Lines 365-366: "Furthermore, it was shown several times that industry-sponsored trial publications are more comprehensively reported [19], what we also found is true for trial protocols approved in 2012." - This is an awkward sentence construction.

Line 379: Typo: "RTC" should say "RCT"

One other implication I'd like to potentially see discussed:

Lines 348-350: Overall, 6% of the RCTs from our sample were not registered. When separately assessing published and non-published RCTs, these proportions were 2% and 21%, respectively, indicating that non-registration is more common amongst non-published RCTs.

This seems to me to be a very important finding. If trials that don't register, also don't report, they are essentially invisible to everyone. This is the absolute worst outcome for everyone involved. That is a massive discrepancy in terms of percent between the two is probably worth highlighting.

Any attachments provided with reviews can be seen via the following link:

[LINK]

1 Apr 2022

Dear Dr Speich,

On behalf of my colleagues and the Academic Editor, Dr Aaron S Kesselheim, I am pleased to inform you that we have agreed to publish your manuscript "Non-registration, discontinuation, and non-publication of randomized trials: A repeated meta-research analysis" (PMEDICINE-D-21-05025R3) in PLOS Medicine.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. Please be aware that it may take several days for you to receive this email; during this time no action is required by you. Once you have received these formatting requests, please note that your manuscript will not be scheduled for publication until you have made the required changes.

When making the formatting changes please also make the following update:

* Include a reference for the citation 'Gryaznov et al., BMJ Open, in press' - per our submission guidelines, accepted, unpublished articles can be referenced in the bibliography in the same style as published articles, but substituting “Forthcoming” for page numbers or DOI.

In the meantime, please log into Editorial Manager at http://www.editorialmanager.com/pmedicine/, click the "Update My Information" link at the top of the page, and update your user information to ensure an efficient production process.

PRESS

We frequently collaborate with press offices. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximise its impact. If the press office is planning to promote your findings, we would be grateful if they could coordinate with medicinepress@plos.org. If you have not yet opted out of the early version process, we ask that you notify us immediately of any press plans so that we may do so on your behalf.

We also ask that you take this opportunity to read our Embargo Policy regarding the discussion, promotion and media coverage of work that is yet to be published by PLOS. As your manuscript is not yet published, it is bound by the conditions of our Embargo Policy. Please be aware that this policy is in place both to ensure that any press coverage of your article is fully substantiated and to provide a direct link between such coverage and the published work. For full details of our Embargo Policy, please visit http://www.plos.org/about/media-inquiries/embargo-policy/.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Thank you again for submitting to PLOS Medicine. We look forward to publishing your paper.

Sincerely,

Callam Davidson

Associate Editor

PLOS Medicine

Table 1

Baseline characteristics of included RCTs.

	Industry-sponsored RCTs (n = 179)	Investigator-sponsored RCTs (n = 147)	All RCTs (n = 326)
Planned sample size, median (IQR)a	360 (144–800)	150 (60–426)	250 (100–600)
Proportion of adequately reported SPIRIT items in protocol, median (IQR)	0.74 (0.67–0.79)	0.63 (0.54–0.70)	0.69 (0.61–0.77)
Single center vs. multicenter
Single center	6 (3.4%)	54 (36.7%)	60 (18.4%)
Multicenter	173 (96.7%)	93 (63.3%)	266 (81.6%)
Study design
Parallel	171 (95.5%)	125 (85.0%)	296 (90.8%)
Crossover	4 (2.2%)	9 (6.1%)	13 (4.0%)
Factorial	3 (1.7%)	7 (4.8%)	10 (3.1%)
Cluster	0 (0%)	4 (2.7%)	4 (1.2%)
Otherb	1 (0.6%)	2 (1.4%)	3 (0.9%)
Placebo controlled	94 (52.5%)	37 (25.2%)	131 (40.2%)
Recruitment projection reported in protocol	43 (24.0%)	56 (38.1%)	99 (30.4%)
Research ethics committee approval
Switzerland	87 (48.6%)	78 (53.1%)	165 (50.6%)
United Kingdom	45 (25.1%)	44 (29.9%)	89 (27.3%)
Germany	26 (14.5%)	11 (7.5%)	37 (11.4%)
Canada	21 (11.7%)	14 (9.5%)	35 (10.7%)
Intervention
Drug	152 (84.9%)	55 (37.4%)	207 (63.5%)
Medical devices	20 (11.2%)	33 (22.5%)	53 (16.3%)
Surgical	2 (1.1%)	18 (12.2%)	20 (6.1%)
Behavioral	0 (0.0%)	19 (12.9%)	19 (5.8%)
Otherc	5 (2.8%)	22 (15.0%)	27 (8.3%)
Medical field
Oncology	44 (24.6%)	16 (10.9%)	60 (8.4%)
Surgical	12 (6.7%)	25 (17.0%)	37 (11.4%)
Cardiovascular	19 (10.6%)	11 (7.5%)	30 (9.2%)
Neurology	17 (9.5%)	8 (5.4%)	25 (7.7%)
Otherd	87 (48.6%)	87 (59.2%)	174 (53.4%)

aMissing data for planned sample size for 4 trial protocols was inserted from other sources (i.e., peer-reviewed publication; n = 3; trial registry; n = 1).

bSplit body (n = 2), parallel group with 2 consecutive randomizations (n = 1).

cDietary supplement, radiation, and rehabilitation.

dAnesthetics, dermatology, endocrinology, gastro/intestinal, gynecology, hematology, infectious diseases, intensive care, nephrology, orthopedics, pediatrics, psychiatry, respiratory, rheumatology, and ophthalmology.

IQR, interquartile range; RCT, randomized clinical trial; SPIRIT, Standard Protocol Items: Recommendations for Interventional Trials [19,20].

Table 3

Reasons for trial discontinuation and proportion of results available.

Reasons for discontinuation	All Discontinued RCTs (n = 98)	Industry-sponsored discontinued RCTs (n = 57)	Investigator-sponsored discontinued RCTs (n = 41)	Results available as a peer-reviewed publication	Results available in clinical trial register	Results in peer-reviewed publication and clinical trial register	Results not available
Poor recruitmenta	36 (37%)	16 (28%)	20 (49%)	21 (58%)	16 (44%)	11 (31%)	10 (28%)
Futility	16 (16%)	15 (26%)	1 (2%)	11 (69%)	13 (81%)	9 (56%)	1 (6%)
Harm	6 (6%)	5 (9%)	1 (2%)	5 (83%)	6 (100%)	5 (83%)	0 (0%)
Organizational/strategic reasons	6 (6%)	6 (11%)	0 (0%)	3 (50%)	4 (67%)	1 (17%)	0 (0%)
Benefit	3 (3%)	2 (4%)	1 (2%)	3 (100%)	2 (67%)	2 (67%)	0 (0%)
External evidence	3 (3%)	0 (0%)	3 (7%)	3 (100%)	1 (33%)	1 (33%)	0 (0%)
Limited resources	1 (1%)	0 (0%)	1 (2%)	1 (100%)	0 (0%)	0 (0%)	0 (0%)
Unclear	27 (28%)	13 (23%)	14 (34%)	19 (70%)	12 (44%)	6 (22%)	2 (7%)
Discontinued due to a preventable reasonb	70 (71%)	35 (61%)	35 (85%)	44 (63%)	32 (46%)	18 (26%)	12 (17%)

aTwo studies that stated slow recruitment as reason for discontinuation mentioned in addition another reason (i.e., organizational/strategic reasons n = 1; external evidence n = 1).

bCounting the following reasons as not preventable: futility, harm, benefit, external evidence. Counting the following as preventable: poor recruitment, organizational/strategic reasons, limited resources, and unclear reasons (assuming that discontinuation due to unclear reasons was mainly due to non-data-driven reasons [18]).

RCT, randomized clinical trial.

Table 4

Factors associated with (a) publishing main results in a peer-reviewed journal; (b) discontinuation of trials due to poor recruitment; and (c) discontinuation of trials due to preventable reasons.

Characteristics			Univariable			Multivariable
			OR	95% CI	P value	OR	95% CI	P value
Nonpublication in a peer-reviewed journal	RCT not published in a peer-reviewed journal (n = 70)	RCTs published in a peer-reviewed journal (n = 256)
Proportion of adequate SPIRIT reporting, median (IQR)a	0.66 (0.53, 0.73)	0.70 (0.62, 0.78)	0.69	0.57–0.84	<0.001	0.71	0.55–0.92	0.009
Planned target sample size, median (IQR)b	146 (60, 288)	315 (109, 719)	0.99	0.97–1.01	0.215	0.99	0.98–1.01	0.377
Placebo controlled (vs. not placebo controlled)	30/70 (42.9%)	101/256 (39.5%)	1.15	0.67–1.97	0.607	1.48	0.82–2.66	0.193
Single center (vs. multicenter)	20/70 (28.6%)	40/256 (15.6%)	2.26	1.16–4.01	0.015	1.35	0.64–2.86	0.434
Reported recruitment projection	15/70 (21.4%)	84/256 (32.8%)	0.56	0.30–1.05	0.069	0.75	0.38–1.49	0.409
Industry sponsorship	33/70 (47.1%)	146/256 (57.0%)	0.67	0.40–1.14	0.142	1.03	0.51–2.06	0.937
Discontinued due to poor recruitment	RCTs discontinued due to poor recruitment (n = 36)	RCTs not discontinued due to poor recruitment (n = 265) c
Proportion of adequate SPIRIT reporting, median (IQR)a	0.66 (0.60, 0.75)	0.70 (0.63,0.78)	0.85	0.64–1.13	0.261	0.98	0.69–1.40	0.905
Planned target sample size, median (IQR)b	135 (79, 413)	300 (108, 720)	0.94	0.88–1.02	0.133	0.95	0.89–1.02	0.159
Placebo controlled (vs. not placebo controlled)	15/36 (41.7%)	109/265 (41.1%)	1.02	0.50–2.07	0.951	1.32	0.62–2.81	0.475
Single center (vs. multicenter)	8/36 (22.2%)	39/265 (14.7%)	1.66	0.70–3.90	0.248	0.93	0.34–2.50	0.883
Reported recruitment projection	11/36 (30.6%)	78/265 (29.4%)	1.05	0.49–2.25	0.890	1.08	0.43–2.52	0.862
Industry sponsorship	16/36 (44.4%)	160/265 (60.4%)	0.53	0.26–1.05	0.072	0.54	0.22–1.30	0.170
Discontinued due to preventable reasons	RCTs discontinued due to preventable reason (n = 70) d	RCTs not discontinued due to preventable reason (n = 231) c , d
Proportion of adequate SPIRIT reporting, median (IQR)a	0.68 (0.61, 0.73)	0.70 (0.62, 0.78)	0.82	0.66–1.02	0.080	0.94	0.72–1.24	0.668
Planned target sample size, median (IQR)b	169 (90, 500)	315 (110, 718)	1.00	0.99–1.01	0.549	0.99	0.99–1.01	0.747
Placebo controlled (vs. not placebo controlled)	28/70 (40.0%)	96/231 (41.6%)	0.94	0.54–1.62	0.816	1.01	0.59–1.89	0.859
Single center (vs. multicenter)	14/70 (20.0%)	33/231 (14.3%)	1.50	0.75–3.00	0.251	1.02	0.46–2.27	0.960
Reported recruitment projection	15/70 (21.4%)	74/231 (32.0%)	0.58	0.31–1.09	0.091	0.56	0.28–1.12	0.101
Industry sponsorship	35/70 (50.0%)	141/231 (61.0%)	0.64	0.37–1.09	0.102	0.63	0.32–1.24	0.183

aIn increments of 10%.

bIn increments of 100.

cStudies with unclear discontinuation status excluded.

dCounting the following reasons as not preventable: futility, harm, benefit, external evidence. Counting the following as preventable: poor recruitment, organizational/strategic reasons, limited resources, and unclear reasons (assuming that discontinuation due to unclear reasons was mainly due to non-data-driven reasons [18]).

CI, confidence interval; IQR, interquartile range; OR, odds ratio; RCT, randomized clinical trials; SPIRIT, Standard Protocol Items: Recommendations for Interventional Trials [19,20].