Literature DB >> 35613804

Reporting quality of clinical trial protocols: a repeated cross-sectional study about the Adherence to SPIrit Recommendations in Switzerland, CAnada and GErmany (ASPIRE-SCAGE).

Dmitry Gryaznov1, Belinda von Niederhäusern2,3, Benjamin Speich1,4, Benjamin Kasenda1,5, Elena Ojeda-Ruiz1,6, Anette Blümle7,8,9, Stefan Schandelmaier1,10, Dominik Mertz10,11, Ayodele Odutayo4, Yuki Tomonaga12, Alain Amstutz1,13, Christiane Pauli-Magnus2, Viktoria Gloy1, Szimonetta Lohner14, Karin Bischoff8,9, Katharina Wollmann8,9, Laura Rehner9,15, Joerg J Meerpohl8,9, Alain Nordmann1, Katharina Klatte2, Nilabh Ghosh16, Ala Taji Heravi1, Jacqueline Wong10, Ngai Chow10,17, Patrick Hong10,18, Kimberly A McCord-De Iaco1, Sirintip Sricharoenchai1, Jason W Busse10,19, Arnav Agarwal10, Ramon Saccilotto1, Matthias Schwenkglenks12,20, Giusi Moffa1,21, Lars Hemkens1,22,23, Sally Hopewell4, Erik Von Elm24, Matthias Briel25,10.   

Abstract

OBJECTIVES: Comprehensive protocols are key for the planning and conduct of randomised clinical trials (RCTs). Evidence of low reporting quality of RCT protocols led to the publication of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) checklist in 2013. We aimed to examine the quality of reporting of RCT protocols from three countries before and after the publication of the SPIRIT checklist.
DESIGN: Repeated cross sectional study.
SETTING: Swiss, German and Canadian research ethics committees (RECs). PARTICIPANTS: RCT protocols approved by RECs in 2012 (n=257) and 2016 (n=292). PRIMARY AND SECONDARY OUTCOME MEASURES: The primary outcomes were the proportion of reported SPIRIT items per protocol and the proportion of trial protocols reporting individual SPIRIT items. We compared these outcomes in protocols approved in 2012 and 2016, and built regression models to explore factors associated with adherence to SPIRIT. For each protocol, we also extracted information on general trial characteristics and assessed whether individual SPIRIT items were reported
RESULTS: The median proportion of reported SPIRIT items among RCT protocols showed a non-significant increase from 72% (IQR, 63%-79%) in 2012 to 77% (IQR, 68%-82%) in 2016. However, in a preplanned subgroup analysis, we detected a significant improvement in investigator-sponsored protocols: the median proportion increased from 64% (IQR, 55%-72%) in 2012 to 76% (IQR, 64%-83%) in 2016, while for industry-sponsored protocols median adherence was 77% (IQR 72%-80%) for both years. The following trial characteristics were independently associated with lower adherence to SPIRIT: single-centre trial, no support from a clinical trials unit or contract research organisation, and investigator-sponsorship.
CONCLUSIONS: In 2012, industry-sponsored RCT protocols were reported more comprehensively than investigator-sponsored protocols. After publication of the SPIRIT checklist, investigator-sponsored protocols improved to the level of industry-sponsored protocols, which did not improve. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Entities:  

Keywords:  Clinical trials; EPIDEMIOLOGY; Protocols & guidelines

Mesh:

Year:  2022        PMID: 35613804      PMCID: PMC9125701          DOI: 10.1136/bmjopen-2021-053417

Source DB:  PubMed          Journal:  BMJ Open        ISSN: 2044-6055            Impact factor:   3.006


We had full access to randomised clinical trial protocols from all research ethics committees in Switzerland and a convenience sample of one ethics committee in Germany and one in Canada approved in 2012 and 2016. The sample of trial protocols from Switzerland (n=397) was much larger than the sample from Germany (n=75) or Canada (n=77). The results from multivariable beta regression and logistic regression models were robust in sensitivity analyses using methods outlined a priori in a previously published protocol. All analyses were observational and any causal effect of the published Standard Protocol Items: Recommendations for Interventional Trials checklist cannot be inferred. Included trial protocols came all from three high-income countries limiting the generalisability of the results.

Introduction

Randomised clinical trials (RCTs) are directed by their protocol, which documents the rationale, design and planned reporting of a trial.1 Funding agencies, research ethics committees (RECs), regulatory agencies, medical journals, systematic reviewers and other groups rely on protocols to appraise the quality of a proposed trial.2 With incomplete protocols reviewers typically cannot distinguish between the use of inappropriate methodology and the non-reporting of appropriate methodology. In addition, if details about the application of the trial intervention or situations with unblinding of trial participants are lacking, the resulting uncertainty with treating clinicians may compromise the safety of trial participants. Empirical evidence from meta-research suggested numerous limitations in the reporting of RCT protocols including insufficient descriptions of treatment allocation methods, primary outcomes, sample size calculations, data analysis and the roles of sponsors in trial design or access to data.3–9 About half of protocols approved by French RECs, for instance, were estimated to have subsequent amendments to address deficiencies,10 and a third of amendments submitted to RECs for industry-sponsored trial protocols could have been avoided by preparing more comprehensive protocols.11 12 In response, a minimum set of items to be addressed in trial protocols was developed by the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Initiative, and published in January 2013.13 14 Subsequently, a number of journals publishing trial protocols, funding agencies and RECs endorsed the use of SPIRIT or related recommendations (eg, www.swissethics.ch).15 Researchers have applied the SPIRIT checklist to assess the quality of trial protocols with respect to patient reported outcomes,16 statistical analyses17 and cluster-randomised trials with stepped wedge design.18 However, there is no large-scale empirical study that has longitudinally evaluated the impact of the SPIRIT recommendations on the quality of reporting among RCT protocols. The Adherence to SPIrit REcommendations (ASPIRE) study group is an international collaboration of researchers with a mandate to (a) evaluate the completeness of RCT protocols before and after publication of the SPIRIT statement, (b) determine trial characteristics associated with non-adherence to SPIRIT checklist items and (c) investigate whether the comprehensiveness of RCT protocols varies across countries.19 In the present paper we report the results from our investigation of RCT protocols from Switzerland, CAnada and GErmany (ASPIRE-SCAGE).

Methods

The methods used to conduct the present study have previously been published.19

Identification of included trial protocols

We included trial protocols approved by RECs in 2012 or 2016 that assigned patients or groups of patients at random to one or more interventions to evaluate their effect on health outcomes. We excluded RCTs enrolling healthy volunteers, economic evaluations, animal studies, studies based on tissue samples, observational studies, studies involving only qualitative methods, and studies with a quasi-random method of allocation. Details of the identification of included RCT protocols are presented in online supplemental figure 1. The eligibility of RCT protocols was assessed independently and in duplicate. Any disagreements were resolved by discussion and consensus.

Data extraction

We used a web-based, password protected data extraction tool (http://squiekero.ch) for data collection and storage.19 20 Researchers trained in trial methodology completed a calibration process to improve reliability, and then extracted relevant data from RCT protocols independently and in duplicate, including whether individual SPIRIT items were reported.19 Disagreements were resolved by discussion. Due to limited resources 15% of included protocols were extracted by a single researcher (having extracted at least 100 RCT protocols in duplicate). All researchers extracting data from RCT protocols signed confidentiality agreements and the final database contained only coded data. Our data extraction forms are provided as online supplemental table 1.

Data analysis

The outcomes of interest were the proportion of SPIRIT checklist items that were reported among our cohorts of study protocols, and the proportion of RCT protocols addressing each SPIRIT checklist item. Our primary analysis was based on a rating approach that allowed for partial credit of individually met sub-items or components of major SPIRIT items, because it keeps the hierarchical structure of the SPIRIT checklist and it independently considers all components and sub-items of all individual SPIRIT items.19 Other rating approaches that consider major SPIRIT items only or equally consider items and sub-items, were used in sensitivity analyses. We provided descriptive statistics as frequencies and proportions for binary data and median (IQR) for continuous data. To investigate whether the reporting quality of RCT protocols (as defined by the proportion of reported SPIRIT checklist items) has increased from 2012 to 2016, we conducted multivariable beta regression analysis21 with the proportion of SPIRIT items adhered to per protocol as dependent variable and the following predefined independent variables: (a) approval year (2012 vs 2016), (b) investigator sponsorship versus industry sponsorship, (c) planned sample size (in increments of 1000), (d) single centre versus multicentre RCTs and (e) reported methodological support from a contract research organisation (CRO) or clinical trials unit (CTU) versus no reported support. We included interaction terms in our model to investigate potential interactions of year of approval (2012 or 2016) with either sponsorship of protocols or reported methodological support. We performed a likelihood ratio test to check if the interaction terms improved the goodness of fit of the models. To examine in a sensitivity analysis whether the comprehensiveness of RCT protocols varied across countries we stratified the median proportion of addressed SPIRIT items per protocol by country (Switzerland, Canada, Germany), by year of approval (2012 vs 2016), and by sponsorship (investigator vs industry), and added a country variable to the regression model. In further sensitivity analyses, we used hierarchical logistic regression (response is a binary variable indicating adherence to each SPIRIT item with clustering by protocol; that is, independent variables were included in the model as fixed effects and the protocol as a random effect) instead of beta regression.19 Beta regression allowed us to directly model the proportion of SPIRIT items adhered to per protocol,21 while hierarchical logistic regression allowed us to capture the variability within protocols. For all types of regression analyses we reported coefficients or ORs accompanied by 95% CIs. We used the statistical software R V.3.6.1 for all data analysis. All statistical tests were performed using a significance level of p=0.05.

Patient and public involvement

No patients were involved in the study.

Results

Characteristics of included trial protocols

We included 549 RCT protocols in our study; 257 from 2012 and 292 from 2016 (table 1). The majority of which were individually randomised, multicentre, parallel-group, superiority trials in oncology or cardiovascular medicine, and approved by a Swiss REC. Seventy-seven RCT protocols were from Canada and 75 from Germany. About half of the protocols were investigator-sponsored and half were industry-sponsored. In 2016 there were more investigator-sponsored protocols (162/292, 55.5%) included than in 2012 (119/257, 46.3%). In 2016 the median planned sample size was lower (199; IQR, 100–490) than in 2012 (300; IQR, 100–720). Otherwise, trial characteristics were similar between cohorts. Protocols of industry-sponsored RCTs had, on average, a larger sample size, were predominantly multinational, and more frequently placebo-controlled than those of investigator-sponsored RCTs (table 1).
Table 1

Characteristics of included randomised trial protocols

Characteristics20122016Overall
SponsorshipTotal (n=257)SponsorshipTotal (N=292)Total (N=549)
Industry (N=138)Investigator (N=119)Industry (N=130)Investigator (N=162)
Planned target sample size, median (IQR)450 (184.5–800)150 (63–516)300 (100–720)306.5 (150–621)141 (70–300)199 (100–490)220 (100–597)
Planned centres
 Single centre, No. (%)2 (1.4%)45 (37.8%)47 (18.3%)4 (3.1%)73 (45.1%)77 (26.4%)124 (22.6%)
 Multicentre, national, No. (%)10 (7.2%)30 (25.2%)40 (15.6%)6 (4.6%)41 (25.3%)47 (16.1%)87 (15.8%)
 Multicentre, multinational, No. (%)126 (91.3%)44 (37.0%)170 (66.1%)120 (92.3%)48 (29.6%)168 (57.5%)338 (61.6%)
Unit of randomisation
 Individuals137 (99.3%)113 (95.0%)250 (97.3%)127 (97.7%)158 (97.5%)285 (97.6%)535 (97.4%)
 Clusters0 (0.0%)4 (3.4%)4 (1.6%)1 (0.8%)3 (1.9%)4 (1.4%)8 (1.5%)
 Body parts1 (0.7%)2 (1.7%)3 (1.2%)2 (1.5%)1 (0.6%)3 (1.0%)6 (1.1%)
Study design
 Parallel135 (97.8%)104 (87.4%)239 (93.0%)127 (97.7%)147 (90.7%)274 (93.8%)513 (93.4%)
 Crossover2 (1.4%)9 (7.6%)11 (4.3%)2 (1.5%)10 (6.2%)12 (4.1%)23 (4.2%)
 Factorial1 (0.7%)6 (5.0%)7 (2.7%)1 (0.8%)5 (3.1%)6 (2.1%)13 (2.4%)
Study purpose
 Superiority110 (79.7%)93 (78.2%)203 (79.0%)107 (82.3%)132 (81.5%)239 (81.8%)442 (80.5%)
 Non-inferiority23 (16.7%)19 (16.0%)42 (16.3%)20 (15.4%)24 (14.8%)44 (15.1%)86 (15.7%)
 Unclear5 (3.6%)7 (5.9%)12 (4.7%)3 (2.3%)6 (3.7%)9 (3.1%)21 (3.8%)
Placebo used77 (55.8%)30 (25.2%)107 (41.6%)78 (60.0%)41 (25.3%)119 (40.8%)226 (41.2%)
CTU or CRO support93 (67.4%)56 (47.1%)149 (58.0%)79 (60.8%)83 (51.2%)162 (55.5%)311 (56.6%)
Country
 Switzerland91 (66.0%)89 (74.8%)180 (70.0%)86 (66.2%)131 (80.9%)217 (74.3%)397 (72.3%)
 Canada21 (15.2%)19 (16.0%)40 (15.6%)17 (13.1%)20 (12.3%)37 (12.7%)77 (14.0%)
 Germany26 (18.8%)11 (9.2%)37 (14.4%)27 (20.8%)11 (6.8%)38 (13.0%)75 (13.7%)

CRO, contract research organisation; CTU, clinical trials unit.

Characteristics of included randomised trial protocols CRO, contract research organisation; CTU, clinical trials unit.

Adherence to SPIRIT in protocols from 2012 and 2016

Overall, the median proportion of reported SPIRIT items increased from 72% (IQR, 63%–79%) in 2012 to 77% (IQR, 68%–82%) in 2016 (table 2, figure 1).
Table 2

Adherence to SPIRIT items in randomised clinical trial protocols

Characteristic20122016
SponsorshipTotal 2012(n=257)SponsorshipTotal 2016(n=292)
Industry (n=138)Investigator (n=119)Industry (n=130)Investigator (n=162)
Median (IQR)Median (IQR)Median (IQR)Median (IQR)Median (IQR)Median (IQR)
Absolute number of SPIRIT items reported per protocol (out of 33)25.5 (23.6–26.5)21.3 (18.3–23.7)23.7 (20.7–26.2)25.3 (23.7%–26.9)25.0 (21.3–27.3)25.3 (22.5–27.1)
Proportion of SPIRIT items reported per protocol77% (72%–80%)64% (55%–72%)72% (63%–79%)77% (72%–82%)76% (64%–83%)77% (68%–82%)

SPIRIT, Standard Protocol Items: Recommendations for Interventional Trials.

Figure 1

Proportion of reported Standard Protocol Items: Recommendations for Interventional Trials items by year and trial sponsorship.

Adherence to SPIRIT items in randomised clinical trial protocols SPIRIT, Standard Protocol Items: Recommendations for Interventional Trials. Proportion of reported Standard Protocol Items: Recommendations for Interventional Trials items by year and trial sponsorship. Stratifying by sponsorship, we found that the comprehensiveness increased only in investigator-sponsored RCT protocols (adherence stratified by other study characteristics can be found in online supplemental table 2). The median proportion of reported SPIRIT items in investigator-sponsored protocols increased from 64% (IQR, 55%–72%) in 2012 to 76% (IQR, 64%–83%) in 2016, while it remained unchanged at 77% for both years among industry-sponsored protocols (77%, IQR 72%–80% in 2012, and 77%, IQR 72%–82% in 2016). This pattern was consistent across countries (online supplemental table 3). Sensitivity analyses using different approaches to calculate the proportion of reported SPIRIT items provided similar results (online supplemental table 4). Regarding individual SPIRIT items, we found that the improvement in investigator-sponsored RCT protocols was due to an improvement in a broad range of SPIRIT items (online supplemental table 5); for 25 individual items the proportion of adherent protocols improved in investigator-sponsored RCTs by 10% or more (online supplemental table 6). These 25 items included descriptive (eg, information on study registration, protocol version and date, name and contact details of sponsor) as well as methodological aspects (eg, comparator choice explained, or allocation concealment mechanism). The largest improvements occurred with ‘trial registration’(SPIRIT item 2, +41.1%), ‘plans to disseminate trial results to key stakeholders/publication provided’ (SPIRIT item 31a, +36.7%), ‘description of process for making amendments’ (SPIRIT item 25, +34.4%) and ‘declaration of interests’ (SPIRIT item 28, +31.6%). In industry-sponsored protocols, adherence to individual SPIRIT items remained practically stable from 2012 to 2016, that is, items with low adherence in 2012 remained low in 2016. SPIRIT items with particularly low adherence (<30%) in both industry-sponsored and investigator-sponsored protocols were ‘names of protocol contributors/authors’ (SPIRT item 5a), ‘research question described and justified’ (SPIRIT item 6a), ‘eligibility criteria for study centres’ (SPIRIT item 10) in applicable RCTs, ‘location of participant recruitment’ and ‘estimated recruitment rate’ (SPIRIT item 15), ‘information about who will have access to the full dataset’ (SPIRIT item 29) and ‘description of plans for granting access to full trial protocol’ (SPIRIT item 31c) (online supplemental table 5).

Multivariable regression analysis

Using multivariable beta regression, we found that four characteristics were independently associated with greater reporting of SPIRIT items: multicentre RCTs (OR, 1.18; 95% CI 1.05 to 1.33; p=0.006), RCTs with reported methodological support from CTUs or CROs (OR, 1.44; 95% CI 1.31 to 1.57; p<0.001), industry-sponsored RCTs (OR, 1.20; 95% CI 1.09 to 1.33; p<0.001) and RCTs approved in 2016 (OR, 1.33; 95% CI 1.22 to 1.45; p<0.001) (online supplemental table 7, figure 2).
Figure 2

Association between comprehensiveness of trial protocols and trial characteristics, accessed by multivariable beta regression. *Interaction terms were added to the multivariable model one at a time. CRO, contract research organisation; CTU, clinical trials unit.

Association between comprehensiveness of trial protocols and trial characteristics, accessed by multivariable beta regression. *Interaction terms were added to the multivariable model one at a time. CRO, contract research organisation; CTU, clinical trials unit. Adding the interaction term of year of approval and sponsorship to the model, improved the model fit (likelihood ratio test, χ2=30.01, p<0.01) and provided evidence for a differential improvement in the adherence to SPIRIT over time (2012 vs 2016) for industry-sponsored and investigator-sponsored protocols suggesting that there was an improvement in investigator-sponsored protocols but not in industry-sponsored protocols (interaction p<0.001). We did not find evidence for an interaction between year of approval and CTU/CRO support (interaction p=0.79), that is, protocols with or without reported support from CTUs or CROs improved to a similar extent from 2012 to 2016. Limiting our multivariable regression to investigator-sponsored protocols in an exploratory analysis, we found a notable interaction suggesting a more pronounced improvement in Swiss protocols compared with protocols from Canada or Germany (interaction p=0.032; online supplemental table 8). Sensitivity analyses using hierarchical logistic regression instead of beta regression confirmed all results.

Discussion

Main findings and interpretation

This study of 549 RCT protocols approved by RECs in Switzerland, Canada and Germany before (2012) and after (2016) the publication of the SPIRIT recommendations suggested a small overall improvement in reporting comprehensiveness. This change was driven by an increase in the median proportion of reported SPIRIT items in investigator-sponsored RCTs from 64% in 2012 to 76% in 2016. Protocols of industry-sponsored RCTs remained, on average, unchanged (median of 77% SPIRIT items reported in both years). The reporting of investigator-sponsored protocols improved for the majority of individual SPIRIT items, and was independent of the planned sample size, reported support from a CTU or CRO, and centre status (singlecentre vs multicentre) of RCTs. Single centre status, no reported support from a CTU or CRO, investigator sponsorship, and approval in 2012 were independently associated with lower adherence to the SPIRIT checklist. These results were similar across countries, but the improvement in investigator-sponsored RCT protocols appeared more pronounced among Swiss protocols compared with protocols approved in Canada or Germany. SPIRIT items with particularly low adherence in investigator-sponsored and industry-sponsored protocols concerned the names of protocol contributors/authors, the justification of the research question, details about the planned participant recruitment, information about who will have access to the full dataset, and plans for granting access to the full trial protocol. Our findings suggest an international improvement in the comprehensiveness of investigator-sponsored RCT protocols probably due to a combination of reasons including the publication of the SPIRIT checklist and its implementation by research institutions, funding agencies and medical journals; the ongoing discussion about the importance of protocol publication, thoughtful planning of RCTs, and minimising reporting biases in the scientific community; and efforts to teach RCT methodology to clinician scientists in undergraduate and postgraduate courses. The more pronounced improvement of Swiss investigator-sponsored protocols could be related to a SPIRIT-based protocol template and guidance provided by swissethics22 that were particularly welcomed by academic researchers or other changes in the context of the new Swiss legislation on human research from 2014.

Strengths and limitations

Strengths of our study include full access to RCT protocols and associated documents from RECs in three countries. We used standardised methods and procedures for data extraction and protocol assessment at all RECs and involved only trained methodologists in this process. This included use of piloted extraction forms with detailed written instructions as well as calibration exercises with all data extractors. More than 95% of included protocols approved in 2012 and over 80% of protocols approved in 2016 were extracted independently and in duplicate. To minimise chance associations, we considered only a limited number of variables in our statistical models.23 Our results proved robust in sensitivity analyses applying alternative assumptions and statistical approaches. The fact that all Swiss RECs participated in this study strengthens the representativeness of our data for Switzerland and the additional inclusion of a German and a Canadian REC allowed for an international comparison to some extent. Our study has several limitations. First, we used a convenience sample of two RECs outside of Switzerland (Freiburg in Germany, Hamilton in Canada) but we cannot be certain if they are representative of other RECs in these or other countries. Second, we used RCT protocols that had already been approved by RECs, therefore SPIRIT items such as ‘research ethics approval’ and ‘consent forms provided’ were always fulfilled and could not discriminate more comprehensive from less comprehensive protocols. Third, although we had adequate statistical power to detect even interactions within the subgroup of investigator-sponsored protocols, the number of included protocols approved outside of Switzerland was relatively small (28%; 152/549), limiting the precision of estimates for German and Canadian protocols. Fourth, 15% of included protocols were not evaluated in duplicate which could have increased the risk of bias in our study. However, these protocols were from different RECs in Switzerland and they were handled by one of the two most experienced data extractors only, so we feel that a relevant increase in the risk of bias is unlikely. Fifth, we are not aware of the fact that any of the participating RECs explicitly endorsed SPIRIT guidance, however, in Switzerland a new protocol template provided by swissethics became available which was influenced by SPIRIT impacting the generalisability of our results. In addition, it remains unclear to what extent our findings can be extrapolated to trial protocols from middle-income or low-income countries and to protocols from medical disciplines underrepresented in our sample (eg, dentistry or geriatrics; online supplemental table 9). Finally, our findings are not proof of causality due to the observational nature of this study, however, it is plausible that the publication of the SPIRIT statement at least contributed to an increase in the comprehensiveness of investigator-sponsored protocols. Investigations of a potential time trend with gradually increasing comprehensiveness of investigator-sponsored protocols by year tertiles did not suggest a continuous development, but rather a one-step-effect (online supplemental figure 2).

Comparison with other studies

Few studies in the literature have used16 or planned to use17 18 24 the SPIRIT checklist as a tool to assess the comprehensiveness of trial protocols. One study investigated 75 RCT protocols from the UK National Institute for Health Research Health Technology Assessment programme on the reporting of patient-reported outcomes and the association with general protocol completeness according to SPIRIT.16 They found that these investigator-sponsored UK RCT protocols from 2012 and 2013 reported, on average, 63% of SPIRIT checklist items, which is very similar to our findings for investigator-sponsored RCT protocols from 2012. Apart from the ongoing study using protocols from UK RECs (ASPIRE-UK19), we are not aware of any other study evaluating the comprehensiveness of RCT protocols before and after the publication of the SPIRIT statement in industry-sponsored and investigator-sponsored protocols. There are studies assessing the quality of RCT protocols using measures other than the SPIRIT checklist. An analysis of drug trial protocols submitted to three Dutch RECs in 2010/2011 focused on critical comments by RECs.25 They found that applicants of investigator-sponsored trials received more critical comments on participant selection, methodology and statistical analysis than applicants of industry-sponsored trials, resonating with our findings of less comprehensive investigator-sponsored protocols compared with industry protocols in 2012. Similarly, studies by Getz et al used the proportion of protocols with substantial amendments as a measure of RCT protocol quality in the industry setting showing that more comprehensive protocols could have prevented amendments.11 12 Finally, a study of 596 published RCT protocols from 2001 to 2011 assessed protocol quality (high vs low) based on reporting of the allocation method, allocation concealment and intention-to-treat analysis.26 This study found a substantial improvement in some methodological aspects of protocols (eg, allocation concealment), but acknowledged the overall low proportion of high-quality protocols with 24% in 2001–2004, 31% in 2005–2008 and 37% in 2008–2011.

Implications

Incomplete protocols may jeopardise the clinical research process at all stages with potentially harmful consequences for patients, decision-makers in healthcare, the scientific community and society as a whole. Whether there is indeed an association between better reported or more comprehensive RCT protocols and better methodology, successful trial conduct and/or publication of RCTs remains to be established. Based on the RCT sample of this study, we will examine the relationship between completeness of RCT protocols and risks for premature discontinuation or non-publication of RCTs as well as potential improvements between 2012 and 2016 in terms of fewer trial discontinuations and non-publications particularly for investigator-sponsored RCTs in subsequent investigations.19 Our results show improvement in the reporting quality of investigator-sponsored trial protocols such that they became consistent with industry protocols. About why industry protocols have not improved according to SPIRIT between 2012 and 2016, we can only speculate. It might be that routines and processes for writing trial protocols have been well established at companies earlier explaining our finding of consistently low adherence to specific SPIRIT items in 2012 and 2016 in industry-sponsored protocols. So, as long as regulators do not make specific protocol templates mandatory for all applicants, industry may not adapt routines and templates according to SPIRIT. Our finding of insufficient reporting of names of protocol contributors/authors, the justification of the research question, details about the planned participant recruitment, information about who will have access to the full dataset, and plans for granting access to the full trial protocol guides involved stakeholders with respect to further needs for protocol improvement. The identified items constitute important pieces of information to enable a valid assessment of the relevance, feasibility and transparency of planned clinical trials.

Conclusions

This before-and-after study suggests that the comprehensiveness of investigator-sponsored RCT protocols from Switzerland, Canada and Germany improved after publication of the SPIRIT checklist, achieving a similar reporting quality as industry-sponsored protocols. Single centre status, no reported support from a CTU or CRO, investigator sponsorship, and approval in 2012 were independently associated with lower adherence to SPIRIT. Further means are needed to improve the reporting of RCT protocols particularly with respect to protocol authorship, justification of the research question, participant recruitment, access to the full dataset and plans for granting access to the full trial protocol. Future research should clarify the relationship between protocol quality and success of subsequent trial conduct and publication.
  22 in total

1.  What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models.

Authors:  Michael A Babyak
Journal:  Psychosom Med       Date:  2004 May-Jun       Impact factor: 4.312

2.  Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.

Authors:  An-Wen Chan; Asbjørn Hróbjartsson; Mette T Haahr; Peter C Gøtzsche; Douglas G Altman
Journal:  JAMA       Date:  2004-05-26       Impact factor: 56.272

3.  SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials.

Authors:  An-Wen Chan; Jennifer M Tetzlaff; Peter C Gøtzsche; Douglas G Altman; Howard Mann; Jesse A Berlin; Kay Dickersin; Asbjørn Hróbjartsson; Kenneth F Schulz; Wendy R Parulekar; Karmela Krleza-Jeric; Andreas Laupacis; David Moher
Journal:  BMJ       Date:  2013-01-08

4.  Adverse event reporting in publications compared with sponsor database for cancer clinical trials.

Authors:  Orit Scharf; A Dimitrios Colevas
Journal:  J Clin Oncol       Date:  2006-08-20       Impact factor: 44.544

5.  Comparison of protocols and registry entries to published reports for randomised controlled trials.

Authors:  Kerry Dwan; Douglas G Altman; Lynne Cresswell; Michaela Blundell; Carrol L Gamble; Paula R Williamson
Journal:  Cochrane Database Syst Rev       Date:  2011-01-19

6.  Reporting on blinding in trial protocols and corresponding publications was often inadequate but rarely contradictory.

Authors:  Asbjørn Hróbjartsson; Julie Pildal; An-Wen Chan; Mette T Haahr; Douglas G Altman; Peter C Gøtzsche
Journal:  J Clin Epidemiol       Date:  2009-09       Impact factor: 6.437

7.  Frequency and reasons for outcome reporting bias in clinical trials: interviews with trialists.

Authors:  R M D Smyth; J J Kirkham; A Jacoby; D G Altman; C Gamble; P R Williamson
Journal:  BMJ       Date:  2011-01-06

8.  Systematic evaluation of the patient-reported outcome (PRO) content of clinical trial protocols.

Authors:  Derek Kyte; Helen Duffy; Benjamin Fletcher; Adrian Gheorghe; Rebecca Mercieca-Bebber; Madeleine King; Heather Draper; Jonathan Ives; Michael Brundage; Jane Blazeby; Melanie Calvert
Journal:  PLoS One       Date:  2014-10-15       Impact factor: 3.240

9.  Reporting quality of stepped wedge design randomized trials: a systematic review protocol.

Authors:  Alex Thabane; Brittany B Dennis; Olga Gajic-Veljanoski; James Paul; Lehana Thabane
Journal:  Clin Epidemiol       Date:  2016-07-08       Impact factor: 4.790

10.  Systematic evaluation of patient-reported outcome (PRO) protocol content and reporting in UK cancer clinical trials: the EPiC study protocol.

Authors:  Khaled Ahmed; Derek Kyte; Thomas Keeley; Fabio Efficace; Jo Armes; Julia M Brown; Lynn Calman; Chris Copland; Anna Gavin; Adam Glaser; Diana M Greenfield; Anne Lanceley; Rachel Taylor; Galina Velikova; Michael Brundage; Rebecca Mercieca-Bebber; Madeleine T King; Melanie Calvert
Journal:  BMJ Open       Date:  2016-09-21       Impact factor: 2.692

View more
  2 in total

1.  Nonregistration, discontinuation, and nonpublication of randomized trials: A repeated metaresearch analysis.

Authors:  Benjamin Speich; Dmitry Gryaznov; Jason W Busse; Viktoria L Gloy; Szimonetta Lohner; Katharina Klatte; Ala Taji Heravi; Nilabh Ghosh; Hopin Lee; Anita Mansouri; Ioana R Marian; Ramon Saccilotto; Edris Nury; Benjamin Kasenda; Elena Ojeda-Ruiz; Stefan Schandelmaier; Yuki Tomonaga; Alain Amstutz; Christiane Pauli-Magnus; Karin Bischoff; Katharina Wollmann; Laura Rehner; Joerg J Meerpohl; Alain Nordmann; Jacqueline Wong; Ngai Chow; Patrick Jiho Hong; Kimberly Mc Cord-De Iaco; Sirintip Sricharoenchai; Arnav Agarwal; Matthias Schwenkglenks; Lars G Hemkens; Erik von Elm; Bethan Copsey; Alexandra N Griessbach; Christof Schönenberger; Dominik Mertz; Anette Blümle; Belinda von Niederhäusern; Sally Hopewell; Ayodele Odutayo; Matthias Briel
Journal:  PLoS Med       Date:  2022-04-27       Impact factor: 11.613

2.  A longitudinal assessment of trial protocols approved by research ethics committees: The Adherance to SPIrit REcommendations in the UK (ASPIRE-UK) study.

Authors:  Benjamin Speich; Ayodele Odutayo; Nicholas Peckham; Alexander Ooms; Jamie R Stokes; Ramon Saccilotto; Dmitry Gryaznov; Belinda von Niederhäusern; Bethan Copsey; Douglas G Altman; Matthias Briel; Sally Hopewell
Journal:  Trials       Date:  2022-07-27       Impact factor: 2.728

  2 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.