Reporting quality of abstracts of trials published in top five pain journals: a protocol for a systematic survey.

INTRODUCTION: Abstracts of randomised controlled trials (RCTs) are often the first and the only source read in a journal by busy healthcare providers. This necessitates good reporting of abstracts. The quality of reporting of abstracts, though gradually improving over time, is still not uniform across medical journals. Improvement in completeness of reporting of abstracts has been documented in general medical journals after the publication of the consolidated standards of reporting trials (CONSORT) extension for abstracts in 2008. Currently, this aspect has not been assessed with regards to pain journals. This study aims to compare the completeness of reporting of abstracts before and after the publication of CONSORT statement for abstracts in five pain journals. METHODS AND ANALYSES: The abstracts of RCTs published from 1 January 2005 to 31 December 2007 (pre-CONSORT) and from 1 January 2013 to 31 December 2015 (post-CONSORT) will be assessed for the quality of reporting. Studies without abstracts, non-English abstracts, abstracts not reporting on RCTs or on humans and conference abstracts will be excluded. A thorough search of MEDLINE will be carried out in April 2016. All identified studies will be screened for inclusion based on titles and abstracts. Data will be extracted by two sets of independent reviewers for each abstract in duplicate regarding compliance with CONSORT statement for abstracts. Full-text review will be performed to obtain additional characteristics which are likely to affect reporting quality. The unadjusted and adjusted differences in the mean number of items reported will be analysed using a two sample t-test and generalised estimation equation in SPSS. ETHICS AND DISSEMINATION: As far as we know, this is the first study to evaluate reporting quality of abstracts of pain journals based on CONSORT extension for abstracts. The findings of this study will be disseminated by a presentation at a conference and through publication in a peer-reviewed journal. Ethics committee approval was not sought for this survey. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

This is the first review on quality of reporting of abstracts of randomised controlled trials in pain journals.

Review of abstract quality independently and in duplicate and evaluating the possible factors contributing to quality of reporting.

Comparison of reporting before and after the publication of consolidated standards of reporting trials extension for abstracts to assess possible improvement.

Only MEDLINE search will be carried out for a prespecified time period and only abstracts of pain trials published in five pain journals will be considered.

Restriction to abstracts published in English and on humans are additional limitations.

Introduction

Evidence originating from randomised controlled trials (RCTs) is considered superior to other sources of evidence in clinical research.1 Abstracts of RCTs are often the initial source on which decision about full-text reading is made. Many busy clinicians make healthcare decisions based on the information available in the abstracts.2 This could be due to limitations of time, non-availability of the full-text due to non-subscription of journal or non-English language of the relevant article. Researchers, especially those doing systematic reviews rely on the content of the abstract to perform initial screen to include potential studies for meta-analysis.3 Incomplete reporting of the essential details of the study in the abstract can therefore lead to inaccurate interpretation of the findings and possibly, wrong application in clinical practice. Hence, complete and structured reporting of the abstracts is necessary for meaningful and quick understanding of the study details. The consolidated standards of reporting trials (CONSORT) statement was first developed by the CONSORT group in 1996 to provide a minimum set of recommendations for reporting of RCTs.4 The most recent statement published in 2010 consists of a 25-item checklist for reporting of RCTs.5 The CONSORT extension for abstracts published in 2008 provides the list of 17 minimum items to be reported by the authors in the abstract that are considered necessary for good interpretation of the RCTs.6 Previous studies have documented poor quality of reporting of abstracts in major medical journals before the publication of CONSORT statement7 and subsequent improvement in the reporting details following publication of the CONSORT statement for abstracts.8 However, non-adherence to the CONSORT statement for abstracts was observed in four high-impact general medical journals even after 2 years of publication of these guidelines.9 Similarly, a mere 2.4% points improvement in proportion of items complying with CONSORT statement for abstracts was seen from pre-CONSORT period in major anaesthesia journals.10 Despite an increase in the number of RCTs in pain and palliative care domains published over the years,11 the assessment of quality of reporting of RCTs has been limited and even these papers report mixed findings with respect to improvement over the years.12 13 Moreover, the quality of reporting of abstracts related to RCTs in pain journals has not been evaluated until now, necessitating this study. Given the complex and multidimensional nature of pain, non-uniform methods and different outcome domains are used in trials published in pain journals. In the absence of complete reporting of abstracts, this can lead to misleading interpretations with implications on clinical decisions.

Objectives

The purpose of this study is to inform pain practitioners and researchers on the current quality of reporting of abstracts and how reporting of abstracts of RCTs actually needs to be carried out. The specific objectives to fulfil this purpose are (1) to assess the number of items reported from the CONSORT abstract statement in five pain journals before and after the publication of CONSORT extension for abstracts and (2) to explore the factors associated with the quality of reporting of abstracts.

Methods

Study design

This study will be a methodological review. A thorough search of MEDLINE will be conducted in April 2016 for the RCTs published in the year 2005–2007 and 2013–2015 in top five exclusive pain journals (based on impact factor) as per the Journal Citation Report 2014 published by Thomson Reuters;14 Pain (5.213), Pain Physician (3.542), European Journal of Pain (2.942), Clinical Journal of Pain (2.527) and Pain Practice (2.361). The search strategy will include terms for RCTs (randomised control*, clinical trial*), journal names (as above), exclusions for other type of articles (study protocol, review, cohort, case–control, case series, guideline and editorial) and limits set for the specific time periods of interest (1 January 2005 to 31 December 2007 and 1 January 2013 to 31 December 2015). The search strategy that we adopted for searching of the relevant abstracts is described in online supplementary appendix 1. All the RCTs published in these five journals during these years will be included based on prespecified criteria: the abstract should be a report of an RCT, published in English language, and involving human subjects. Studies will be excluded if the abstract is not available, they are published only as abstracts (eg, conference proceedings), still recruiting or are duplicate publications. A summary of our objectives, outcomes, hypotheses and methods of analysis is depicted in table 1.

Table 1

Summary of objectives, outcomes, hypotheses and methods of analysis

Objectives	Outcomes	Explanatory variables	Hypothesis	Methods of analyses
Primary: to assess the quality of reporting of abstracts of RCTs in top pain journals before and after publication of CONSORT extension for abstracts	Overall quality of the reporting of abstracts Quality of reporting of the individual items	Timing of publication (pre vs post-CONSORT publication)	The quality of reporting of abstracts is better for period after publication of CONSORT statement for abstracts than before	Unadjusted and adjusted* regression using generalised estimating equations (GEE)
Secondary: to explore the factors associated with quality of reporting of abstracts		CONSORT endorsement by the journal Number of centres (single vs multicentric) Sample size (≤100 vs >100) Type of intervention (pharmacological vs non-pharmacological) Significance of results for primary outcome (significant vs non-significant) Source of funding (industry funded vs non-industry funded)		GEE

*This analysis will be adjusted for the number of centres, sample size, type of intervention, significance of results for primary outcome and source of funding.

CONSORT, consolidated standards of reporting trials; RCTs, randomised controlled trials.

Sample size calculation

The primary objective of this study is to compare the mean number of reported items in prepublication versus postpublication of the CONSORT extension to abstracts based on the corresponding checklist.6 We hypothesise that there will be significant improvement in the mean number of reported items post-CONSORT extension to abstracts. An earlier review assessed the quality of abstracts in general medical journals before and after the publication of CONSORT statement for abstracts and observed an 18% improvement in the reporting quality of abstracts.8 The mean difference in the number of items reported in this study was 3.05; 95% CI (CI 2.44 to 3.65); p<0.001. Based on this study, we estimated that the sample size required in each study period (pre-CONSORT vs post-CONSORT) with a significance of 0.05 and a power of 0.8 is 111 to observe a similar difference. Considering further 3% improvement/year in reporting over the past 2 years since this publication and 8 years from CONSORT statement for abstracts, a sample size of 122 was determined. This basic calculation assumes that the comparison of means would be based on a t-test. To account for possible clustering of articles published in the same journal, we inflated the sample size by a factor of 1.796 (variable inflation factor) to n=220—assuming an intraclass correlation coefficient of 0.034 and an average number of articles of 24.4 per journal. The primary analysis will also be adjusted for potential confounding using the following six variables: endorsement of CONSORT statement by the journal, number of centres involved in the RCT, the type of intervention (pharmaceutical vs other), sample size, significance of the results of the trial and funding status (industry vs non-industry). Therefore, we adjusted the sample size upward by adding five articles for each variable for a total sample size of n=250. If more than 250 eligible articles are found, 250 will be randomly selected for inclusion.

Data extraction and synthesis

Data will be extracted regarding the compliance of the abstract to the CONSORT statement for abstracts.6 Additional details will also be obtained with regards to endorsement of the CONSORT statement for RCTs and for abstracts of RCTs by the journals, whether the study is carried out at a single centre or multiple centres, total number of patients recruited in the study, whether the study involved pharmacological intervention, whether the study was industry sponsored and whether the study reported statistically significant results. General information regarding journal name, author, year of publication and free availability of the full text of the article will also be extracted. Full-text review will be performed to obtain additional information for analysis. Screening and data abstraction will be carried out independently and in duplicate (each abstract will be reviewed by two reviewers for pre-CONSORT and post-CONSORT period) using a customised data extraction form in Microsoft Excel and between reviewer agreements will be measured using the κ statistic.15 Each of the four reviewers (SK, SB, MW and LPFA) will review half of the abstracts for both the study periods. An initial trial run involving 10% of the eligible articles will be undertaken to improve the clarity regarding inclusions and exclusions and to increase accuracy and consistency among the reviewers. A simple customised instruction manual, examples contained in the CONSORT checklist16 and CONSORT elaboration and explanation guidance document17 will be used by all the reviewers to assess the articles for data extraction. Disagreements will be resolved through consensus between the reviewers and if it persists, by arbitration by the senior author (LT).

Statistical analyses

The characteristics of the included articles will be analysed using descriptive statistics reported as mean (SD) or median (first quartile, third quartile) for continuous variables depending on the distribution and number (per cent) for categorical variables. We will describe the number (per cent) of articles reporting each item by period of publication (pre-CONSORT vs post-CONSORT). We will also report the mean or median number of reported items by period of publication. The mean (median) number of items reported (0–17) for each period (prepublication and postpublication of CONSORT extension for abstracts) will be calculated and the unadjusted and adjusted differences will be estimated using a two sample t-test and generalised estimation equations (GEEs), respectively.18 The means or medians will be reported along with their SDs or IQR. The mean (median) differences and adjusted means (medians) will be also reported with 95% CIs and p values. Next, the compliance with the 17 items of the CONSORT statement for abstracts for years 2005–2007 vs 2013–2015 will be compared using individual χ2 tests. This will be followed by an adjusted analysis using GEE. For binary outcomes (item reported yes or no), we will assume the binomial distribution and unstructured correlation matrices. The adjusted ORs, 95% CI and p values will be reported. Finally, the incidence rate ratios (IRRs) for reporting items for the period 2013–2015 compared to the period 2005–2007 will be estimated using GEE, assuming a Poisson distribution and an unstructured correlation matrix. Adjusted IRRs, 95% CIs and p values will be reported. The criterion for statistical significance will be set at α=0.05.

For the GEE, adjustments will be made for (1) whether or not the journal endorses the CONSORT statement, (2) number of centres (multiple centres versus single centre), (3) type of intervention (pharmaceutical versus all others), (4) sample size (≤100 vs >100), (5) results of trial (statistically significant versus not significant) and (6) funding status (industry funded versus non-funded) with journal as a grouping factor—to adjust for potential clustering or similarity in articles published in the same journal. Descriptive data will be presented as counts and percentages. Data will be analysed using Statistical Package for Social Sciences (SPSS) V.16.0 (SPSS, 2009, Chicago, Illinois, USA).

Discussion and dissemination

Ethics approval was not sought for this survey as it only involved assessment of previously published information. Pain journals are increasingly publishing RCTs to disseminate high-quality evidence to their readers in clinical practice similar to the journals belonging to other medical subspecialties. However, a general reading of the abstracts of RCTs in pain journals suggests that the quality of reporting across various journals is variable with some journal abstracts communicating adequate information and some grossly insufficient for accurate interpretation. Uniform and complete reporting of various aspects of the study design, methods and results help the reader to interpret the abstract accurately and to make well-informed decisions for better patient care. Patients or their families, who seek authentic information regarding problems relating to their pain and who possibly wish to enrol for trials that might benefit them, are likely to make inaccurate judgments if reporting is incomplete.19 Similarly, a structured and detailed reporting of RCTs helps guideline developers and policymakers as they rely heavily on RCTs. Incomplete information makes it difficult to trust the findings resulting in suboptimal use of these RCTs.20 Evidence-based pain management based on accurate reporting of trials and their correct interpretation has shown to improve patients' outcomes and satisfaction.21 Hence, it is imperative for authors to report complete details of their research and for journals to ensure good reporting is adhered to by authors.

It is in this context, we chose five top pain journals as per their impact factors to assess their quality of reporting of abstracts of RCTs for this methodological review. This assessment becomes important in view of the increasing quantity of publications in the recent years on the subject of pain. In addition to the compliance with 17 components of CONSORT checklist, we will assess in this review certain other characteristics of the article that might affect the reporting quality of RCTs. We hypothesise that the reporting quality in these journals will vary depending on the journal's endorsement of the CONSORT, number of sites the study is conducted, sample size, type of intervention, significance of the result of primary outcome and funding of the study.

We expect RCTs from journals that endorse CONSORT,22 23 multicentric studies,23 24 studies with larger sample size,23 25 studies involving pharmacological intervention,23 26 studies reporting significant results for their primary outcome27 and industry sponsored studies27 to be more compliant with the CONSORT extension for abstracts. Since substantial years (eight) have passed from the time of publication of CONSORT statement for abstracts in 2008, we hypothesise that the overall quality of study abstracts will be better for the post-CONSORT statement for abstracts time period than for the pre-CONSORT period.

On completion, this study will be submitted to a peer-reviewed biomedical journal for publication and the findings will also be presented at an upcoming conference.

To conclude, the results of this study are likely to clarify the current standards of reporting of abstracts in pain journals and improvement if any, over time compared to the period before CONSORT statement for abstracts were published. In case the current reporting quality is found to be inadequate, this comparative analysis will emphasise the need for journals to consider incorporating the CONSORT statement for abstracts in the guidelines for authors.