Quality of missing data reporting and handling in palliative care trials demonstrates that further development of the CONSORT statement is required: a systematic review.

OBJECTIVES: Assess (i) the quality of reporting and handling of missing data (MD) in palliative care trials, (ii) whether there are differences in the reporting of criteria specified by the Consolidated Standards of Reporting Trials (CONSORT) 2010 statement compared with those not specified, and (iii) the association of the reporting of MD with journal impact factor and CONSORT endorsement status. STUDY DESIGN AND
SETTING: Systematic review of palliative care randomized controlled trials. CENTRAL, MEDLINE, and EMBASE (2009-2014) were searched.
RESULTS: One hundred and eight trials (15,560 participants) were included. MD was incompletely reported and not handled in accordance with current guidance. Reporting criteria specified by the CONSORT statement were better reported than those not specified (participant flow, 69%; number of participants not included in the primary outcome analysis, 94%; and the reason for MD, 71%). However, MD in items contributing to scale summaries (10%) and secondary outcomes (9%) were poorly reported, so the proportion of MD stated is likely to be an underestimate. The reason for MD provided was unclear for 54% of participants and only 16% of trials with MD reported a MD sensitivity analysis. The odds of reporting most of the MD and other risk of bias reporting criteria were increased as the journal impact factor increased and in journals that endorsed the CONSORT statement.
CONCLUSION: Further development of the CONSORT MD reporting guidance is likely to improve the quality of reporting. Reporting recommendations are provided.

Missing data (MD) in palliative care trials are not reported and handled in a clear, complete, and transparent manner.

Criteria specified by the Consolidated Standards of Reporting Trials (CONSORT) 2010 statement were better reported than those specified by other reporting guidelines.

The odds of reporting most MD and other risk of bias reporting criteria were increased as the journal impact factor doubled and in journals that endorsed the CONSORT statement.

Assessment of the risk MD pose to the power, precision, generalisability and validity of trial findings is substantially hindered by poor reporting.

Further development of the CONSORT MD reporting guidance would potentially strengthen reporting standards. Reporting recommendations are provided for published trial reports.

Introduction

Missing data (MD) present a significant risk to the power, precision, generalizability, and validity of randomized controlled trial (RCT) findings. Transparent reporting of MD is therefore crucial to the critical appraisal of trial results by clinicians, patients, policymakers, journalists, and researchers [1], [2]. Palliative care trials in particular have large amounts of MD, as well as differential rates and reasons of missingness across trial arms that potentially introduce bias [3].

To strengthen the reporting standards of MD in trial reports, the 2010 Consolidated Standards of Reporting Trials (CONSORT) statement highlights the potential impact of MD on the validity of intention-to-treat analyses [4]. The guidance recommends using a participant flow diagram that demonstrates post-randomization losses and exclusions (with reasons) and to report the number of participants included in each analysis [4]. It does not however provide specific guidance on how to report methods to handle MD, assumptions about the MD mechanism, and MD sensitivity analyses. It could be argued that the application of the general CONSORT principles would encourage the reporting of all these [5], but it is unknown if this is the case.

Guidelines that specify additional criteria for reporting MD are available. Akl et al. [2] conducted a systematic survey of literature to identify recommendations for reporting MD in RCTs. They selected 13 articles including the CONSORT 2010 statement for parallel trials [6] and the CONSORT extensions for patient-reported outcomes [7], harm [8], and cluster trials [9]. From these recommendations, they provided reporting guidance covering the proportion, reasons, patterns, analytical methods, and interpretation of MD [2].

Several groups have provided further guidance on the prevention and handling of MD, including recommendations for reporting [1], [10], [11], [12], [13], [14]. The Methods Of Researching End of Life Care collaboration proposed that palliative care trials should specifically report the proportion of attrition because of death and illness [12]. Trialists are also encouraged to report comparisons of baseline characteristics between trial arms of participants with observed data and missing values to assess the effect of MD on the balance of trial arms for measured participant characteristics [13] - although it is important to note that trials are not necessarily powered for such analyses, and the absence of differences does not guarantee lack of bias.

No single method of accommodating MD is recommended for all trials [15]; however, principled methods, which do not attempt to replace the missing values directly but instead combine information from observed data with explicit assumptions about the MD, are advocated [16]. These methods, such as multiple imputation, generate statistical information about the MD and/or its mechanism, thus taking account of the uncertainty in estimating the missing values [10], [11], [16], [17]. In contrast, ad-hoc methods (e.g., complete case analysis [CCA] and last observation carried forward) either exclude participants with MD or impute a single value for the MD and then treat the data set as if it were the fully observed data [16]. These are generally discouraged as they rely on strong assumptions that are often unlikely to be true. CCA in particular is inefficient as it ignores the information that is typically provided by incomplete cases [16]. However, all methods to analyze MD are based on assumptions that cannot be verified from the partially observed data; therefore, conducting MD sensitivity analyses that explore the sensitivity of the results to different assumptions about the missingness mechanism is strongly recommended [11], [18].

An important consideration, especially in palliative care, is that data unmeasured because of death should not be considered as missing but rather as undefined [19]. Such data are referred to as ‘truncated due to death’ to distinguish them from MD in living participants [20]. The handling of truncated data presents a different problem to MD in those alive, and a distinction should be made between the methods to handle such data. This is a developing area of research, and several methods, taking different approaches to conditioning on survival, are advocated [20].

The compliance of palliative care trial reports with guidance on MD reporting and handling is unknown. Given the extent and risk of bias posed by MD in palliative care trials, clear reporting should be given particular attention in this field. Two reviews of trials across disciplines published in high-impact medical journals found that 45% of trials used CCA as their primary analysis [21], [22], and a MD sensitivity analysis was reported in 21% [21] and 35% [22], respectively. These reviews however assessed a limited number of criteria relating to MD reporting and primarily focused on the amount and handling of MD, without specifically differentiating between MD in those alive and truncated data due to death.

Furthermore, it is also important to understand which factors are associated with the implementation of MD reporting guidance. A Cochrane review of whether journal endorsement of CONSORT affected the completeness of RCT reports concluded that ‘despite relative improvements when CONSORT is endorsed by journals, the completeness of reporting remains suboptimal’ [23]. Moreover, a review that compared methodological characteristics of RCTs published in higher vs. lower impact clinical journals found that loss to follow-up was reported more commonly in higher impact journals (80% vs. 68%) [24]. Assessment of the effect of journal impact factor (JIF) and CONSORT endorsement status on the quality of MD reporting will be a useful first step to understand how implementation of MD reporting guidance can be optimized.

This systematic review of RCTs evaluating palliative interventions in patients with advanced life-limiting illness aimed to assess (1) to what extent MD are reported and handled in accordance with current reporting guidance, (2) whether quality of reporting differs between MD criteria specified by the CONSORT 2010 statement and MD criteria not specified by CONSORT, and (3) whether JIF and CONSORT endorsement status are associated with the quality of MD reporting and how this compared to the reporting of other methodological risks of bias such as allocation concealment and blinding.

Materials and methods

This systematic review was done as part of a larger project. Detailed description of the methods is available elsewhere [3].

Eligibility criteria

Eligible studies were RCTs that included participants with an advanced life-limiting illness with no possibility of remission and tested a palliative intervention that aimed to improve quality of life rather than modify the disease process or improve survival. Outcomes were patient reported and included among others symptom control, psychospiritual, and quality of life outcomes. Trials published between January 2009 and April 2014 were included to capture current reporting practice that overlaps with the publication of the CONSORT 2010 statement. There were no language restrictions.

Search strategy and study selection

CENTRAL, OVID MEDLINE, and EMBASE (January 2009–April 2014) were searched by an information specialist who combined the validated Cochrane PaPaS palliative care search strategy [25] and the sensitivity maximizing Cochrane Highly Sensitive Search Strategy for identifying RCTs in MEDLINE [26] (Appendix A; last searched May 2014; see on the journal's Web site at www.elsevier.com). A random sample of 100 trials was initially chosen from those identified by the search to obtain a representative rather than exhaustive sample for this methodological review of a broad subject area such as palliative care. However, a large proportion of the studies screened were not eligible for inclusion, and consequently, all the identified trials were screened to identify the desired number of trials. Two reviewers independently conducted the screening and selection of trials. Any discrepancies were discussed in a meeting, and arbitration by a third reviewer was not required. Authors were only contacted if there was insufficient information to make a decision about inclusion based on the published article and supplementary material.

MD criteria

In this review, MD are defined as observations that were intended to be made but were not [16]. This included MD that occur at different levels: (1) unit-level MD, where the participant was not available to provide data and (2) item-level MD, where individual questions from questionnaires, surveys, or scale measures are missing; and with different patterns: (1) monotone MD, where the participant withdraws completely from the trial and provides no further data and (2) intermittent MD, where the participant does not provide data at particular time points while remaining in the trial. The MD reporting criteria are listed in Table 1, and the definitions used to determine whether the criteria were successfully reported are presented in Appendix B (see on the journal's Web site at www.elsevier.com).

Table 1

Quality of MD reporting according to current reporting guidance

Reporting category	MD reporting criterion	Proportion of trials reporting criteria
Proportion of MD	Account for all participants who enter the studya	69% (75/108)
	Report number of participants not included in the primary outcome analysisa	94% (101/108)
	Report number of participants with MD in each arm in the primary outcome analysisab	87% (85/98)b
	Report amount of item-levelc MD in the primary outcome analysis	10% (5/50)d
	Report MD trend over time for primary outcomes measured repeatedly	All time points: 7% (5/69)Some time points: 48% (33/69)
	Report amount of MD for secondary outcomes	For all: 9% (9/99)eFor some: 18% (18/99)
Reasons for MD	Report reason for MDa	71% (66/93)f
	Report amount of MD because of death	65% (60/93)f
	Report amount of MD because of illness or disease progression	46% (43/93)f
Minimizing MD	Report plans to minimize MD	27% (29/108)
Risk of bias posed by MD	Report comparison of baseline characteristics of those with observed data	6% (6/93)f
	Report comparison of baseline characteristics of those with MD	0%
Justification of MD analytical approach	Report assumed mechanism of MDg	3% (3/108)
	Report criteria for MNAR (informative MD)	1% (1/108)
	Report pattern of missingness	0%
	Compare baseline characteristics of those with and without MD	13% (12/93)e
Statistical methods to handle MD	Report methods used to handle MD	48% (45/93)e
	Report methods used to handle truncated data because of death	5% (3/60)h
	Report MD sensitivity analyses	16% (15/93)f
	Report any changes to the planned MD analysis	0%
Impact of MD on the trial findings	Discuss impact of MD on the interpretation of findings	46% (43/93)f

Abbreviations: MD, missing data; MNAR, missing not at random.

The Consolidated Standards of Reporting Trials 2010 statement recommendation.

In non–crossover trials only.

For example, missing individual questions from a questionnaire, survey, or scale.

In 50 trials, the primary outcome was a scale summary.

In trials that measured secondary outcomes only.

About 15 trials reported no MD and therefore were excluded.

For example, missing completely at random, missing at random, or MNAR.

In trials that reported data was truncated due to death.

Data extraction

Duplicate data extraction was conducted. A data extraction form that combined the MD reporting recommendations of the CONSORT 2010 statement [27] with those by other groups was developed and piloted [1], [2], [10], [11], [12], [13]. Although further MD reporting guidance is provided in some of the extensions of the CONSORT statement [7], [8], [9], as MD are ubiquitous across trial designs, this review differentiated between the CONSORT 2010 recommendations for two-group parallel trials—which provide guidance for the reporting of all RCTs [6]—and those not specified in this guidance. If more than one primary outcome was reported or no primary outcome was specified, the first outcome reported was considered to be the primary outcome. For repeated measures, where the primary end point was not clearly specified, the final observation was taken as the primary end point. The reporting of other methodological risks of bias (random sequence generation, allocation concealment, and blinding) was also extracted, as were the reported reasons for MD and methods used to handle MD.

Data synthesis and analysis

The proportion of trials that reported each criterion and the reported reasons and methods to handle MD were tabulated. The relationship of the quality of reporting MD and other methodological risks of bias with the JIF (as reported by Thomson Reuters Journal Citation Report 2014 [28]) and with the CONSORT endorsement status of the journal (determined on July 7, 2015 in accordance with the list reported on the CONSORT Web site [29]) was assessed using logistic regression with either log-transformed JIF or endorsement status as the independent variable and the reporting criterion as the dependent variable. To reduce the risk of multiple testing, a priori it was decided to test only those criteria that were reported by at least 10% of trials and for the reasons for MD to test only if any reason was provided. Some journals published a number of trials, and such clustering was taken into account using the cluster command in STATA (StataCorp LLC, College Station, Texas), version 13 [30]. All analyses were conducted using STATA, version 13 [31]. The level of statistical significance was set at 5% for all analyses.

Results

The search yielded 1,923 identified references that were screened; 179 articles were read in full, and 108 were included in the review (Fig. 1). The full texts for four articles were not available (Appendix C; see on the journal's Web site at www.elsevier.com), and two included articles were translated from French and Chinese to English. Study demographics are presented in Appendix D (see on the journal's Web site at www.elsevier.com).

Fig. 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses study flow diagram.

Quality of MD reporting

Table 1 presents the proportion of included trials that reported the MD criteria specified by current guidance.

Proportion of MD

Sixty-nine percent of trials (75 of 108) accounted for all participants who entered the study, 94% (101 of 108) reported the number of participants not included in the primary outcome analysis, and 87% of non–crossover trials (85 of 98) reported the number of participants with MD in each arm. Fourteen percent (15 of 108) of trials reported that there were no MD. Of trials reporting a scale summary, only 10% (5 of 50) reported item-level MD. When the primary outcome was measured repeatedly, MD at each time point was reported by 7% (5 of 69) of trials. Only 9% (9 of 99) of trials reported the amount of MD for all secondary outcomes. However, for 59% (58 of 99), the proportion could be assumed to be the same as for the primary outcome as the reason for MD seemed to relate to unit nonresponse (i.e., the participant was missing) through the use of terms such as attrition or withdrawal—although the definition of these terms was not specified.

Reasons for MD

Of the trials that reported MD (n = 93), 71% (66 of 93) reported the reason for MD for all participants, 16% (15 of 93) for some participants, 9% (8 of 93) did not report any reasons, and in 4% (4 of 93), it was unclear. In this palliative care sample, 65% (60 of 93) of trials reported the proportion of MD because of death and 46% (43 of 93) the proportion due to illness. For more than half of all the participants with MD, the reported reason for missingness was described as ‘loss to follow-up’ or ‘withdrawal’ with no further qualification of how these were defined or description of the underlying reason for the ‘loss to follow-up’ or ‘withdrawal’ (Table 2). Only two trials specified that they differentiated between withdrawal from the trial and withdrawal from the intervention or control while remaining under follow-up.

Table 2

Reported reasons for MD (total number of participants with MD = 5,903)

Reason	% (Number of participants)
Death	18 (1,063)
Illness/disease progression	6 (342)
Adverse effects/events	2 (112)
Unclassifieda	53 (3,158)
Otherb	6 (371)
Not reported	15 (856)

Abbreviation: MD, missing data.

Described as loss to follow-up or withdrawal with no further details of the underlying reason.

These included, for example, reported exclusions from the analysis because the ‘treatment failed’ or there was a ‘protocol violation’, as well as ‘researcher error’, ‘response not evaluable’, and ‘care transferred to another setting’.

Minimizing MD

Twenty-seven percent (29 of 108) of trials reported an attempt to minimize MD during the design or conduct phase of the trial. The most common reported method was the exclusion of participants with an expected poor prognosis or poor performance status. Other methods included limiting the number of outcomes and frequency of data collection, sending telephone/mail reminders, flexibility in the protocol, and monetary incentives.

Risk of bias posed by MD

Six trials that reported MD compared baseline characteristics of participants with observed data; two conducted a significance test, of which one found evidence of a difference between trials arms at the 5% significance level. No trials compared the characteristics of those with MD in different trial arms.

Justification of MD analytical approach

Three trials reported in the Methods section that they assumed MD were missing at random [32], [33], [34]. One trial also specified the missing not at random (MNAR) assumptions that formed part of their sensitivity analysis [35]. No trials specifically reported whether the reasons for missingness were considered likely to be MNAR, that is, related to the value of the outcome that was missing. Twelve trials reported a comparison of the baseline characteristics of those with observed and MD; and in seven of these, there was a statistically significant difference in at least one characteristic.

Statistical methods to handle MD

The method of handling MD was reported in 48% (45 of 93) of trials with MD. In a further 37% (34 of 93) of trials, the method was not reported, but the text and tables indicated that only complete cases were included in the final analysis. In 15% (14 of 93) of trials, the method of analysis could not be deduced, and it was assumed that CCA was used as this is the default method in most statistical packages, and it is likely that if other methods to handle MD were used, these would have been reported. Of the trials that reported MD, 78% (73 of 93) used only one method of analysis: 60% (56 of 93) used or were assumed to have used CCA, 3% (3 of 93) other ad-hoc methods, and 15% (14 of 93) principled methods. Fourteen percent (13 of 93) of trials used more than one ad-hoc method, and 8% (7 of 93) used both principled and ad-hoc methods. Only three trials [33], [36], [37] reported that they handled truncated data due to death differently or separately to MD for other reasons, with only one using a recommended method that accounts for participants who died [37].

Sixteen percent (15 of 93) of trials used more than one method to analyze MD and reported a MD sensitivity analysis. Of these, 12 reported an ad-hoc method of accommodating MD as the primary analysis, and 11 used CCA and one other method of analysis.

Impact of MD on trial findings

The potential impact of MD on the interpretation of the trial results was discussed in 46% (43 of 93) of trials with MD. This was most often described in the limitations sections of the article, and few articles specifically discussed the potential for MD to bias the treatment effect estimate [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49].

Differences in quality of reporting between criteria

The four MD reporting criteria specified in the CONSORT 2010 statement were reported in more than 69% of trials (range, 69–94%) (Table 1). Additional criteria from other guidelines were less well reported (range, 0–65%), with comparison of baseline characteristics of those with MD, pattern of missingness, and changes to methods of handling MD not reported in any of the selected trials.

Reporting of MD according to JIF and CONSORT endorsement status

The median JIF of the included trials was 2.8 (interquartile range, 1.8–4.5; range, 0–55.9), and 41 trials were published in journals that endorsed the CONSORT statement. Univariable logistic regressions (Table 3) demonstrated that as the JIF doubled the odds of reporting all but the reasons for MD were increased. This was similarly the case for trials reported in journals that endorsed the CONSORT statement, except that the odds ratio for the reporting of plans to minimize MD was 1.00. At the 5% significance level, the odds of accounting for all participants, reporting the number of participants not included in the analysis, comparing baseline characteristics of those with and without MD, reporting method of handling MD, and reporting allocation concealment were statistically significantly increased as the JIF doubled. Journals that endorsed the CONSORT statement as of July 2015 only had statistically significantly higher odds of reporting both the method of handling MD and an MD sensitivity analysis (Table 3).

Table 3

Univariable logistic regressions of MD reporting criteria and other risk of bias CONSORT reporting criteria with JIF and whether the journal endorsed the CONSORT statement

Reporting criterion	JIF			CONSORT endorsement status
	Odds ratio per doubling JIF	P	95% Confidence interval	Odds ratio	P	95% Confidence interval
MD reporting criteria
Account for all participantsa	1.54	0.001	1.20–1.97	2.46	0.1	0.73–8.23
Report number of participants not included in the primary outcome analysisa	1.39	0.001	1.15–1.69	1.20	0.8	0.31–4.70
Report reasons for MDa,b	0.88	0.5	0.63–1.23	0.65	0.5	0.20–2.17
Report plans to minimize MD	1.16	0.17	0.94–1.42	1.00	1.00	0.40–2.49
Compare baseline characteristics of participants with and without MD	1.50	<0.001	1.20–1.87	1.11	0.83	0.42–2.92
Report methods used to handle MD	1.40	0.002	1.13–1.73	2.53	0.03	1.08–5.94
Report MD sensitivity analyses	1.20	0.4	0.81–1.80	3.48	0.03	1.15–10.50
Discuss impact of MD on the interpretation of findings	1.14	0.2	0.93–1.41	1.85	0.1	0.85–4.04
Other CONSORT 2010 risk of bias reporting criteria
Report method of sequence generationa	1.09	0.2	0.94–1.26	1.79	0.2	0.79–4.03
Report method of allocation sequence concealmenta	1.29	0.01	1.06–1.57	2.03	0.2	0.36–6.55
Report blinding participants & personnela	1.09	0.2	0.96–1.25	1.32	0.6	0.51–3.41
Report method of blinding outcome assessmenta	1.04	0.5	0.91–1.19	1.41	0.5	0.55–3.62

Abbreviations: MD, missing data; CONSORT, the Consolidated Standards of Reporting Trials; JIF, journal impact factor.

CONSORT 2010 recommendation.

Assessed as whether they reported any reasons for MD.

Discussion

MD in palliative care RCTs are not reported in a clear, complete, and transparent manner. Criteria specified by the CONSORT statement were better reported. However, item-level and secondary outcome MD were poorly reported, thus the proportion of MD reported in published articles is likely to be an underestimate. The reported reasons for MD were unclassified in more than half of the participants and the risk of bias posed by MD and justification of the analytical approach were the least well reported categories. Guidance on methods to handle MD was poorly implemented, and only one trial reported the use of a recommended method to handle data truncated by death. The odds of reporting most of the MD and other risk of bias criteria tested were increased as the JIF increased and in journals that endorsed the CONSORT statement.

One possible explanation for poorer reporting of MD criteria not specifically recommended by CONSORT is that the journal word count requirements restrict what could be stated. However, a review of MD in health technology assessment monographs, which do not have a word count limit, found MD was rarely discussed and that MD sensitivity analyses were only reported in 30% of trials [50]. This review preceded the CONSORT 2010 statement, which has addressed MD somewhat, but our study illustrates that further development of the CONSORT MD guidance could be a key step to the improvement of MD reporting.

The results of our review are consistent with a Cochrane review of the reporting of criteria recommended in the 1996 and 2001 CONSORT statements. This review found allocation concealment and the method for sequence generation to be statistically more likely to be reported in CONSORT-endorsing journals [51]. Inconsistencies in reporting may be explained partly by a study that found that only 41% of journal editors made the CONSORT statement part of their peer-review process, 47% made it part of their editorial process [52], and only 38% of journals mentioned CONSORT in their instructions to authors [52]. As the median JIF for the palliative care trials included in this review was only 2.8 and fewer than half of the trials were published in journals that endorsed the CONSORT statement, it is important to develop a better understanding of how to optimize implementation of the CONSORT statement across journals, so that future developments in the reporting of MD guidance can improve RCT reporting practice.

It is important that reasons for MD are reported. This is crucial to the assessment of how MD can be minimized, whether MD introduces bias through differential reasons for missingness across trial arms, and the most appropriate methods to handle MD. This review demonstrated that terms such as withdrawal and loss to follow-up are often used without being defined. Definitions of these terms vary in the literature, and there is no specification that the underlying reasons for withdrawal or loss to follow-up should be reported [53], [54], [55]. Future guidance on MD reporting should highlight this and recommend that the underlying reason is reported and that any ambiguous terms are defined.

Choice of methods to handle MD was also reported inadequately. CCA was used as the sole method to analyze MD in 60% of trials, although a large proportion of MD in this population is expected to be due to death and disease progression and therefore potentially cause bias in a CCA. Only 16% of trials conducted any MD sensitivity analysis with most not using a principled method for the primary analysis and only one study reporting the sensitivity of the effect estimate to an MNAR assumption. This is worse than the findings of two previous reviews of trials published in high-impact medical journals [21], [22]. Our review provides further evidence that better implementation of guidance to handle MD is required and importantly includes trials published across journals with a range of impact factors. Although CONSORT does not aim to make recommendations about how trials should be analyzed [6], it does indirectly affect trial analysis as transparent reporting can highlight gaps in implementation [6]. Therefore, we recommend that future guidance includes a specific requirement to report how MD are handled for both participants who are alive and those whose data were truncated by death (if applicable), with justification of the methods used, and an MD sensitivity analysis.

This review had several limitations. The protocol for the review was submitted for publication to PROSPERO (August 2014); however, as this was a methodology review with no direct health-related outcomes, it did not meet the eligibility criteria. It was then submitted to the Cochrane Methodology Review register (August 2014); however, this was not undergoing updates at this time because of resource constraints. In view of the time constraint to commence the review, the protocol was not submitted for external peer review although it was reviewed by all authors of this article and the director of a local trials unit. Because of the evolving nature of palliative care, identification of relevant literature can be difficult, and some trials may have been missed. Four identified trials could not be included as the full texts could not be retrieved. It was decided a-priori that data reported elsewhere would not be assessed because clinicians, researchers, and policymakers often rely solely on the peer-reviewed journal article to make a judgment about the methodological risks of bias, although supplementary online material was reviewed. Finally, it was not possible to determine retrospectively whether all the journals that endorsed the CONSORT statement did so specifically at the time of publication of each study.

This systematic review is the first to assess how well MD are reported according to current guidance. We have used these findings to provide further reporting recommendations that build on those proposed by the CONSORT 2010 statement [6] and other groups [1], [2], [10], [11], [12], [13], [14], [20], [56] (Fig. 2). Our recommendations focus specifically on what should be reported in trial reports rather than on which methods should be used to minimize and handle MD, in keeping with the scope of the CONSORT statement. The rigorous methods, evolving nature, and wide recognition and endorsement of the CONSORT statement [6] make it ideally placed to facilitate better reporting of MD. Incorporation of these recommendations into future CONSORT statements will potentially improve the awareness of journal editors, peer reviewers, researchers, clinicians, patients, journalists and policymakers on what should be reported with regard to MD to enable a more accurate assessment of the risk of bias posed; the quality of reporting; and the quality of trial conduct and handling of MD. Further research is required to understand how to improve the implementation of reporting guidance.

Fig. 2

Recommendations for reporting missing data.