Modified intention to treat reporting in randomised controlled trials: systematic review.

OBJECTIVES: To determine the incidence and characteristics of randomised controlled trials that report using the modified intention to treat approach, and how the approach is described.
DESIGN: Systematic review. DATA SOURCES: PubMed, Embase, Cochrane central register of controlled trials, ISI Web of Knowledge, Ovid, HighWire Press, Science-Direct, Ingenta, Medscape, BioMed Central, Springer, and Wiley, from inception to December 2006. MAIN OUTCOME MEASURES: Incidence of trials in which use of modified intention to treat was reported, and how the approach was described (classified according to the type and number of deviations from the intention to treat approach).
RESULTS: 475 randomised controlled trials reported use of a modified intention to treat analysis. Of these, 76 (16%) were published in five highly cited general medical journals. The incidence of all trials that reported use of modified intention to treat published in journals indexed in Medline increased from 0.006% in 1982-6 to 0.5% in 2002-6 (P<0.001 for linear trend). When the description of the modified intention to treat was examined in each trial, 192 (40%) reported one type of deviation from the intention to treat approach, 261 (55%) reported two or more types, and 22 (5%) did not describe any type. In 266 (56%) of the trials the deviation was related to the treatment received, in 196 (41%) to a post baseline assessment, in 118 (25%) to a baseline assessment, in 108 (23%) to a target condition, and in 23 (5%) to follow-up. Post-randomisation exclusions occurred in 380 (80%) trials. The results reported by 270 of the 352 (77%) superiority trials favoured the drug under investigation. All of the 123 trials using equivalence or non-inferiority methods to investigate interventions reported results that favoured their assumptions.
CONCLUSIONS: Randomised controlled trials that report using a modified intention to treat are increasingly being published in the medical literature. The descriptions of such an approach were ambiguous, and may cover any type of descriptions for exclusion, such as missing data and deviation from protocol. Explicit statements about post-randomisation exclusions should replace the ambiguous terminology of modified intention to treat.

Introduction

Missing data and deviations from protocol may drive exclusions in randomised controlled trials. Excluding patients after randomisation may introduce non-comparability of characteristics across treatment groups and consequently lead to bias.1 2 3 4

The intention to treat principle implies that patients are analysed according to their original allocation, regardless of the treatment they actually received. Accordingly, withdrawals, losses to follow-up, and crossovers are ignored in a strict intention to treat analysis.5

Several studies of randomised controlled trials published in Medline and in top medical journals between 1993 and 19996 7 8 reported that fewer than half the trials used the intention to treat approach, and in many cases the term was inappropriately described or participants were improperly excluded.6

Following revision of the CONSORT (consolidated standards of reporting trials) statement,9 a study10 of 403 randomised controlled trials published during 2002 in 10 medical journals found a higher proportion of trials reporting use of intention to treat than before the statement was revised. Among trials that reported such an approach, however, only 39% analysed all participants as they were initially randomised, and the term “intention to treat” had many different interpretations.10 The introduction of ambiguous terminology into an intention to treat analysis can be misleading and may encourage investigators to inappropriately exclude participants after arbitrarily interpreting the intention to treat principle. Trials using a modified intention to treat approach are now appearing in the medical literature with varying descriptions. For example, a trial of an antifungal drug for the prevention of infection in neutropenic patients described the modified intention to treat as allowing “all patients that received at least one dose of study drug” to be included in analysis.11 This approach described a case of deviation from protocol which differs from that for cases of missing data, with “a modified intent-to-treat principle that included all randomised participants who had at least one post-baseline measurement for the primary outcome.”12

In 2004 an updated CONSORT statement warned that using the term modified intention to treat may lead to confusion and to inaccurate results.13 No appropriate survey has been done on the extent to which modified intention to treat is relevant to modern randomised controlled trials. We evaluated the incidence and characteristics of randomised controlled trials that reported using the modified intention to treat approach, assessed the significance given to the approach by authors, and discussed the implications of the approach.

Methods

We systematically searched PubMed, Embase, the Cochrane central register of controlled trials, and ISI Web of Knowledge and medical databases with full text journal articles (Ovid, HighWire Press, Science-Direct, Ingenta, Medscape, BioMed Central, Springer and Wiley). To identify relevant randomised controlled trials published in the English language from the inception of each database platform until 31 December 2006, we used the search terms “modified intention to treat”, “modified intent to treat”, and “modified ITT”. We independently screened titles and abstracts for potentially relevant studies and selected them on the basis of reading the full text versions. One reviewer also screened the references of reviews, systematic reviews, and meta-analyses to identify potentially eligible studies. Articles were included only if the study was a randomised controlled trial with at least one analysis done using the modified intention to treat approach. We excluded reviews, meta-analyses, cost effectiveness studies, pharmacokinetic studies, abstracts, posters, editorials, commentaries, letters, and published protocols of study designs.

Study selection and data extraction

Before systematically extracting data from each trial, we tested the data extraction method on a random sample of 30 articles. Interpretation of different items was discussed and classified. Subsequently, we independently and in duplicate extracted the following data using a standardised form: characteristics of the trial (including the journal and year of publication), type of primary and secondary outcomes reported, clinical area of interest, intervention studied, description of the modified intention to treat approach, any potential description of intention to treat, number of patients included in the study, P values, and the number of participants excluded from the analysis. We also extracted information on sample size calculations, reporting of equivalence or non-inferiority assumptions, reporting of funding, and authors’ competing interests.

Incidence calculation

We expressed incidence as the ratio of all trials that used the modified intention to treat approach over all the published randomised controlled trials in each year. To estimate the denominator for the incidence, we identified randomised controlled trials published each year from 1982 to 2006 by searching Medline (via Ovid) and limiting the publication type to “randomised controlled trial.” We also calculated the incidence of trials in general medical journals using modified intention to treat, where the denominator was represented by the randomised controlled trials published in the same general medical journals. According to ISI Web of Knowledge, the highly cited journals in general medicine are the New England Journal of Medicine, Lancet, JAMA, the Annals of Internal Medicine, and the BMJ.

Types of deviation from intention to treat

We retrieved the types and number of deviations from a true intention to treat analysis that occurred in analyses reported as modified intention to treat and accordingly classified the trials into six categories (box). To avoid self interpretation, we categorised the deviations by rigorously following the author’s descriptions of modified intention to treat and independently of other descriptions relating to exclusions of participants. We independently reviewed each eligible trial for deviation. Differences were resolved by discussion, and a consensus was reached.

Since exclusions seem to be the direct consequence of the modified intention to treat description, we tested the hypothesis that the number of deviations from intention to treat was positively associated with the proportion of occurrences of exclusions after randomisation.

Treatment—treatment received by participants: “mITT [modified intention to treat] population comprised all patients who were randomized to treatment and received at least one dose of study drug”14; “mITT consisted of patients who received at least six doses of study drug”15

Baseline assessment—presence of a baseline assessment: “mITT included patients with at least one baseline observation”16

Target condition—participants randomised but subsequently excluded from the analysis if found to lack the specific outcome or diagnosis at entry (either factor being difficult to determine or only suspected at enrolment): “The mITT population consisted of those patients who were randomly assigned to study treatment minus those who were not H pylori positive”17

Post baseline assessment—presence of a post baseline assessment: “mITT includes all randomized patients who have . . . at least one post-baseline measurement”18

Follow-up—lack of follow-up or participants failed to return for follow-up appointments: “All participants who completed follow-up were analyzed as a part of the group to which they were randomized. This was not a strict intent-to-treat analysis as some study participants were lost to follow-up”19

Other—trials that could not be placed into the other categories

Data analysis

We used the κ coefficient to determine the degree of agreement between reviewers. Categorical variables are described by frequencies and percentages. To evaluate the trend over time in the proportion of published randomised controlled trials that reported the use of a modified intention to treat approach, we used a χ2 test for linear trend with one degree of freedom. We used a χ2 test and a Kruskal-Wallis test with one degree of freedom to assess the hypothesis that trials reporting more than one deviation from intention to treat had greater proportions of post-randomisation exclusions than trials reporting one or no deviation.

Statistical analyses were carried out using Stata version 8.2. We considered P values less than 0.05 to be significant.

Results

The search generated 1010 records, containing 475 eligible randomised controlled trials published in 198 journals indexed by Medline (fig 1).

Fig 1 Study screening process

The estimated incidence of trials reporting the use of modified intention to treat analysis increased over time (P<0.001 for linear trend). At least 76 trials that reported using a modified intention to treat approach were published in highly cited medical journals: 35 in JAMA, 17 in the New England Journal of Medicine, 13 in Lancet, 9 in the Annals of Internal Medicine, and 2 in the BMJ. In four journals (JAMA, Annals of Internal Medicine, Lancet, New England Journal of Medicine), the overall incidence of randomised controlled trials in which use of a modified intention to treat analysis was reported increased from 0.3% in 1997 to 7.9% in 2006 (P<0.001 for linear trend; table 1).

Table 1

 Incidence of randomised controlled trials (RCTs) in which use of a modified intention to treat (mITT) analysis was reported

Years of publication	No (%) of RCTs using mITT
	Indexed in Medline	Published in 4 highly cited general medical journals
1982-6	1/15 958 (0.006)	—
1987-91	0/28 969 (0)	—
1992-6	6/46 493 (0.013)	—
1997-2001	71/56 427 (0.126)	9/1431 (0.6)
2002-6	378/72 178 (0.524)	65/1331 (4.9)

In 12 trials (0.2%) the modified intention to treat approach was used as a secondary analysis. In these studies the primary analysis was by intention to treat (11 trials) and by per protocol (one trial).

Of the 352 superiority trials, 270 (77%) reported statistically significant results that favoured the drug under investigation. All of the 123 trials using equivalence or non-inferiority methods to investigate interventions reported results that favoured their assumptions.

A for profit agency funded 418 (88%) of the trials. In 326 (68%) the authors explicitly declared financial ties with industry. In 13 of the 26 trials for which funding was not reported, at least one author wrote on behalf of a for profit agency. Table 2 describes the characteristics of the 475 randomised controlled trials (full details of the trials are available from the author).

Table 2

 Characteristics of randomised controlled trials (n=475) in which use of modified intention to treat was reported

Characteristics	No (%)
Specialty
Infectious disease	136 (29)
Neurology or psychiatry	67 (14)
Rheumatology	43 (9)
Cardiology	41 (9)
Oncology or haematology	28 (6)
Gastroenterology	22 (5)
Gynaecology and obstetrics	17 (4)
Other	121 (25)
Type of comparison
Active drug v active drug	249 (52)
Active drug v placebo	200 (42)
Non-drug interventions*	22 (5)
Invasive procedures†	4 (1)
Type of intervention
Drugs for infectious disease	136 (29)
Antibiotics	94 (69)
Antifungals	24 (18)
Antivirals	12 (8)
Vaccine	6 (4)
Cyclo-oxygenase-2 inhibitors	45 (9)
Lipid lowering drugs	21 (4)
Monoclonal antibodies	19 (4)
Growth factors	14 (3)
Bisphosphonates	14 (3)
Other	226 (48)
Study design
Non-inferiority	67 (14)
Equivalence	56 (12)
Phase II	19 (4)
Quality of reporting
Flow chart	234 (49)
Adequate sequence generation	251 (53)
Adequate allocation concealment	124 (26)
Blinding:
Double blind	364 (77)
At least patient and investigator (or outcome assessor) blinded	154
Single blind or open or not described	111 (23)
At least outcome assessor blinded	23
Reporting sample size	313 (66)
Sections reporting modified intention to treat
Title	0 (0)
Abstract	78 (16)
Introduction	4 (1)
Methods:
Statistical section	281 (59)
No statistical section	144 (30)
Results	215 (45)
Tables	164 (35)
Tables (only)	9 (2)
Discussion	38 (8)
Funding
For profit agency	418 (88)
Along with not for profit organisation	18
Not for profit organisation	31 (7)
Not reported	26 (5)
Report of competing interest
Reported employment with funding agency	201 (42)
Reported financial ties other than employment	125 (26)
No report of conflict of interest	125 (26)
Reported no potential conflict of interest	21 (4)
Support or employment from not for profit organisation or government	3 (1)

*For example, psychological, diet.

†For example, stent implant, surgery.

Types of deviation in the modified intention to treat description

Of the 475 trials, 192 (40%) reported one type of deviation from the intention to treat approach, 189 (40%) reported two, 67 (14%) reported three, five (1%) reported four, and 22 (5%) did not report any type.

In 266 (56%) trials, the main criterion for exclusion in the modified intention to treat analysis was treatment related. Moreover, in 185 (70%) such trials the treatment related approach was accompanied by at least one additional type of deviation.

Post baseline assessment related modified intention to treat was described in 196 (41%) trials. In 160 (82%) of these the approach was accompanied by at least one other type of deviation. In 118 (25%) trials, the exclusion used to justify the criteria for modified intention to treat was the absence of a baseline assessment, and in 108 (92%) trials it was associated with at least one other type of deviation.

Of the 108 (23%) trials describing a target condition based modified intention to treat, 78 (72%) listed at least one additional type of deviation. Only 23 (5%) trials based their criteria for modified intention to treat on loss to follow-up, and 11 (48%) of these included one or more type of deviation. Eighty two (17%) trials presented descriptions for a modified intention to treat approach that did not fall into any of the defined categories (table 3). Figure 2 shows the overlap among the four most common types of deviations.

Table 3

 Types and number of deviations from standard intention to treat in randomised controlled trials (n=470)* Values are numbers (percentages) of trials unless specified otherwise

No of deviations	Type of deviation							Median (interquartile range)
	Treatment related	Post-baseline assessment	Target condition	Follow-up	Baseline assessment	Other	Total	No of populations	Proportion of exclusions
None							22/475 (5)	222 (50-624)	5 (3-8)
One	81 (42)	35 (18)	28 (15)	12 (6)	10 (5)	26 (14)	192/475 (40)	219 (97-467)	6 (2-13)
Two:
First deviation	121 (64)	5 (3)	10 (5)	2 (1)	48 (25)	3 (2)	189/475 (40)	387 (217-633)	6 (3-13)
Second deviation		102 (54)	51 (27)	5 (3)	9 (5)	22 (12)
Three:
First deviation:	59 (88)		1 (1)	1 (1)	6 (9)		67/475 (14)	296 (177-531)	8 (2-16)
Second deviation		8 (12)	10 (15)	2 (3)	42 (63)	5 (7)
Third deviation		42 (63)	4 (6)	1 (1)		20 (30)

*Five trials reported four types of deviation: median sample 145 (interquartile range 142-409); median proportion of exclusions 36 (2-40).

Fig 2 Number (percentage) of four most common types of deviations from intention to treat. Each type of deviation overlapped with at least one other type

Post-randomisation exclusions

Overall, 380 (80%) of the trials reported exclusion of at least one participant owing to specifications for the modified intention to treat. In the remaining 94 trials no exclusions were reported, despite the description of the modified intention to treat approach. In one trial, the modified intention to treat described a patient crossover to another group. Of the trials reporting at least one exclusion, the median percentage of exclusions was 6% (interquartile range 2%-13%).

Statistical analyses did not indicate any positive association between the number of conditions provided in the description of the modified intention to treat and the proportion of participants excluded from the analysis (P=0.11).

Discussion

The publication of randomised controlled trials in which the use of the modified intention to treat approach is reported has significantly increased in the medical literature over time. Substantial variability was observed in how the approach was defined. In at least 58% of the trials that explicitly described the modified intention to treat approach, more than one criterion was reported. Therefore a predictable pattern of modified intention to treat is difficult to identify. Although an explanation for the emergence of use of a modified intention to treat approach in the medical literature remains unclear, this phenomenon may reflect the difficulty that investigators face when managing missing data or deviations from protocol (present in most trials) and at the same time avoid using the intention to treat terminology.

Missing data and modified intention to treat

Missing data are a technically challenging problem in randomised controlled trials because the outcome of interest is unknown and the application of a true intention to treat is impractical.20 21 The literature reports different types of missing data and different reasons for data being missed.22 Indeed, in our cohort of trials different terms were used, such as post baseline assessment, missing outcome, lost to follow-up, or “last observation carried forward,” which imply the reporting of missing data under the description of a modified intention to treat approach. It is possible that authors may want to define the modified intention to treat approach as the use of intention to treat is impossible in cases of missing data. However, the description of modified intention to treat is not a good indicator for missing data not only because different types of descriptions may be used but also because a missing data related modified intention to treat often can be associated with at least one other condition. In our analysis, 82% of the 196 randomised controlled trials using a post baseline assessment related modified intention to treat listed one other type of deviation from the approach. This association between two types of deviation may generate some difficulties in understanding which exclusions were related to a missing outcome. Usually missing data cases are known as “available cases” and the possible solution is either to omit the missing data or to use an imputation method, although controversies remain.20 23

Deviation from protocol and modified intention to treat

A separate issue is the deviation from protocol when participants are excluded. Several reasons are cited: patients did not start treatment or did not receive the entire treatment course, patients were ineligible because they were mistakenly enrolled or because the eligibility criteria were too broad, patients were randomised before information on eligibility was obtained and therefore became ineligible, and patients died or developed the outcome of interest before receiving treatment.6 24 All of these reasons were mentioned in the descriptions of the modified intention to treat reported in our survey. Indeed, possible deviations from protocol such as those related to treatment, baseline assessment, or target condition were associated with at least one other type of deviation in 70-92% of the cases. A substantial percentage of the missing data in these associations was related to this modified intention to treat approach, making any type of assumption difficult. In one trial, the modified intention to treat was described with three types of deviation and six patients were excluded: “modified intention to treat approach, including all patients who received at least one dose of randomised treatment, had a lipid measurement at baseline, and at least one lipid measurement after the start of treatment.”18 However, it is not possible to know which exclusions resulted from protocol deviation (absence of baseline assessment or treatment not received) or a missing outcome (absence of a post baseline assessment). Furthermore, if one relies on the strict description reported in the case of a deviation from protocol, the outcome of interest of a participant who is excluded should be known. In this situation, a true intention to treat analysis could still be used.

Non-inferiority and equivalence assumption and modified intention to treat

In non-inferiority and equivalence trials both an intention to treat analysis and a per protocol analysis are considered valid.25 However, an intention to treat analysis is often conservative and the main concern is that it may increase the chance of erroneously concluding non-inferiority or equivalence. In our survey we identified 123 non-inferiority or equivalence trials that reported on the use of modified intention to treat. In these trials excluding participants who did not adhere fully to the protocol can be justified. Exclusions may, however, affect the balance between the randomised groups and lead to bias if rates and reasons for exclusion differ between groups.26 27

Inconsistent description

Investigators sometimes were inconsistent in how they described the modified intention to treat approach—the same authors gave different descriptions in two publications, even though the treatment under investigation and the hypothesis being tested were the same in both studies.11 28 The exclusion rationale does not necessarily reflect the description. In one trial, investigators described a treatment related modified intention to treat approach and excluded three patients because they did not start treatment after randomisation,29 whereas the real reasons for these exclusions were stroke in one patient and withdrawal in the others. In another study it was unclear why the authors described a modified intention to treat (“all patients that took at least one dose of drug”) that they considered equivalent to intention to treat. In this study, all of the enrolled patients satisfied the criteria for a modified intention to treat and no patient was excluded from the analysis, indicating a true intention to treat approach.30 In a study testing a monoclonal antibody, the authors justified their treatment related modified intention to treat as a “standard practice in drug registration studies seeking to identify treatment effects,” ignoring the essential methodology of carrying out randomised controlled trials.31

Implications

The unpredictable and multiple deviations in the descriptions of modified intention to treat approaches may result in authors arbitrarily describing the approach and carrying out exclusions. For example, although in most cases of the descriptions of treatment related modified intention to treat the excluded patients were those who never received a drug, in 12% of these 266 trials the required amount of treatment for patients to be included ranged from two days to six months. These different choices are difficult to interpret.

Excluding participants is incompatible with intention to treat and may bias results. In comparison with the as treated or per protocol analysis, the intention to treat analysis is more conservative.1 32 33 34 35 Even a few exclusions may produce misleading results or become relevant when trials are considered for meta-analyses.3 A survey using individual patient data from 14 meta-analyses on cancer found an overestimation of the treatment effect when exclusions occurred.3 However, a meta-epidemiological study that used data from 14 meta-analyses on osteoarthritis reported that the extent and direction of bias due to post-randomisation exclusions can be unpredictable.4

In our sample of studies, we cannot know if the results might have been conservative if exclusions were avoided. However, 77% of the 352 superiority trials reported results that favoured the experimental drug. The prevalence of these trials with significant results was high and unexpected. The results might not reflect true differences, because the null hypothesis of no difference is likely to be true.36 Importantly, there is a tendency to report statistically significant results in published trials.36 This preponderance can be a consequence of many outcomes, and authors may selectively report secondary analysis depending on the P value.37 38 39 One study reported that the method described for handling protocol deviation was different from that described in the protocol in 19 of 43 trials examined.40 We were unable to investigate any possible discrepancy between the description of the modified intention to treat in the protocol and the one reported in the respective published trial. The erratic and multiple conditions under which modified intention to treat is reported may induce arbitrary multiple secondary analysis driven by the “level” of exclusion.

Strengths and limitations of the study

Our comprehensive search of 12 databases yielded 475 randomised controlled trials. The abstracts did not always report on modified intention to treat; terminology related to a modified intention to treat approach was present in only 16% (table 2) and therefore eligible trials would be missed by searches of general databases, such as Medline or Embase. Most of the included articles were from databases that host electronic journals. Our study may have underestimated the incidence of trials using the modified intention to treat approach because many journals are not hosted by databases that provide full text articles. Additionally, trials labelled as using the intention to treat approach when they used the modified intention to treat approach owing to the type of deviation present in the description of the analysis may have escaped our search. Therefore, this study may have further underestimated the incidence of trials using the modified intention to treat approach.

We included trials that reported the use of modified intention to treat analyses only. This criterion limited our ability to compare funding, conflicts of interest, methodological quality, and post-randomisation exclusions between trials that report using standard intention to treat analyses and those that report using modified intention to treat. Further studies are necessary to address these issues.

Conclusion

Published trials have deficiencies in their design.41 However, reports indicate that journal endorsement of the CONSORT statement is associated with improvement in the quality of reporting of randomised controlled trials.42 43 In this scenario, the increasing appearance of randomised controlled trials that report use of the modified intention to treat approach can be considered a shortcoming in the quality of reporting. The misuse of the intention to treat definition or the introduction of the modified intention to treat term cannot overcome problems related to missing data or protocol deviations. Inconsistent and multiple descriptions of the modified intention to treat approach may cause failure to achieve the standard intention to treat principle, which is the best method that preserves randomisation. We recommend the intention to treat approach as the method of choice for analysis in trials investigating the superiority of an intervention. Authors should provide complete information on post-randomisation exclusions if there are appropriate reasons to exclude participants.

The intention to treat principle is often improperly described and inadequately applied

Deviation from intention to treat by excluding participants from analysis after randomisation, may introduce bias

The publication of trials that report on use of a modified intention to treat is significantly increasing in the medical literature

The descriptions of the approaches described deviations from the true intention to treat method generating post-randomisation exclusions

Overlapping descriptions of missing data with deviation from protocol cases render the modified intention to treat challenging for interpretation