Reporting dose in complex self-management support interventions for long-term conditions: is it defined by researchers and received by participants? A systematic review.

BACKGROUND: The minimum clinically effective dose, and whether this is received in randomised controlled trials (RCTs) of complex self-management interventions in long-term conditions (LTCs), can be unclear. The Template for Intervention Description and Replication (TIDieR) checklist states that dose should be clearly reported to ensure validity and reliable implementation.
OBJECTIVES: To identify whether the expected minimum clinically effective dose, and the dose participants received is reported within research articles and if reporting has improved since the TIDieR checklist was published.
METHODS: Four databases were systematically searched (MEDLINE, PsycINFO, AMED and CINAHL) to identify published reports between 2008 and 2022 for RCTs investigating complex self-management interventions in LTCs. Data on reporting of dose were extracted and synthesised from the eligible articles.
RESULTS: 94 articles covering various LTCs including diabetes, stroke and arthritis were included. Most complex interventions involved behaviour change combined with education and/or exercise. The maximum dose was usually reported (n=90; 97.8%), but the expected minimum clinically effective dose and the dose received were reported in only 28 (30.4%) and 62 (67.4%) articles, respectively. Reporting of the expected minimum clinically effective dose and the dose participants received did not improve following the publication of the TIDieR checklist in 2014.
CONCLUSIONS: Interpreting results and implementing effective complex self-management interventions is difficult when researchers' reporting of dose is not in line with guidelines. If trial findings indicate benefit from the intervention, clear reporting of dose ensures reliable implementation to standard care. If the results are non-significant, detailed reporting enables better interpretation of results, that is, differentiating between poor implementation and lack of effectiveness. This ensures quality of interventions and validity and generalisability of trial findings. Therefore, wider adoption of reporting the TIDieR checklist dose aspects is strongly recommended. Alternatively, customised guidelines for reporting dose in complex self-management interventions could be developed. PROSPERO REGISTRATION NUMBER: CRD42020180988. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first systematic review to explore whether dose is being reported as the guidelines recommend in randomised trials of self-management interventions.

Double-screening and data extraction were completed, following piloting, ensuring all eligible papers were included and accurate data extracted.

Determining complex self-management study eligibility was challenging, but we developed a systematic approach to limit potential bias.

Quality assessment of eligible papers was not conducted, but it could have been interesting to see if quality of study correlated with quality of reporting.

Background

It is estimated that 30% of the UK population live with a long-term condition (LTC) and that LTCs account for 70% of health and social care spending within the National Health Service (NHS).1 This prevalence extends globally, where LTCs are the leading cause of ill health and result in 70% of all deaths,2 with a growing awareness of the importance of monitoring prevalence and developing interventions to overcome LTCs, due to the ageing population, predicted increase in LTCs and the associated costs.3 4 Therefore, the management of LTCs is a priority for the NHS. LTCs are defined as ‘diseases of long duration and are the result of a combination of genetic, physiological, environmental and behavioural factors’.5 The current evidence base suggests LTC treatment should focus on supporting effective self-management to result in better health outcomes.6 Self-management here is defined in conjunction with the US Institute of Medicine definition, echoed by the Department of Health; ‘self-management is defined as the tasks that individuals must undertake to live with one or more chronic conditions. These tasks include having the confidence to deal with medical management, role management and emotional management of their conditions.’7 8

Complex self-management interventions are known to improve a variety of health outcomes in LTCs, including self-efficacy (confidence in ability to execute specific behaviours), patient activation (confidence, skills and knowledge to manage their own healthcare), self-rated health, clinical outcomes and social outcomes.9 Complex self-management interventions contain several interacting components that aim to change patients’ behaviour. However, determining which parts of the complex intervention are necessary to result in a potential benefit can be difficult. Therefore, complex self-management interventions should go through stages of development before being evaluated, typically in randomised controlled trials (RCTs), to identify how much of which components result in the best outcomes.10 Once decided on, at least the expected minimum clinically effective dose of the complex self-management intervention should be compared with standard care for the LTC to see if health outcomes improve. However, in published reports of RCTs, it is often unclear how the minimum clinically effective dose of the intervention was determined or, indeed, what the researchers believe the expected minimally clinically effective dose to be.

The concept of dose refers to the number of intended units of each intervention (dose delivered) and the extent of engagement of participants with the intervention (dose received).11 Treatment fidelity refers to the extent to which the intervention is delivered as expected, how much of the intervention is received and the amount of treatment enactment of the intervention by participants. Focusing on fidelity of treatment receipt, if the number and length of sessions received is in line with that stated in the protocol, it is essential researchers determine what they expect the minimum clinically effective dose to be and measure if it is received by participants within the trial, so fidelity of treatment receipt can be assessed.12 13 This is determined through discussions between those involved in the development of the intervention, to decide what they expect the minimum number of sessions attended and engagement with the intervention is to result in a meaningful change. There are two possible explanations for why this information is not reported, either researchers are not having these conversations during intervention development, or they are not reporting what this should be in their methods and papers. Collecting and reporting this information ensures the quality and integrity of the intervention and enables assessment of how valid and generalisable the findings are.11 Additionally, not stating the expected minimum clinically effective dose and if it has been delivered and received makes it difficult to interpret RCT results. If trial results are non-significant and fidelity of treatment receipt is not reported, it is unclear if this result is due to a lack of effectiveness or failed implementation of the intervention. Ensuring non-significant effects are due to lack of intervention effectiveness helps to avoid a type 2 error, whereby the treatment is deemed not effective when the findings are due to confounding variables, such as poor implementation.14

To improve the reporting of all types of interventions the Template for Intervention Description and Replication (TIDieR) checklist15 was developed in 2014. The 12 items explain how interventions should be described in published articles, so that trials with effective interventions can be replicated validly and implemented into standard practice reliably. The intervention details required for non-pharmacological interventions, such as the behavioural and educational components used in complex self-management interventions, are explained. Focusing on dose, item 8 of the checklist highlights ‘when and how much’, whereby RCT articles should clearly state the number of sessions in the intervention, their duration and over what time period they are delivered. Also, items 11 and 12 of the checklist state that the planned, delivered and received doses should be included to ensure both adherence and fidelity can be assessed (outlined in table 1). No previous, published reviews within the LTC complex self-management literature have reviewed whether dose and fidelity are being reported in this way.

Table 1

Extract from the TIDieR checklist of the relevant item descriptions for this review

TIDieR checklist item	Description
Item 8	When and how much: Describe the no of times the intervention was delivered and over what period of time including the number of sessions, their schedule and their duration, intensity or dose
Item 11	How well (planned): If intervention adherence or fidelity was assessed, describe how and by whom, and if any strategies were used to maintain or improve fidelity, describe them
Item 12	How well (actual): If intervention adherence or fidelity was assessed, describe the extent to which the intervention was delivered as planned

TIDieR, Template for Intervention Description and Replication.

This systematic review aimed to identify how complex self-management intervention doses for people with LTCs are reported in RCTs. We assessed this by evaluating whether what the researchers believed to be the minimum clinically effective dose was stated, how this dose was determined, if the dose received by study participants was stated and how it compared with the expected minimum clinically effective dose (fidelity of treatment receipt). We also aimed to determine if reporting of expected minimum clinically effective dose and treatment dose received improved following the publication of the TIDieR checklist in 2014. Finally, we aimed to identify whether reporting of expected minimum clinically effective dose and treatment dose received differed depending on whether the primary outcome results were statistically significant or not. We hypothesised that reporting of dose would have improved since the publication of the TIDieR checklist and that studies with non-significant primary outcomes may report dose more clearly than studies with a significant outcome in an attempt to explain their results.

Methods

Search strategy for systematic review and inclusion and exclusion criteria

The systematic review was conducted in accordance with Preferred reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)16 (online supplemental file 3). MEDLINE, CINAHL, AMED and PsycINFO were systematically searched. The full search strategies were developed in consultation with the UCL Library team and can be found in online supplemental file 1. Publications were included if published between January 2008 and June 2020, to identify if there was a trend towards improved reporting of treatment dose from 6 years before to 6 years after the TIDieR checklist was published (2014). An update of the review was conducted, searching the literature between June 2020 and January 2022. The same methodological process was followed.

Population: people with LTCs.5

Intervention: complex self-management support with structured session(s) (containing several interacting components that aim to change patients’ behaviour), delivered to people with LTCs.7 8

Comparator: any.

Outcome: any.

Study design: RCTs.

Does not include human participants.

Not a complex self-management support intervention with structured sessions, for example, exercise or psychotherapy only interventions.

Interventions delivered to carers, healthcare professionals, etc.

Only published as an abstract.

Ongoing studies.

Exclusion criteria

The articles from the database searches were exported into EndNote, duplicates removed and brief screening completed (e.g., removing systematic reviews). Those remaining were uploaded into Abstrackr (http://abstrackr.cebm.brown.edu/) and two reviewers (TAR and AB) independently screened titles and abstracts against the inclusion criteria, classifying articles as included, excluded and maybe eligible. For the update, Rayyan was used instead of Abstrackr, as the software was more user friendly. Forward and backward citation screening was performed on eligible papers. Identified discrepancies were discussed with ST to reach a final decision for full text data extraction.

Data extraction and analysis

Data were independently extracted by TAR and AB onto a word-based proforma designed for the study and any disagreements discussed until consensus was reached.

For all studies, we extracted trial authors, country, year of publication, intervention name, intervention description and components, LTC disease area, maximum intervention dose that could be delivered in the context of their study, expected minimum clinically effective dose, any rationale given for this, actual dose received, fidelity of treatment receipt and intervention delivery, and statistical significance of the primary outcome.

Within the articles, reporting of dose was determined by the number and length of sessions available to participants and how many they attended. Minimum expected clinically effective dose was either explicitly stated or stated as the number of sessions needed to be attended to be considered a ‘completer’ or to be included in the per protocol analysis. If no detail was provided, then this was recorded as ‘not reported’. An example of the data extraction process can be seen in online supplemental file 4. Due to the subjective interpretation of some data points, we piloted this process to ensure accurate and consistent interpretation. The items included from the TIDieR checklist are outlined in table 1.

As this was a review of trial reporting, rather than of trial findings, a formal quality assessment was not undertaken. Simple summary statistics were used to report the percentage of trials reporting the various aspects of dose.

No patients were involved in the research project.

Results

In the original search, after database searching and deduplication, 14 661 titles and abstracts were screened for data extraction and 124 full-text articles screened for eligibility, of which 82 were included in the synthesis. For the update, 2311 titles and abstracts were screened, 35 were full-text screened, with 12 papers included (see figure 1) PRISMA flow diagram.

Figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) systematic review flow diagram. LTC, long-term condition; RCTs, randomised controlled trials.

Characteristics of included RCTs

The population and intervention characteristics varied among the RCTs included. With 27 different LTCs investigated across the 94 articles, including diabetes, cancer survivors, chronic obstructive pulmonary disease (COPD), dementia, arthritis, stroke, serious mental illness and HIV. The complex self-management interventions investigated included Chronic Disease Self-Management Programme (CDSMP17), Arthritis Self-Management Programme (ASMP18), health education programmes,19–21 health education combined with exercise programmes,22–24 Cognitive Behavioural Approaches,25 26 and problem-solving and goal-setting.27–29 The number of sessions for each intervention ranged from 2 to over 30. A summary of the LTCs, self-management interventions and number of sessions are presented in tables 2–4, respectively. Further details of all included articles are supplied in online supplemental file 5, with the full reference list of included trials in online supplemental file 2.

Reporting of dose

Of the 94 trials included, 90 (97.8%) reported the maximum number of sessions that could be delivered, 72 (78.3%) reported the length of these sessions and 28 (30.4%) reported the expected minimum clinically effective dose. Of the 28 reporting the expected minimum clinically effective dose, 12 (42.9%) justified how this had been determined. In addition, 62 (67.4%) reported what dose participants received and 48 (52.2%) discussed if this was equal to, or greater than, that scheduled to be delivered in the protocol (fidelity of treatment receipt). It was unclear in 44 articles (47.8%) whether the expected minimum clinically effective dose had been received by participants, as no detail was provided. Of the 48 studies where this information was present, in 36 (75.0%) participants received the expected minimum clinically effective dose, which for 11 of these (22.9%) was also the maximum dose available.

No improvement in reporting of dose since the publication of the TIDieR checklist was observed. Of the 31 articles published between 2008 and 2014 and the 63 published between 2015 and 2022, 6 (19.4%) and 22 (34.9%), respectively, reported the expected minimum clinically effective dose. Of the 31 articles published between 2008 and 2014 and the 63 published between 2015 and 2022, 22 (71.0%) and 40 (63.5%), respectively, reported the number of sessions received and 15 (48.4%) and 28 (44.4%), respectively, reported the length of sessions received. The percentage of trials reporting the expected minimum clinically effective dose, as number of sessions and the treatment dose participants received per year are represented in figure 2.

Figure 2

Bar graph illustrating the percentage of trials reporting the expected minimum clinically effective dose and the treatment dose received by year.

Reporting of the expected minimum clinically effective dose or the dose received did improve based on the statistical significance of the trial’s primary outcome. Of the 55 articles with a significant primary outcome result and the 39 with a non-significant primary outcome result, 12 (21.8%) and 16 (41.0%), respectively, reported the expected minimum clinically effective dose. Of the 55 articles with a significant primary outcome result and the 39 with a non-significant primary outcome result, 31 (56.4%) and 31 (79.5%), respectively, reported the dose received.

Discussion

The included trials covered a variety of LTCs and self-management interventions. As expected, almost all the trials included in this systematic review reported the maximum number of sessions and just over three-quarters reported the length of sessions in the complex self-management intervention. Less than one-third reported the expected minimum clinically effective dose and, when this was reported, less than half explained how this had been determined. Just over two thirds reported the number of sessions dose received and under half reported length of sessions dose participants received and within these even fewer discussed whether there was fidelity of treatment receipt, that is, if the dose received was equal to or greater than that specified in the protocol. Improvements in the reporting of the expected minimum clinically effective dose or the dose received were not seen after the TIDieR checklist was published in 2014. However, there was an improvement in the reporting of these doses depending on whether the primary outcome was statistically significant or not, with those with non-significant results reporting the expected minimum clinically effective dose and dose received more often than those with statistically significant differences.

Results in context

In RCTs of complex self-management interventions in people with LTCs, it is often difficult for the maximum dose to be received by all participants, due to the complexity of both the participants’ disease and the intervention itself. However, the number of sessions attended and amount of contact with the intervention leader(s) is often associated with improved patient outcomes.20 30 It is well documented that receiving four of the six sessions available in CDSMP results in a beneficial clinical effect.31 Of the nine papers investigating CDSMP in this review, four papers discussed this minimum clinically effective dose. If no minimum clinically effective dose is stated, interpreting whether the dose participants received was greater than, or equal to, the minimum dose needed to see an improvement (fidelity of treatment receipt) is almost impossible, unless all participants receive the maximum dose available, which is uncommon.14 If the minimum clinically effective dose is stated and received by participants, then a negative result might be interpreted as an ineffective intervention. If the dose is not received then a negative result could be due to poor implementation of the intervention, rather than a lack of effectiveness. Therefore, by not reporting the dose received, potentially effective interventions could be abandoned, due to the results not being able to be interpreted in relation to the dose received, resulting in a type 2 error.14 32

If the dose received is stated and is low, further investigation can be done by trial authors or other researchers to determine how it relates to patient outcomes, that is, due to poor trial and/or intervention design. Collecting this information and reporting it enables those implementing the intervention to know what and how much needs to be received to ensure the best outcomes. In the Ackerman et al trial,33 27% of those approached to participate declined, as they could not attend all six ASMP sessions, and of those who were recruited many did not attend. Adaptations were made to avoid this, such as booking venues close to participants’ homes and scheduling on varying days and times. As the authors provided this detail, future researchers are aware of these potential challenges and, in their trials, could adapt the intervention to be delivered another way, that is, home-based, via telephone or web-based to make it more accessible and improve recruitment and retention. Also, if policymakers have this information when designing guidelines and making recommendations for scaling up interventions into standard care, effects seen in trials are more likely to be translated into routine care.34–36

In addition, researchers must take the time within the early developmental phases of an intervention to ensure the expected minimum clinically effective dose is estimated as accurately as possible, through pilot studies, systematic reviews and/or longitudinal research.10 Although difficult, this focus on early development would prevent fully funded RCTs going ahead when the minimum clinically effective dose has not been determined or measured.

Even when fidelity is mentioned within trial papers, the focus is often on how it was assessed rather than the actual findings, limiting the use of fidelity data to interpret the trial findings, and making the fidelity assessment almost useless.37–39 Understanding the reasons why fidelity is poorly reported is complex, but it is thought to be attributed to lack of knowledge and the practicalities of comprehensively assessing fidelity within an RCT.40 Despite the extra resources needed to conduct a full assessment of fidelity, the economic and scientific costs of not completing and reporting fidelity outcomes are far greater.14 Variations in intervention delivery within trials may influence efficacy and result in biased conclusions.

Although the TIDieR checklist was designed to improve reporting of interventions, no improvement in the reporting of the expected minimal clinically effective dose and dose received was found in this review. Also, within the articles, there was little to no mention of the TIDieR checklist and reporting of interventions in accordance with it, in line with other systematic reviews. Investigating implementation in the cardiovascular medicine literature, Palmer et al41 found over one-fifth failed to report the dose of the treatment received (item 11). Within behaviour change research similar results to this review have been found,42 with the maximum dose available always reported, but other elements of dose poorly described.

An improvement in reporting of dose was seen in studies reporting non-significant results. It is possible that, due to publication bias, reporting standards of studies that are published with non-significant results are of higher quality than studies with significant results.

An alternate explanation for poor reporting is that researchers may be less familiar with the TIDieR checklist, due to the dissemination being less extensive than other reporting guidelines, for example, Consolidated Standards of Reporting Trials (CONSORT) and PRISMA.41 Therefore, broader dissemination of the TIDieR checklist or incorporating the checklist within item 5 of the CONSORT statement, could improve reporting, as the information would be required by journals for publication.41 Poor implementation of the TIDieR checklist could also be due to the guidelines being too broad and generic and difficult for authors to adapt to their own interventions.43 Making the TIDieR checklist clearer and developing customised versions for specific intervention types could increase implementation of the checklist guidelines and ultimately improve intervention description and reporting.44

Limitations

The subjective nature of determining the eligibility of trials based on whether the intervention was a complex self-management intervention, could have introduced bias. All those marked potentially eligible were discussed by the study team to limit any potential bias and if there were any doubts the paper was included for data extraction. If consensus on eligibility could not be met, the paper was sent to a third reviewer (ST), with extensive experience in self-management support interventions for a final decision. Through these discussions, decisions around eligibility for inclusion were as consistent as possible given the flexible and varied definition of complex self-management interventions within the literature.

Also, a formal quality assessment was not completed, as we were not looking at the outcome measures. It could be of interest to compare the quality of study with the accuracy of dose reporting, but this was not within the scope and capacity of this review.

Future research

Following this review, reporting standards of complex self-management intervention doses do not appear to have improved since the publication of the TIDieR checklist. Ensuring that guidelines provide recommendations for how to define and assess dose within complex self-management interventions is vital for accurate reporting to enable interpretation and implementation of trial results. Therefore, either the TIDieR checklist should be updated or novel, specialised methodological guidelines developed to ensure that dose in these trials is determined, measured and reported as accurately as possible. Additionally, looking at whether quality of study correlates to quality of reporting dose could be completed.

Conclusion

Reporting of the minimum clinically effective dose, the dose received in the trial and the fidelity of treatment receipt are not consistent in studies of complex self-management interventions for LTCs. Although this detail is outlined in the TIDieR checklist, published in 2014, there has been no improvement in reporting following its publication. Currently, we recommend that when publishing RCTs, researchers should describe the intervention dose according to the TIDieR checklist. This will enable clinicians and policy-makers to reliably replicate the interventions in future trials and/or interpret findings to implement them into clinical practice. Going forward, the TIDieR checklist could be made clearer with versions for specific intervention types and wider dissemination of the checklist to increase implementation of the guidelines and improve intervention reporting. To facilitate this, funders, reviewers and journal editors should encourage dose and fidelity of treatment receipt to be collected and discussed, to increase reporting in this way.

Table 2

LTCs investigated in the 94 articles included in the systematic review

LTCs investigated	No of trials (%)
Type 1 and/or type 2 diabetes	25 (27)
Fibromyalgia	2 (2)
Epilepsy	2 (2)
Chronic hepatitis C	1 (1)
Cancer survivorship	4 (4)
Dementia/neurocognitive disorder	2 (2)
Hypertension	3 (3)
Arthritis	11 (11)
HIV	2 (2)
Spinal cord injury	3 (3)
Chronic obstructive pulmonary disease	4 (4)
Amputation	2 (2)
Stroke	8 (9)
Multiple sclerosis	1 (1)
Psychosis	3 (3)
Serious mental illness	3 (3)
Heart failure	3 (3)
Asthma	2 (2)
Myocardial infarction	2 (2)
Generic chronic somatic disease	1 (1)
Depression	1 (1)
Chronic kidney disease	2 (2)
Chronic fatigue syndrome	1 (1)
Coronary heart disease	1 (1)
Skin picking	1 (1)
Chronic pain	2 (2)
Multimorbidity	2 (2)
Total	94 (100)

LTCs, long-term conditions.

Table 3

Complex self-management interventions in the 94 trials included in the systematic review

Complex self-management intervention	No of trials (%)
Chronic Disease Self-Management Programme	9 (10)
Health education	32 (35)
Health education combined with exercise	14 (15)
Cognitive and behaviour change approach	10 (11)
Problem-solving and goal-setting	16 (17)
Arthritis Self-Management Programme	3 (3)
Other	10 (11)
Total	94 (100)

Table 4

Number of sessions delivered in the 94 trials included in the systematic review

No of sessions	No of trials (%)
1	0
2–6	44 (48)
7–12	34 (37)
>12	15 (16)
Unclear	1 (1)
Total	94 (100)