Systematic scoping review of frameworks used to develop rehabilitation interventions for older adults.

OBJECTIVES: Rehabilitation interventions for older adults are complex as they involve a number of interacting components, have multiple outcomes of interest and are influenced by a number of contextual factors. The importance of rigorous intervention development prior to formal evaluation has been acknowledged and a number of frameworks have been developed. This review explored which frameworks have been used to guide the development of rehabilitation interventions for older adults.
DESIGN: Systematic scoping review.
SETTING: Studies were not limited for inclusion based on setting. PARTICIPANTS: Studies were included that featured older adults (>65 years of age).
INTERVENTIONS: Studies were included that reported the development of a rehabilitation intervention. PRIMARY AND SECONDARY OUTCOME MEASURES: Data were extracted on study population, setting, type of intervention developed and frameworks used. The primary outcome of interest was the type of intervention development framework.
RESULTS: Thirty-five studies were included. There was a range of underlying medical conditions including mild cognitive impairment and dementia (n=5), cardiac (n=4), stroke (n=3), falls (n=3), hip fracture (n=2), diabetes (n=2), breast cancer (n=1), Parkinson's disease (n=1), depression (n=1), chronic health problems (n=1), osteoarthritis (n=1), leg ulcer (n=1), neck pain (n=1) and foot problems (n=1). The intervention types being developed included multicomponent, support based, cognitive, physical activities, nursing led, falls prevention and occupational therapy led. Twelve studies (34%) did not report using a framework. Five frameworks were reported with the Medical Research Council (MRC) framework for developing and evaluating complex interventions being the most frequently cited (77%, n=17).
CONCLUSION: At present, the MRC framework is the most popular for developing rehabilitation interventions for older adults. Many studies do not report using a framework. Further, specific guidance to assist this complex field of rehabilitation research is required. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Rigorous intervention development prior to formal evaluation is important.

A number of frameworks have been developed but are inconsistently used or inconsistently reported.

Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews guidelines were followed for this systematic scoping review.

An electronic database search aimed to capture all studies reporting intervention development.

Studies involving older people and specifically reporting the development of a rehabilitation intervention were included.

Introduction

‘Rehabilitation is concerned with lessening the impact of disabling conditions’ (Young, p677)[1] and is a complex process requiring a holistic approach that considers physical, social and psychological function. Rehabilitation interventions for older adults are complex as they involve a number of interacting components, are often tailored to individual needs, have multiple outcomes of interest and are influenced by a number of environmental and contextual factors.2

The need to develop a robust evidence base for complex rehabilitation interventions has led to an increased focus on developing and evaluating these interventions. Interventions initially showing promise in small-scale testing are often ineffective when scaled into large multicentre randomised controlled trials (RCTs). For example, an inpatient falls prevention programme that was effective during an observational study,3 failed to prevent falls to a significant degree compared with a control in a multisite RCT.4 A review, including this example, explored the reasons for the difference in outcomes, citing different contextual factors (staffing, length of stay).5 However, while intervention development was reported by this example, a framework was not used and may have resulted in a lack of sound theoretical underpinning and understanding of the intervention mechanisms of action. The importance of rigorous intervention development prior to formal evaluation has been acknowledged by healthcare researchers in other fields6 and a number of frameworks have been developed. These frameworks include the Medical Research Council (MRC) guidance for developing and evaluating complex interventions,7 Criteria for Reporting the Development and Evaluation of Complex Interventions (CReDECI),8 intervention mapping9 and the 6 Steps in Quality Intervention Development (6SQuID).10 Although there are a number of intervention development frameworks, the lack of methodological detail and specificity to rehabilitation interventions may mean that researchers are using the frameworks in different ways or not using the frameworks at all.

Therefore, the aims of this review were to (1) to ascertain if intervention development frameworks are being used in older people rehabilitation research, (2) to document which frameworks have been used and (3) to explore how those frameworks are being used, what methods are employed, and how much detail is provided. This review will help researchers and clinicians to consider a range of frameworks for their studies and is the first step towards establishing more detailed guidance.

Methods

Review design

Systematic scoping review. This study was initially designed as a systematic review but was adapted at a late stage due to advice from reviewers and the editors.

Inclusion criteria

Target population of intervention

Studies were included if their participants were older people who were >65 years (either through study inclusion criteria, mean sample age of study population or are described as older or elderly).

Intervention

The interventions being developed or described focused on rehabilitation. The definition of rehabilitation used was ‘the process of returning to a healthy or good way of life, or the process of helping someone to do this after they have been in prison, been very ill or the process of returning something to a good condition.’11 To be a rehabilitation intervention, the paper had to report that the intervention: involved the individual(s) being rehabilitated; consisted of more than one session to indicate a process; aimed to create a change in the individual(s)’ state or ability from doing the intervention; took place either after something or to prevent something (eg, an incident/illness) and was described or labelled as ‘rehabilitation’ by the authors.

Types of studies

Studies were included if they stated an aim or intent to either report the intervention that had been developed or to document the process or synthesis as justification or background for the next stage of intervention testing. This included mixed-method studies, RCTs, controlled clinical trials, experimental studies, qualitative based analysis studies, cohort, cross-sectional and case–control studies. Systematic reviews (all types) were considered for inclusion so reference lists could be explored for further studies that may not have been identified in the search strategy. Types of publications were also considered. Study protocols were considered for inclusion, however, abstracts, thesis, dissertations and conference proceedings were excluded due to the level of detail characteristic of these manuscripts (eg, limited word counts with abstracts and significant word counts with thesis). Where possible if studies were part of a series of publications, the other material available was sought and the most prominent paper detailing the intervention development process included.

Types of data and outcomes

Studies were included if they reported or described ‘intervention development’ or ‘developing an intervention’. Studies without a framework were included but only if they met the predetermined criteria that sufficient information and detail on the intervention development process or methods was presented. Studies that claimed to have completed an intervention development process but did not include any information on the process or method were excluded due to lack of data. All studies were assessed for inclusion by two authors and any discrepancy on the decision of a paper was discussed by all authors to reach a group consensus.

Studies were not limited nor selected according to their outcomes.

Search methods for identification of studies

The search strategy aimed to find both published and unpublished studies. A phased search strategy included search terms: ‘developing and evaluating complex interventions’ (all fields), ‘development’ OR ‘develop*’ (title), ‘intervention’ (title), ‘older’ OR ‘old*’ OR ‘elderly’ (all fields).

Electronic searches

Initially, a limited search of MEDLINE and CINAHL was undertaken to identify and refine index terms used to describe relevant articles. Index terms and keywords were taken from known studies that reported their intervention development process and the search strategy refined to ensure these papers were captured.

A second full search using all identified keywords and index terms was then undertaken across relevant databases, including: The Cochrane Central Register of Controlled Trials (The Cochrane Library, latest issue), MEDLINE, EMBASE, AMED, CINAHL and PsycINFO. The search was completed in October 2017. A full search strategy is presented in online supplementary material 1.

Searching other resources

The reference lists of full-text studies were searched for related material that could be included or were more relevant for data extraction.

Data collection and analysis

Selection of studies

Each study identified for inclusion was considered independently by two reviewers at all stages: title screening, abstract screening and full paper review for inclusion. Discrepancies between reviewer’s decisions were recorded and discussed between the other authors to achieve an outcome.

Data extraction and management

Data were extracted from the included papers using a bespoke data extraction tool, the main categories of which were: study population, setting, type of intervention developed and frameworks used. If a framework was cited then a more detailed review of the components used was completed. Microsoft Excel was used as the data management software and compiled into a single database once agreement of included studies and data extraction had been completed. The review has been reported according to Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines and a checklist completed.12

Assessment of methodological quality in included studies

Included studies were not assessed for methodological quality. However, each study was critiqued according to the latest reporting standards for the development and evaluation of complex interventions in healthcare (CReDECI2).13 Assessment of the reporting standard of the studies would not influence their inclusion in the review.

Data synthesis

Data were collated and narratively described using tables and text.

Patient and public involvement

Patients were not involved.

Results

Thirty-five studies were included in the review for data extraction.14–48 The flow diagram depicting the number of studies identified and excluded at each stage is provided in figure 1.

Figure 1

Flow diagram depicting the number of studies identified and excluded at each stage (each stage of the review process is depicted by a box in a sequential design as suggested by PRISMA-ScR12). PRISMA-ScR, Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews.

Description of the included studies

The most common population descriptor was community dwelling or older adults of a certain age.17 19 27 29 31 44 45 Underlying conditions included mild cognitive impairment and dementia (n=5),18 28 30 32 34 cardiac conditions (n=4),15 24–26 stroke (n=3),22 38 40 falls and fear of falling (n=3),21 35 48 hip fracture (n=2),37 39 diabetes (n=2),14 41 42 breast cancer,20 Parkinson’s disease,16 depression,23 chronic health problems,33 osteoarthritis,36 leg ulcer,43 neck pain46 and foot problems.47

The types of interventions reported were varied and included multicomponent (n=12),14 19 23 24 26–29 36 39 42 45 support based (n=5),15 16 18 30 41 cognitive interventions (n=5),31 33 34 40 48 physical activities (n=3),17 32 46 nursing (n=2),25 43 falls prevention,21 occupational therapy,22 poststroke care,38 podiatry47 and dietary advice.37

The included studies were from the UK (n=17),14 15 17 19 20 22 31 34–42 47 Netherlands (n=6),26 27 30 44 45 48 USA (n=4),18 24 25 28 Canada,16 India,23 Germany,29 46 Hong Kong,32 Italy,33 and Belgium.43 Ten studies were linked to other publications reporting the same intervention or other aspects of the development process.22 25 26 28 29 31 35 39 42 43

The reporting standard of the included studies was mixed with an average score of 4.4 (range=1–13) out of 13. All reported elements of the development and pilot phase of the checklist with only four studies reporting the evaluation stage.27 35 36 44 A table of the reporting standards for all included studies is provided in online supplementary material 2.

What frameworks were reported

Thirteen studies did not report using a framework to assist their intervention development.18 21 23–25 28 32 34 35 37 46 48 In total five frameworks were reported. The MRC guidance was the most frequently used (77%, n=17).15 19 20 22 26 27 29 30 33 36 38–42 47 The other frameworks were intervention mapping (n=3),16 44 45 conceptual modelling (n=1),31 intervention/programme theory (n=1)17 and the Van Meijel model (n=1).43 Descriptions and key references for the frameworks are provided in online supplementary material 3.

What methods were used for the different framework sections

MRC guided studies

A variety of different methods were used in the different stages of the MRC guidance within the included studies (see table 1). Most15 22 27 30 38 40–42 reported their intervention development process according to the three MRC framework stages. These are: (1) identifying the evidence base, (2) developing theory and (3) modelling processes and outcomes. Some only referenced the guidance and did not report the stages as distinct phases19 20 29 36 39 47 or described their own stages (such as evidence exploration, tune-up with experts and fine-tuning with patients).33 Three papers adapted and added a fourth stage their development process.14 26 40

Table 1

Presentation of the methods used for each element of the Medical Research Council (MRC) framework

Study reference	Methods used in IDF element (a)	Methods used in IDF element (b)	Methods used in IDF element (c)	Methods used in IDF element (d)
MRC guidance
Avery et al 14	Exploratory work Interview with general practitioners Interactive workshop (patients)	Identification of active intervention ingredients Systematic review	Assessing usability Use by adults with type 2 diabetes Structured interview	Pilot study Acceptability and feasibility (semistructured interviews)
Barley et al 15	Studies to inform intervention Systematic review Qualitative study (clinicians and patients)	Integration of findings Findings from the informative studies Iterative evidence review	Modelling of the intervention Focus group Evidence review
Bruce et al 19	Systematic reviews Clinical guidelines review Expert views Observations (clinicians) Piloting of manual (patients)
Burgess et al 20	Phase 0 (theoretical) Review of literature Expert consultation	Phase I (piloting and modelling) Pilot study Qualitative interviews with participants to explore acceptability
Cunningham et al 22	Identify evidence Review of literature (clinical guidelines, systematic reviews)	Model the intervention for delivery Piloting of manual (patients)	Test feasibility Piloting of intervention
Ettema et al 26	Identified existing evidence Systematic review	Identified and developed theory Systematic review Derived the questionnaire Analytical study (patient characteristics/outcomes)	Modelled process and outcomes In-depth interviews (patients) Survey (clinicians)	Face validity Expert meetings (national experts) Expert meetings (clinicians)
Faes et al 27	Existing evidence Literature reviews Project team meetings	Theoretical understanding Literature review Focus groups (experts) Interviews (patients and caregivers) Observations Expert meetings	Intervention modelling Focus groups Delphi surveys Interviews (patients and caregivers) Literature review Project team meeting Observations Interviews (experts) Expert consultations
Hinrichs et al 29	Development Literature review Cohort study (patients)
Kerkhof et al 30	Theoretical Literature reviews Focus groups Design of tool (users and stakeholders) Mock-up and testing of app (patients) Interviews Development of theoretical framework and manual	Modelling Pilot study Interviews Observational analysis Case study (methods used) Questionnaires Inductive content analysis	Exploratory trial Exploratory randomised controlled trial (RCT) Literature search Quantitative study Qualitative evaluation
Menichetti and Graffigna33	Evidences exploration Systematic review	Tune-up with experts Expert group discussion	Fine-tuning with patients Semistructured interviews
Patel et al 36	Literature review Pilot study Process evaluation (observations of programme delivery, participant interviews)
Redfern et al 38	Preclinical phase Literature review Analysis of current service Interviews (patient representatives) Observational study (patients) Reviewing patient information leaflets	Phase 1: modelling Consensus meeting (researchers and experts) Modification of data collection database Developing computer algorithm Development of patient intervention leaflets	Phase 2: exploratory trial Pilot study (semistructured interviews)
Roberts et al 39	Development of the intervention (phase 1 of MRC) Realist literature review Surveys (patients and rehabilitation teams) Focus groups (patients and rehabilitation teams)
Sadler et al 40	Identifying existing evidence and theory Literature search	Developing the theoretical foundation of the intervention Qualitative literature review Interviews (patients, spouse, carers and professionals) Stakeholder consultation (researchers, clinicians and service users) Scoping of literature	Modelling process and outcomes No formal method given ‘designed’	Assessing feasibility of the intervention Feasibility study (questionnaires preintervention and postintervention, qualitative data from participants and professionals delivering intervention)
Sturt et al 41	Preclinical phase Literature search	Phase I studies Iterative process between evidence and intervention components Study (patients)
Troughton et al 42	Development ‘iterative process’ Team and expert meetings Literature review Qualitative study (observation, telephone and face-to-face interviews and focus groups) Pilot study (intervention)	Feasibility and piloting Phased pilot study
Wylie et al 47	Remodelling of intervention (feasible and acceptable in setting, refined recruitment processes and outcomes) Pilot RCT (intervention)

IDF, Intervention Development Framework.

All except the study by Wylie et al 47 reported using a literature review in their development work. The literature review was most commonly used to identify relevant evidence or theories to underpin the intervention being developed (n=11). Other methods used included: expert consultation (n=2), qualitative interviews with either clinicians or patients (n=7) and observations or surveying patients (n=8).

A variety of terms were used to describe the second stage of their development process, with some categorising this as theoretical development, whereas others were focusing on modelling. There was a wide range of research methods reported in this second stage, including literature reviews (n=4), expert consultations (n=3), qualitative interviews and focus groups (n=4), observations (n=2), and pilot studies (n=5).

Nine studies then described a feasibility or modelling stage.14 15 22 26 27 30 33 38 40 This phase included pilot studies (n=3), qualitative focus groups and interviews (n=6), where data were collected. One study27 reported eight different research methods at this stage including a Delphi consensus process.

The four studies that added a fourth stage into their development processes varied in terms used to describe it, including ‘pilot study’,14 ‘face validity’26 and ‘assessing feasibility of the intervention’.40 Two of the studies reported completing a pilot or feasibility study within this stage14 40 whereas the third included expert meetings.26

Other framework guided studies

Six studies used a variety of intervention development frameworks.16 17 31 43–45 Reporting of the research methods used in these studies were varied even when the same framework was used (table 2). Intervention mapping9 was used in three studies, one of which provided no detail on the methods used in each section,16 whereas the other two reported very detailed processes and methods.44 45 Table 2 describes the different intervention development frameworks and the research methods used within each framework.

Table 2

Presentation of the methods used for each element of the other intervention development frameworks

Study reference	Methods used in IDF element (a)	Methods used in IDF element (b)	Methods used in IDF element (c)	Methods used in IDF element (d)	Methods used in IDF element (e)	Methods used in IDF element (f)
Intervention mapping
Beaudet et al 16	Assessing needs and preferences Interviews (patients)	Developing intervention Theory and model selection Intervention proposal validation (patients and stakeholders)	Formalising	Testing and evaluating Pilot testing (intervention)
van Stralen et al 44	A needs assessment of the study population and the definition of programme objectives Literature search Focus group interviews (patients) Interviews (stakeholders)	Defining the performance objectives, specifying what changes are needed Literature review Delphi study (experts) Theoretical models review	Selecting theory-based intervention methods and practical strategies to change health behaviour and its determinants Literature search Search of existing interventions Focus group interviews (patients)	Developing an intervention programme in which all strategies are integrated, as well as selecting, testing and producing intervention materials Brainstorming sessions (experts and patients)	Developing a programme adoption and implementation plan Pilot study (implementation and recruitment)	Anticipating a process and effect evaluation of the programme Process and effect evaluation
Walters et al 45	Needs assessment Literature search Survey Project management group consultations Interviews (experts)	Programme objectives Survey Literature search Project management group consultations Interviews (experts and workers)	Theory-based methods and practical applications Literature search Project management group consultations Interviews (experts and workers)	Programme plan Project management group consultations Interviews (experts and workers) Pilot study (training)	Programme implementation Literature search Consultation with stakeholders Idea collection (workers and instructors)	Evaluation plan Evaluation (questionnaire and discussion of workers and training)
Conceptual modelling
Kingstone et al 31	PPIE involvement	Development of conceptual model Interviews (patients and clinicians	Agreement of conceptual model Consensus process (researchers)
Intervention/programme theory
Blamey et al 17	Logic model of intervention theory
Van Meijel model
Van Hecke et al 43	Collection of building blocks needed for the design of the intervention Literature review Interviews (problem and needs analysis) Focus groups (clinicians)	Intervention design Expert commentary	Validation of the nursing intervention Qualitative study (patients) Evaluation

IDF, Intervention Development Framework; PPIE, Patient and Public Involvement and Engagement.

Discussion

Principal findings

Thirty-five studies were included in the review for data extraction. Twelve studies did not cite a framework to assist their intervention development. Five frameworks were reported by the 23 studies who did use one. The MRC guidance was the most frequently used with 77% (n=17). The other four frameworks were intervention mapping, conceptual modelling, intervention/programme theory and the Van Meijel model but these were only used in a small number of studies. Of the numerous potential frameworks, researchers could be using this study highlights that most researchers felt that the MRC, is at present, the most appropriate for their use. Although the quality of the studies in this review was not measured against any standardised measure, the studies that used the MRC guidance provided considerably more details about the components of intervention development than the studies using other frameworks providing a greater degree of confidence that the results had been rigorously collected and not biased. This may indicate that the MRC is written in a way that helps researchers follow a process more easily. However, three studies also adapted and added to the MRC process, indicating that there are further aspects to consider that are not addressed in that guidance.

Strengths and weaknesses of this study

This review was conducted in line with PRISMA-ScR guidelines following a systematic process, using predefined eligibility criteria and independent assessment by two reviewers at each stage. As with all reviews, there may have been studies that were missed due to the parameters of the review, such as, the definition of rehabilitation that was used. Data extraction was completed using a standardised spreadsheet by all authors and despite regular review meetings, there was discrepancy in the interpretation of research methods and the level of detail extracted. For example, what is counted as a ‘literature review’ could for one study be a Cochrane review while for another it is a non-systematised narrative description of the field of research.

The findings of this review are limited by the information available about intervention development within the identified literature. It is acknowledged that many journals prefer to publish detail on the intervention content with little focus on the development process and this was evident in this review. Intervention development frameworks are a relatively recent development and studies conducted before the MRC guidance was introduced in 2000 may have had limited methodological literature to guide their intervention development. This review searched all literature from the date of inception of the electronic database and this search strategy may have biased the number of studies not reporting the use of a framework. It is anticipated that over the coming years there will be many more studies reporting the use of a framework and providing more details on that process. Details on the intervention development may not be through journal publications, but through online supplementary material, discussion series, study or institution websites, or online data repositories.

This review did not report on the quality of the studies. While quality assessments are standard practice in systematic reviews,49 there is not yet a quality assessment tool for intervention development studies. The intention of this review was to make comment on the current state of the literature relating to intervention development. Studies were not included or excluded based on their quality, but on their detail of the intervention development process and methods. A critique against the reporting standards was included as a compromise and to compare the included studies to the recognised publishing standards.

Strengths and weaknesses in relation to other studies

To our knowledge, this is the first review of intervention development frameworks used in developing rehabilitation interventions for older people.

The MRC guidance7 from the UK provides a structure to the development and evaluation process for complex interventions. However, the MRC guidance is brief and has been criticised for not dealing well with the complexity of complex interventions.50–52 Although the MRC framework was the most commonly cited framework, the included papers provided varying levels of detail over how the framework was used, and a lack of clarity over whether all three stages of development were explored. The lack of consistency and detail may be a result of the limited practical guidance offered by the MRC framework. There were, however, common approaches used in the papers citing the MRC framework which included: literature reviews, consultation with stakeholders, interviews with patients and clinicians, consensus methodologies and pilot work. It is clear from this review that there is not a consistent approach to developing rehabilitation interventions for older adults and further work is needed to establish how, and which, research methods should be used within the different stages of intervention development.

Other frameworks to support intervention development include the 6SQUID which was based on the experiences of Wight et al 10 in developing public health interventions. Although this framework provides more detail, there is still a lack of methodological detail on how to undertake each element. It also has a public health focus which may not consider all aspects needed in the development of a complex rehabilitation intervention. In providing a rationale for the development of the 6SQUID framework, Wight et al provided a summary and appraisal of existing intervention development frameworks in public health and included both the MRC framework and intervention mapping which were identified in this review. Intervention mapping is an involved and detailed process,9 which may account for it being referred to in only three papers in this review. Möhler et al 8 published CReDECI through a three-stage consensus process. This aimed to improve quality of the reporting on the underlying theory of an intervention, the components and interactions of an intervention as well as any contextual factors. While its merits are acknowledged, the primary focus was on the evaluation phase and the criteria provide little detail on how to undertake the process of intervention development. The capability, opportunity, motivation and behaviour (COM-B) model and theoretical domains framework53 is another intervention development framework that is becoming increasingly popular in the behavioural change literature but has not widely been used in rehabilitation research as yet.

Meaning and implications

Many studies did not use an intervention framework and in those that did, there was a lack of consistent detail regarding the intervention development process. Rigorous intervention development is necessary to avoid costly trials of underdeveloped interventions that have no theoretical basis, however, there is a distinct lack of practical guidance to help researchers determine when an intervention is sufficiently developed. It is acknowledged that each rehabilitation intervention is by its very nature complex and therefore reliant on the experience of the individuals developing it, as well as the context and circumstances it is to be delivered in. A rigid framework that dictates exactly how an intervention should be developed may, therefore, not be appropriate as it would not allow for the nuances of each individual intervention and the different approaches that may be more pertinent to their circumstances. Nevertheless, there does appear a need to provide researchers with further detail on the indicators of good practice and what to consider when undertaking quality intervention development.

Recommendations

Following this review, a number of recommendations can be made, including:

Researchers should carefully plan and clearly detail the process of developing rehabilitation interventions for older people using a recognised framework such as MRC.

Rehabilitation journals need to welcome further detail on the intervention development process using online supplementary material.

A consensus process is needed to depict best practice and provide guidance on developing a rehabilitation intervention for older adults.

Conclusion

The MRC guidance is the most popular framework being used by researchers developing rehabilitation interventions for older adults. However, many studies do not report using a framework to guide their development. Further, specific guidance to help researchers choose and use the best framework for their intervention is needed.