Contextual factors influencing complex intervention research processes in care homes: a systematic review and framework synthesis.

BACKGROUND: Care homes are complex settings to undertake intervention research. Barriers to research implementation processes can threaten studies' validity, reducing the value to residents, staff, researchers and funders. We aimed to (i) identify and categorise contextual factors that may mediate outcomes of complex intervention studies in care homes and (ii) provide recommendations to minimise the risk of expensive research implementation failures.
METHODS: We conducted a systematic review using a framework synthesis approach viewed through a complex adaptive systems lens. We searched: MEDLINE, Embase, CINAHL, ASSIA databases and grey literature. We sought process evaluations of care home complex interventions published in English. Narrative data were indexed under 28 context domains. We performed an inductive thematic analysis across the context domains.
RESULTS: We included 33 process evaluations conducted in high-income countries, published between 2005 and 2019. Framework synthesis identified barriers to implementation that were more common at the task and organisational level. Inductive thematic analysis identified (i) avoiding procedural drift and (ii) participatory action and learning as key priorities for research teams. Research team recommendations include advice for protocol design and care home engagement. Care home team recommendations focus on internal resources and team dynamics. Collaborative recommendations apply to care homes' individual context and the importance of maintaining positive working relationships. DISCUSSION: Researchers planning and undertaking research with care homes need a sensitive appreciation of the complex care home context. Study implementation is most effective where an intervention is co-produced, with agreed purpose and adequate resources to incorporate within existing routines and care practices.

Care homes are complex social ecosystems; implementing controlled research in these heterogeneous settings is challenging.

Many expensive, resource-intensive controlled studies in care homes have resulted in neutral findings.

This review identifies contextual factors that may mediate outcomes of complex intervention research studies in care homes.

Avoiding procedural drift and engaging in participatory action and learning are key to successful implementation.

We provide recommendations that researchers and care home teams can use to support complex intervention research implementation.

Background

Care homes (CHs) play a vital role in public health infrastructure, supporting citizens with complex needs [1]. CH is an umbrella term, describing 24-hour care facilities, including those with and without on-site registered nurses, sometimes referred to as nursing and residential homes, respectively. There are marked differences in organisational size, financial support, cultures of care and population between homes [2].

CHs are complex social ecosystems where individuals live. This can pose significant challenges to research paradigms reliant on data [3-6]. Many large CH complex intervention studies have produced neutral findings [7-14]. The combined research award for these cited studies from the UK amounted to over £8.5million. There is uncertainty about whether the neutral findings were attributable to intervention ineffectiveness, or a consequence of study implementation processes or insensitive measurement tools [15]. For clarity, the term implementation in this article refers to how research intervention processes were carried out during the research study and process evaluation period. It does not refer to implementing research findings thereafter. Previous research has queried, ‘did the trial kill the intervention?’ [16] We wanted to explore this question further in a CH context.

A complex adaptive systems approach to research evaluation in CH settings

Research intervention studies need to consider the influence of context to identify factors which might confound the intervention and to maximise translation into practice [3, 17–22]. CHs can be described as a complex adaptive system of interconnected sub-systems where people, tasks, technologies, the physical environment and organisational culture interact [20, 23–29]. Novel interventions in CHs can disrupt and adapt dynamic system relationships. This can lead to the emergence of potentially undesirable outcomes not anticipated in the study design (or by the research team) [17, 18, 22, 30]. These unpredictable dynamic effects can pose complications for the validity of fixed evaluation measures [21, 24, 31]. A complex adaptive systems approach to CH research evaluation may more reliably capture how an intervention is working in context [6, 22].

This systematic review adopted a human factors (HF) perspective [32]. HF accommodates complex systems theory and identifies influences on human behaviour and how these relate to work performance. The HF perspective is relevant because attempts to modify care practices in CHs involve interactions between people, processes, technologies and organisational systems [25].

Rationale and aim

Process evaluations explore the relationship, similarities and differences between an intervention as planned and implemented [23, 33–37], taking account of contextual factors and their potential influence on study outcomes [20, 22, 23, 38–40]. We aimed to explore the challenges of implementing complex intervention research studies in CHs, identify common and generalizable themes between CH process evaluations, and devise recommendations on how to mitigate against expensive research implementation process failures.

Method

The review protocol was registered online and is reported in accordance with appropriate methodological guidance [41-43].

Search strategy, data sources, inclusion criteria and screening

An experienced medical librarian supported the search. We searched MEDLINE and Embase via OVID from inception to 25 November 2019. Searches of CINAHL via the EBSCO Host platform and ASSIA (Applied Social Sciences Citation Index and Abstracts) via ProQuest followed on 2 December 2019. No date restrictions were applied. We used Medical Subject Headings, keywords and synonyms, entered in grouped stages. (Appendix 1, Table A1.1). We completed a grey literature search (including OpenGrey) on 30 January 2020 (Appendix 1, Table A1.2).

Eligible papers needed to be process evaluations of primary complex intervention research conducted in CHs for older people that used quantitative, qualitative or mixed methods published in the English language. Protocols, secondary data analysis and evidence reviews were excluded. After removing duplicates using Endnote software (Clarivate Analytics, USA) we automated the management of the screening process using Covidence software (Veritas Health Innovation, Australia). Titles and abstracts were screened independently by two reviewers within the team (SK, GP, AC, JBu, GA LI, JBl, MM). In cases of disagreement, or a need for wider discussion, a third reviewer in the team (SK, GP, JBu) was involved. One reviewer in the team (SK, GP, MM, GA) assessed full text articles for eligibility with support from the team in cases of uncertainty.

Data extraction, and critical appraisal

Extracted summary characteristics included: first author and year of publication, country, intervention topic, sample size, methods and participants.

As there are no formal appraisal tools specific to process evaluations, we engaged with experts at the Cochrane Collaboration Qualitative Methods Group and repurposed an appraisal method devised by Shepherd et al. [44, 45]. Two reviewers (JBl, GP, GA, AK, JBu, LI) conducted the appraisals independently for each paper. We focused on the last two questions that appraise reliability and usefulness of findings according to our review question. In the event of a discrepancy in answers to these questions a third reviewer was involved.

We used a ‘best-fit’ framework synthesis approach: a theory-based method to determine salience and connections in qualitative data [45-49]. This method accommodates reports of complex interventions and adds transparency to data coding [38, 49]:

i) Familiarisation,

ii) Identifying and developing a thematic framework,

iii) Indexing (coding extracted data according to the framework),

iv) Charting (presenting evidence summaries), and

v) Interpretation (drawing associations between key themes and concepts identified in the evidence) [46].

Familiarisation and identifying and developing a thematic framework

The first two stages occurred iteratively. To identify a salient conceptual HF framework, we searched Web of Science using the ‘Behaviour of interest; Health context; Exclusions; Models or Theories’ (BeHEMoTh) template (Appendix 1, Table A1.3) [50]. The Systems Engineering Initiative for Patient Safety (SEIPS 2.0 and SEIPS original) models were chosen as the best conceptual fit [48-52].

SEIPS is a well-known HF model akin to Donabedian’s ‘Structure-Process-Outcome’ approach for measuring quality of care [53]. It targets patient safety and discusses the importance of engaging patients and professionals in collaboration to pursue design-driven improvements. We modified the SEIPS 2.0 graphical representation to apply to CHs (Figure ) [51]. It places the person at the centre as the focal work system, with interconnected work systems influencing task performance, safety and well-being [51].

Figure 1

Modified image of the SEIPS 2.0 model [51].

The model exemplifies that CHs are a system of sub-systems. It identifies the interactions that take place between CH staff and their environment, and how these interactions and feedback loops may contribute to desirable or undesirable adaptations to, and outcomes from, work systems and care processes [51, 52]. Outcomes are identified as performance indicators of system behaviour. Poor experiences at a process level that lead to undesirable outcomes suggest a need for system redesign and identifying contributory factors at the work system level [51]. Outcomes from work processes are identified as having an effect over a short or longer term (Figure ).

Indexing and charting

Work system sub-categories included in the initial coding framework were based on criteria presented in the original SEIPS model [52]. The framework was piloted on two relevant process evaluations prior to performing the systematic search and was developed further following detailed familiarisation [46, 54, 55]. We also included an ‘Other’ category that comprised additional relevant contextual factors not captured by the SEIPS model [46]. The final synthesis framework had three levels: (i) work system, (ii) work system sub-category and (iii) work system context domain.

We then performed a qualitative content analysis and indexed the data according to the three levels of the synthesis framework. Data were indexed by two reviewers (JBl, GP) using line-by-line coding within NVivo software (QSR International). Extracted data from the results and discussion sections included quotations from research participants (first-order constructs), and quotations and interpretations of process evaluation authors (second-order constructs) [56]. These data were coded and tabulated as barriers or enablers to successful research implementation processes and outcomes.

Interpretation

Themes were generated inductively from the indexed content domain data to identify salient concepts from a complex adaptive systems perspective [34, 56]. Thematic content was refined iteratively between two reviewers (GP, JBl). Recommendations for action that can contribute to successful CH research implementation processes were mapped against three categories: (i) research team responsibilities, (ii) CH staff responsibilities and (iii) collaborative responsibilities (Figure ).

Stakeholders (including family representatives and CH staff) were involved in: assisting in prioritising the review findings to CH staff and suggesting how to present graphical outputs from the synthesis process.

Results

Characteristics of included process evaluations

We identified 33 process evaluations (32 CH interventions) from 8,097 search results (Appendix 2, Figure A2.1). Summary characteristics of the included papers are presented in Appendix 2, Table A2.1. Included studies were published between 2004 and 2019; most were published after 2016 (n = 20/33, 60%). Studies were conducted in high-income countries only: UK-11 [54, 55, 57–66], Netherlands-6 [39, 67–71], Canada-5 [72-76], Germany-3 [77-79], Norway-2 [80, 81], Australia-1 [82], Belgium-1 [83], Portugal-1 [84], Sweden-1 [85], USA-1 [86] and one multi-national European study [87]. One process evaluation was made available to the review team ahead of publication [65, 66]. Critical appraisal details are presented in Appendix 2, Tables A2.2, A2.3. All eligible reports were rated as having adequate quality to proceed with data synthesis [44].

Indexing

The consolidated three-level framework illustrated in Figure depicts each of the work systems introduced in Figure . The indexing process involved assigning data, coded as barriers or enablers to successful research implementation processes and outcomes, to the relevant work system and context domain on the synthesis framework. For example, in reference to Figure , if a process evaluation reported that the health status of the residents mitigated their participation in the intervention, extracted narrative data were indexed at the Person(s) work system (level 1), the Residents and Family sub-category (level 2) and the Resident Health Status context domain (level 3).

Figure 2

Synthesis framework. Level 1: Work System, Level 2: • Work System Sub-category (in bold), Level 3: o Context Domain (in plain text).

Charting

A heatmap matrix displaying the distribution of indexed context domains within and across the included studies is presented in Figure . Colour coding reflects the frequency of appearance of the context domains on a high to low scale. Context domains were also ranked according to their frequency of appearance across the 33 papers. Appendix 3 (Figure A3.1) presents this context domain frequency hierarchy mapped according to work system. A radar plot and descriptive summary of the indexed data at a work system level is presented in Appendix 3 (Figure A3.2). The radar plot demonstrates that influential contextual factors aligned primarily to Organisation, Task and Person(s) work systems.

Figure 3

Heatmap matrix displaying the frequency of indexed content across 28 ranked context domains identified from n = 33 process evaluations.

A descriptive summary of the most frequent context domains is presented in Table 1. Readers are encouraged to explore the expanded contextual factor dataset in Appendix 4. There are approximately four example quotations for each barrier and enabler to successful trial implementation process in CHs across the 28 context domains.

Table 1

Summary of the context domain data indexed according to the synthesis framework presented in Figure 2 and ranked according to frequency across the n = 33 process evaluations

Framework level1: Work system2: Work system sub-category3: Context domain	Context domain frequency in n = 33 papers (%)	Context domain rank (1–28)	Context domain summary
TASK Research Load Compatibility with Routines	27 (81.8)	1	The compatibility of the research intervention objectives with the working care routines of the CH mediated successful implementation. Interventions that adopted a clear, practical approach that could be integrated into existing routines helped maintain engagement with staff for longer. Some interventions were also perceived to be incongruent with habitual care routines and others were not deemed significantly different from existing practice to deserve a behavioural change.‘. . . “you know we already use. . .the assessments. . .and the products. . .if it was going to be a case that you will be introducing new ways of doing things. . .but that’s not what it was about, so, no, I wouldn’t do that again” ([Internal Facilitator] IF, type B, 12 month, [England]). In this example, the IF only attended one teleconference and then did not participate further in the programme,’ [87] p.8.
TASK Research Load Time and Resources Required	25 (75.8)	2	The amount of time and resource investment required for the project impacted the implementation. Some CHs could not meet the demands of the study due to time and resource constraints, whereas others requested further investment in terms of training. Being able to complete project training within work hours was important so staff acknowledged the intervention as part of a working day as opposed to an additional extra.‘Lack of time emerged as a prominent barrier, particularly time to train and involve colleagues. Furthermore, because few physicians found the time to take part in the two-day seminar, it was difficult to motivate them to participate in the intervention,’ [80] p.7.
PERSON(S) Care Staff Perceived Benefit, Motivation and Attitude to Change	24 (72.7)	3=	Maintaining CH staff engagement was a key driver to successful implementation. Conversely, negative attitudes towards the intervention spread scepticism about its value within the CH. Observing a positive response from residents was a strong motivating factor to continue.‘One manager argued that as many care assistants had been in their role for considerable periods of time, they had become rigid and developed a closed mind, which made them highly resistant to change and less willingly and able to adjust to new situations,’ [84] p.227.
TASK Research Load Collaborative Approach	24 (72.7)	3=	Interventions that stimulated collaboration, rapport-building between team members, and mutual understanding between CH staff and researchers were well received. Clearly defined responsibilities within roles helped CH team dynamics. Insufficient guidance from management and unclear definition of roles reduced individual responsibility and was perceived by staff as poor planning and poor leadership. Enabling opportunities for periodic meetings to discuss resource allocation with care home managers/senior staff were recommended.‘[managers] depicted that the workload in care of older people continuously increases; they were frequently requested by their superiors to execute change in their nursing homes (such as implementing new administrative or clinical routines). Yet, they experienced limited opportunities to discuss the relevance of a new routine, why it was proposed, the rationale for change, the time frame for an implementation, and/or how to proceed,’ [85] p.90.
TASK Staff Workload Existing Pressures	23/33 (69.7)	5=	Existing workload of care staff before a novel intervention was introduced contributed to CH divergence from intended implementation strategies. This contextual factor may pose significant limitations on adopting new practices and sustaining adherence to a protocol that requires specific actions (e.g. timing of data collection or reporting) that are deemed by staff to be of lesser priority.‘. . . the nursing staff in all [Nursing Homes] took high levels of sick leave over the 3 months prior to baseline, indicating that the remaining nursing staff in the [Nursing Homes] were subjected to a high workload and time pressure. These conditions definitively hinder the application and integration of new innovations,’ [78] p.11.
ORGANISATION Resources Staff Turnover	23 (69.7)	5=	Care home staff, including managers, have a high level of turnover. This impacted on the continuity of projects, as new staff were not familiar with the research intervention processes. Changes in CH managers posed significant challenges to study delivery.‘Staff turnover sometimes resulted in situations in which only a part of the team was truly well informed about the care program. Although attempts were made to train new staff members, the situation remained suboptimal. While the turnover of nursing staff had adverse consequences, the change of DSCU [Dementia Specialist Care Units] leader, psychologist or physician was even more detrimental, for they had a leading role in implementing the care program. When these key stakeholders were absent for a period, there was often a drop in attention for implementing the care program,’ [63] p.7.

The six most frequent context domains were indexed in 70–82% of the dataset (Table 1, and Appendix 3, Figure A3.1). Within this group, four context domains originated from the Task work system, one from Organisation and one from the Person(s) work system. Within the Task work system, ‘Research load’ was the most frequent sub-category (framework level two). Novel research activities were often experienced as an extra component to an existing busy work schedule.

The descriptive content analysis summarised in Table 1 and Appendix 4 helps to infer commonality between studies. To derive plausible recommendations on ways to improve quality, efficiency and experience of delivering and participating in CH intervention studies; we performed an inductive thematic analysis on the extracted first-order and second-order narrative data across the 28 framework domains.

Theme 1: procedural drift

Procedural drift refers to how studies lost momentum over time and deviated from the intended intervention protocol [88]. In most cases protocol deviations and intervention adaptations were normalised over time. Studies often started with adequate engagement and enthusiasm but the impetus and commitment to maintain and prioritise research activities, allocate necessary resources, address time constraints and discuss the intervention internally, dissipated. When staff had been instructed to participate without consultation there tended to be a lack of engagement with the research study from the start.

Negative feedback between staff about the study could spread very quickly throughout the organisation contributing to waning interest. This accentuated any existing fragility in team dynamics at an organisational level. If the purpose and objectives of the research process were not continually reaffirmed and supported the novel activities were often avoided, forgotten or practised less frequently. This led to negative consequences for intervention dose, protocol compliance and fidelity overall.

‘Without one or more key persons taking the lead on implementation and on stimulating the care staff to use the forms, it was very difficult to keep everyone focused on using the care program. Also, support of higher management of the organization (for example, by calculating in extra time) facilitated the implementation, because more time and understanding were available during implementation,’ [p.8] [63].

Contributors to procedural drift were identified in all work systems in the synthesis framework, often attributable to a breakdown in positive working relationships. An insightful comment mentioned that as new work routines were introduced, they were, ‘seldom accompanied by suggestions as to what routines should be replaced,’ [p.90] [85].

Theme 2: participatory action and learning

There was a higher chance of sustained engagement and relevant action if CH staff could enact their training with further guidance and mentoring.

‘Our last training was very informative. Having sessions and then several days of practical assistance was essential. Care assistants don’t (just) need more knowledge, they need to practice, they need to implement what they learnt’. [Maria, manager] “It was very important to have the theory coupled with the practice. During individual assistance we were relaxed, we didn’t rush things. We tried to work as we have been taught and this has become routine” [Andrea, care assistant],’ [p.227] [84].

Clarity of roles and responsibilities in implementing the intervention and shared understanding of the purpose of practice change were important for functional team dynamics. Key drivers for success involved attributes of the individuals involved: enthusiasm, commitment, credibility, team cohesion and a sense of pride in delivering the intervention as a collective. These details influenced positive working relationships, collective motivation and enhanced the quality of knowledge sharing across the CH. Data examples are presented in Appendices 4.

The importance of appropriate leadership and a supportive and inclusive management culture played a crucial role in promoting a solution-focused ethos when responding to challenges associated with research implementation processes. Poor communication with CH managers, or managers not being actively involved, was frequently reported as a barrier to the research process. CHs with stronger organisational hierarchies generated more negative comments. CHs that supported a culture of staff development, learning and improvement were more receptive to meaningful participation and sustained engagement in the research process.

‘Care assistants recognised their managers as an essential source of ongoing advice and guidance, but considered them to be too far removed from the reality of life and problems “on the floor”. Care assistants felt that their skills and commitment were rarely acknowledged and that their work was largely undervalued. This was seen to impact negatively on their job performance and morale, which may explain why some of the early benefits of the intervention were not sustained over time,’ [p.228] [84].

Recommendations

Table 2 identifies activities that the research team and CH team can undertake individually as well as activities they can engage with collaboratively. The recommendations describe contingencies to reduce the potential for procedural drift and help sustain positive working relationships at a partnership level to encourage a research culture that engenders participatory action and learning.

Table 2

Anticipatory considerations to promote positive working relationships and reduce the potential for procedural drift interpreted from the indexed dataset

Research team	Collaborative: research team and care home team	Care home team
Protocol Design: How well can the intervention be integrated into existing CH organisational routines?Study initiation: Has enough time been allocated to assessing suitability, building rapport, and assessing readiness to participate prior to commencing the study?Fidelity and Adherence: How much bespoke tailoring of the protocol can be tolerated in the study design?Process evaluation: How will the implementation process be evaluated to explore broader system responses?Activity: How can monitoring implementation activities detect the emergence of ‘procedural drift’?Complexity: How can any perceived complexity of data collection or other implementation activities be simplified?Learning over time: Are adaptations in the delivery of the intervention across sites and over time anticipated?	Individual context Authentic co-production and active collaboration: How can task-oriented dialogue between all stakeholders promote engagement of attitude and action?Awareness of opportunity costs of participation: As new tasks are added to CH work routines, what tasks will be reduced or replaced in the same period?Participatory action and learning: How can training of CH staff be put into practice early and supported through a coaching approach?Commitment: What will help sustain motivation and commitment in working together throughout the study?Collective reflection: How stakeholders feedback their experiences consistently throughout the study duration to align and realign shared expectations?External support: How can the CH access additional external support in delivering the intervention whilst maintaining care quality and resident safety if needed?	Internal resources Resource management: How can necessary resources be released to sustain effective implementation whilst maintaining care quality?Participating residents: How can the intervention reach the residents who are most likely to receive a benefit?Staff development: Is there sufficient staff capacity to attend intervention implementation training during work hours?Resources for key staff: Is there sufficient capacity and commitment for multiple members of staff to help coordinate the study on a day-to-day basis, taking into account sickness and holiday absence?Staff turnover: Is there a robust process for informing new staff about the implementation project and their involvement?
·······························································································································································
Care home engagement Motivation: How can the research team gain an understanding of the CH’s main interest in participating?Study materials: How can study materials be presented in easy-to-use formats to support inclusivity?Managerial support: How can support and training opportunities be extended to the managerial team?Communication enhancement: How can consistent, reliable and inclusive communication channels be established and maintained?	Positive working relationships Relational working: How will the collaboration be nurtured to build rapport, air tensions, identify solutions, and resolve difficulties?Inclusivity: How can all staff throughout the care home and residents (if appropriate) be informed about the study?Goal setting: How can residents’ goals associated with the intervention be shared with staff?Language: How can accessible language and ‘easy read’ formats support effective communication and team dynamics?	Team dynamics Managerial involvement: How can regular meetings with the CH manager take place to align shared expectations and discuss ongoing commitment?Visibility: How can CH staff responsible for supporting the study be identifiable and accessible across the organisation throughout the implementation process?Reflection: How can time be allotted in team meetings to discuss how the study is running in the CH routinely?Sharing good news: How can staff share positive stories about their involvement with the intervention?

Discussion

Summary

We identified recommendations from international evidence for researchers involved in designing, implementing and evaluating CH complex intervention studies and for staff considering participating in such studies. Recommendations are divided into activities which researchers and CH teams must consider individually and those which require collaborative and collective efforts to succeed.

Research teams must not underestimate the effects of restructuring habitual ways of working [89]. CH staff responses to these changes had a mediating effect on successful trial implementation processes. The compatibility of the tested intervention with the CH’s existing work routine was the most prevalent contextual factor discussed within the reviewed process evaluations. Many of the unanticipated behavioural responses of the CH staff posed significant challenges to the intervention implementation process.

Both CH staff and members of the research team made adaptations to and deviations from trial protocols. Further, the complexities of task and person focused factors were compounded by organisational and environment factors such as high staff turnover, inherent variability of CH administration, resource limitations and varying suitability of CH internal environments. The culmination of these effects increased heterogeneity between CH settings [22, 33, 90].

Key findings

We identified two themes: (i) procedural drift and (ii) participatory action and learning.

Procedural drift

Procedural drift (also referred to as practical drift) has its roots in safety science [88, 91, 92]. It refers to the human tendency to change, or deviate from, or avoid a recommended or required sequence of repeated activities over time. Some deviations from a prescribed protocol may enhance long-term sustainability of an intervention and act as a desirable outcome [88]. Other forms of deviation and adaptation may signify a vulnerability to implementation failure. A significant problem identified in the current data is how task-focused activities and staff engagement with them tended to dwindle over time, often coinciding with when the research team decreased active involvement following the start-up phase [93]. If activities diminished, attention to detail, intervention adherence and commitment to the entire implementation process weakened also. Inconsistencies in data collection caused research teams to question their reliability.

Time constraints

CH resources, staff capacity and workload are strained. If staff could not identify meaning behind implementation activities, sustained engagement was unlikely. When time constraints were experienced by staff, habitual ways of working took priority over novel research activities. Researchers must acknowledge the opportunity costs that arise for CH staff participating in research studies evaluating novel interventions.

Mitigating action

When we consider the concept of procedural drift further, we also need to identify ways to detect it and potentially take corrective action. Two important questions arise: drift with respect to what and to whom [88]. From an implementation perspective it is useful to consider the contextual attributes that may foster resilience to any long-term effects or possible catastrophic failures. Further description of different forms of resilience, a system’s capacity to rebound from failure, and system robustness to absorb the effects of failure are discussed elsewhere [88]. Future studies may find benefit in assigning personnel and resources to detect indications of procedural drift as early as possible.

Participatory action and learning

Prominent contextual factors identified across the process evaluations were indexed within the task and organisational work systems. The interdependent nature of positive working relationships means both the research team and the CH team need to be prepared to alter their behaviour [94]. It is a reciprocal partnership. Despite a high number of neutral findings in the main intervention studies to which the process evaluations belonged, there were examples of promising participatory activities undertaken in partnership with stakeholders (Appendix 4) [14].

Understanding the properties of the intervention, agreeing its purpose, feeling equipped to enact training and drawing on prior experience of caring for residents were crucial in promoting cognitive participation among CH staff [14, 89]. Without a corpus of support within the CH it was difficult to maintain constructive research-focused discussion and engage in effective problem resolution.

Authentic engagement and co-production

Authentic engagement and co-production require awareness of inherent power imbalances, time, and sustained effort to be effective [95]. It helps leverage skills and experiences from all parties involved as opposed to instructing a way of working that may not be compatible with a CH’s work routines [96]. This is challenging to achieve [97]. Often it is dependent on personalities, individuals staying in post and other contextual factors specific to the suitability of each individual research project [95, 97, 98]. Intentional stakeholder engagement and co-production methods can also be susceptible to procedural drift [99].

A collective commitment to negotiate, develop mutual understanding, and sustain positive working relationships requires resource allocation from the research team and a receptive infrastructure in the CH to support and maintain co-production and meaningful collaboration [14, 100, 101]. Activities oriented towards developing trust, a sense of collegiality and shared commitment to derive mutual benefit are important precursors [102]. Returning to these principles throughout the process is a suggested means to maintain a shared sense of purpose.

Research study constructs

Staff and health care clinicians involved in implementing the intervention were not passive delivery conduits. There were distinct effects of ‘learning over time’. This has implications for experimental design. For example, in a cluster randomised trial a CH that receives the intervention early may not be directly comparable to a CH that receives the intervention at a later stage: there is learning and adaptation in the interim period.

Strengths and weaknesses

There were strengths in using the SEIPS model as a synthesis framework; it produced broad agreement across the dataset [48, 51, 52]. The 28 context domains appeared in 18–82% of the included papers; the top six domains were present in >70% of papers. The theory-based framework synthesis method allowed descriptive content to be indexed in a structured format but also combined inductive thematic analysis to support enhanced interpretation [46, 48, 49, 103].

The searches were updated on 30 September 2021 and a further 16 eligible process evaluations were identified [104-119]. The context domains indexed in the most recent publications mapped well onto the findings of this review which supports the viability of the synthesis framework. Procedural drift, and the need for a participatory action and learning were pervasive: lack of time, resource constraints and heterogeneity between CH settings were described as barriers to successful implementation of complex intervention trials [104, 105, 109–119]. Several papers suggested it may be more appropriate to design research interventions with high staff turnover and a changing context in mind [106, 110]. The importance of purposefully including flexibility within the trial design to tailor the research process to a CH’s individual context settings was also discussed as an important enabler to successful implementation processes [107, 108, 112].

There are also limitations from the review to acknowledge. First, all synthesis methods are reductive, and this approach comes with risks. Using this framework meant that the data were considered using a HF work system frame of reference. Alternative approaches may have led to alternative areas of focus, such as Normalisation Process Theory [89], or the Consolidated Framework for Implementation Research [120]. However, both examples are synergistic with our review; they also involve the study of interactions between people, processes, technologies and organisational systems.

The search was limited to the English language, and only identified studies from Western high-income countries. Even within these countries the model of care within CHs is heterogenous and this will have an impact on the care context and success of complex intervention research processes [121]. Publications from low and middle-income countries more frequently appear in other languages [122].

Moreover, for pragmatic reasons this review identified studies that used the term ‘process evaluation’ specifically. Broadening the eligibility criteria to include additional studies discussing contextual factors influencing research implementation processes would have increased heterogeneity between study methods.

The SEIPS model does not aim to attribute causality, only plausible contributions to desirable and undesirable outcomes that may occur over short or longer time frames [51, 52]. The work systems to which the data were indexed were coded as discrete elements; however, all intervention activity in this model was inter-related.

Conclusion

This review provides compelling evidence to undertake and report formal process evaluation data alongside intervention effectiveness data in CH complex intervention trials. Exploring contextual data from trial implementation processes and broader system responses to an intervention will maximise the explanatory value of the analysis and provide assurances over a CH trial’s internal validity [16, 22, 33, 90].

Our recommendations outline what is needed to trial CH interventions more consistently, and reduce the risk of expensive research implementation failures [123]. The recommendations aim to inform and improve future CH interventional studies more broadly. They are dependent upon operationalising authentic relational approaches within the context of complex adaptive human systems at scale [6, 90, 94]. It is likely that we will learn more about how to do this better as these recommendations are implemented.

Click here for additional data file.

Click here for additional data file.