Understanding implementation success: protocol for an in-depth, mixed-methods process evaluation of a cluster randomised controlled trial testing methods to improve detection of Lynch syndrome in Australian hospitals.

INTRODUCTION: In multisite intervention trials, implementation success often varies widely across settings. Process evaluations are crucial to interpreting trial outcomes and understanding contextual factors and causal chains necessary for successful implementation. Lynch syndrome is a hereditary cancer predisposition conferring an increased risk of colorectal, endometrial and other cancer types. Despite systematic screening protocols to identify Lynch syndrome, the condition remains largely underdiagnosed. The Hide and Seek Project ('HaSP') is a cluster randomised controlled trial determining the effectiveness of two approaches to improving Lynch syndrome detection at eight Australian hospital networks. To enhance widespread implementation of optimal Lynch syndrome identification, there is a need to understand not only what works, but also why, in what contexts, and at what costs. Here we describe an in-depth investigation of factors influencing successful implementation of procedures evaluated in the HaSP trial. METHODS AND ANALYSIS: A mixed-methods, theory-driven process evaluation will be undertaken in parallel to the HaSP trial. Data will include: interviews of Implementation Leads and Lynch syndrome stakeholders, pre-post implementation questionnaires, audio analysis of meetings and focus groups, observation of multidisciplinary team meetings, fidelity checklists and project log analysis. Results will be triangulated and coded, drawing on the Theoretical Domains Framework, Consolidated Framework for Implementation Research and Proctor's implementation outcomes. ETHICS AND DISSEMINATION: Use of a theory-based process evaluation will enhance interpretation and generalisability of HaSP trial findings, and contribute to the implementation research field by furthering understanding of the conditions necessary for implementation success. Ethical approval has been granted and results will be disseminated via publications in peer-reviewed journals and conference presentations. At trial completion, key findings will be fed back to sites to enable refinement of intervention strategies, both in the context of Lynch syndrome and for the possible generalisability of intervention components in other genetic and broader clinical specialties. HASP TRIAL REGISTRATION NUMBER: Australian New Zealand Clinical Trials Registry (Identifier: ACTRN12618001072202). Registered 27 June 2018. http://www.ANZCTR.org.au/ACTRN12618001072202.aspx. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

RCT Entities:   Population Interventions Outcomes

This in-depth process evaluation capitalises on the unique design of the Hide and Seek Project trial to advance understanding of the role and importance of theory in intervention design, mechanisms of impact and factors affecting implementation success.

The mixed-methods design allows triangulation of results to enhance confidence in study findings.

Acknowledging intervention complexity, this process evaluation can only be used to generate inferences about causation.

This process evaluation will enhance understanding about the generalisability of successful intervention components for implementation.

Findings can be applied to other clinical problems within and beyond the setting of genetics and Lynch syndrome.

Introduction

The real-world implementation of clinical practice change interventions within complex health systems is an ongoing challenge.1–3 Despite a growing body of research dedicated to the development of theories and frameworks to optimise the design and implementation of such interventions,4 5 success remains widely variable.6–9 While randomised controlled trials (RCTs) are important in establishing effectiveness,10 11 their tendency to focus solely on clinical outcomes (eg, morbidity, mortality, symptomatology) and/or service system outcomes (eg, efficiency, equity, patient-centredness), often results in an inability to explain why an intervention was (or was not) successful, hence limiting generalisability.12–14 Undertaking an in-depth process evaluation alongside an intervention trial can provide a more detailed picture of contextual influences and complex causal processes, thereby helping policy-makers, clinicians and researchers interpret trial findings and understand how they might be applied elsewhere.14

The Hide and Seek Project (HaSP): Improving detection of patients with colorectal cancer (CRC) with a high risk of Lynch syndrome (LS)15 is a cluster RCT that will assess the effectiveness of two different implementation approaches aimed at improving detection of LS among patients with CRC, and is the focus of this in-depth process evaluation. LS is a hereditary cancer predisposition conferring an increased lifetime risk of colorectal, endometrial, ovarian and other cancer types.16 For LS carriers, risk management strategies (such as colonoscopic surveillance, risk-reducing surgery) enable cancer prevention, early detection and improved survival.17–19 These health benefits extend to the carriers’ relatives, who can undergo predictive genetic testing to clarify their cancer risks.

Failure to diagnose LS can result in otherwise preventable cancer diagnoses and deaths, as carriers (and their at-risk relatives) remain unaware of their increased cancer risks and the need for heightened surveillance. While early methods of LS detection were largely reliant on reliant on family history assessment,20 21 tumour-based screening of LS-associated cancers by mismatch repair (MMR) protein immunohistochemistry (IHC) and/or microsatellite instability (MSI) analysis has offered a more sensitive means of identifying patients at high risk of LS.22 Those with abnormal tumour-screening results should be referred to a familial cancer clinic (FCC) for further investigations to establish an LS diagnosis (‘LS referral’). To increase detection of LS, evidence-based guidelines recommend universal tumour screening among all newly diagnosed patients with CRC.23 24 However, the condition remains largely underdiagnosed and current genetic referral practices are suboptimal. For example, an Australian study demonstrated that only 46% of patients with CRC at high risk of LS (based on tumour-screening results) were referred to an FCC for potential diagnostic testing.25 Furthermore, Australia lacks a national policy for LS CRC tumour screening, and an estimated 53% of pathology laboratories are yet to adopt a universal approach.26 Suboptimal LS tumour screening and referral rates have also been demonstrated internationally.27–29 While the problem of LS underdiagnosis is likely to be multifactorial (eg, failure to assess familial cancer risk in the primary care setting,30 suboptimal family communication and dissemination of LS risk information),31 these studies highlight a crucial need for hospital-based interventions targeting the LS tumour screening and referral pathway.

Despite clear preventative health benefits, adopting a systematic LS screening approach is challenging and requires organisational and behavioural change among multidisciplinary groups of health professionals involved in the recommended LS referral and diagnosis pathway.2 32 Interventions aimed at improving uptake of evidence-based healthcare practices often have multiple components (acting both independently and interdependently), and target change in multiple behaviour patterns at individual, team and organisational levels.33 34 Owing to these complexities, numerous theoretical frameworks have been developed to guide the design and implementation of such interventions.4 However, the majority of interventions aimed at improving LS detection to date have focused on the uptake of various LS tumour screening strategies,25 35 with limited or no description of the implementation strategies or theoretical framework used (if any). While a US-based study is underway using a theory-driven implementation framework to enhance uptake of universal LS screening, such methods are yet to be applied in the Australian context.36

Furthermore, while theory-based approaches are recommended,4 37 38 there is little direct evidence as to whether they offer benefit over usual (non-theory informed) change processes. The Theoretical Domains Framework (TDF)39 synthesises a range of behavioural change theories to aid the identification of behaviour change determinants (eg, barriers or facilitators), which can then be specifically targeted by evidence-based intervention strategies which employ evidence-based behaviour change techniques (BCTs).40 41 The HaSP trial will compare the effectiveness of two identical implementation approaches, distinguished only by the use of theory—specifically the TDF4 and BCTs40—to identify barriers and design targeted interventions to improve diagnosis of LS across eight Australian hospital networks.

While it will be of interest and use to discover which works best—the theory or non-theory guided approach, simply testing this comparison will not be enough to facilitate our understanding about the factors affecting the success of either. Therefore, aligning with recommendations in the implementation science literature,6 42–44 there is a need to conduct a comprehensive, mixed-methods process evaluation alongside the HaSP trial to investigate how and why change has (or has not) occurred. Process evaluations are exploratory studies that seek to complement (but are distinct from) outcome evaluations by understanding the functioning of an intervention in practice.13 14 45 Recognising the value of process evaluations in informing policy and practice, the UK Medical Research Council (MRC) produced guidance on theory and practice underpinning evaluations.14 This guidance highlights three aspects of implementation complexity that may impact intervention effectiveness: implementation, context and mechanisms of impact.

‘Implementation’, refers to ‘the structures, resources and processes through which delivery is achieved, and the quantity and quality of what is delivered’ (14:p.10). When an intervention is unsuccessful, failure to assess these outcomes can result in an inability to distinguish between interventions that are inherently faulty in concept or design, and those that are poorly implemented.11 ‘Context’, refers to the ways in which ‘external factors influence the delivery and functioning of interventions’ (14:p.10). ‘Mechanisms of impact’ refers to ‘how intervention activities and participants’ (ie, in this case, individuals involved in an intervention who have completed process evaluation measures) interactions with them, trigger change’ (14:p.10). Intervention recipients’ interactions with changes are shaped by context, and will occur differently in the presence of different barriers and facilitators, across intervention sites.43 Capturing the way in which individuals interact with an intervention, and the influence of contexts, facilitates detailed identification of mechanisms of impact underlying behavioural change.14 Understanding the effects of these factors across different settings is thus key to interpreting trial outcomes, determining transferability and informing the design and implementation of subsequent interventions.33

Use of a theory-grounded approach for process evaluation can further assess whether the intervention alters theoretical constructs proposed to mediate change, enabling testing of hypothesised causal pathways. A key gap in the process evaluation literature is the failure to incorporate theory, signifying missed opportunities to advance understanding about mechanisms of impact.46 The HaSP trial presents a novel opportunity to address this gap by examining the impact of behaviour change theory (specifically the TDF) in intervention design and implementation, when compared with an approach that instead draws primarily on clinician intuition and tacit knowledge.

There is a further need to determine the resources associated with the implementation of health system interventions, and to evaluate their cost-effectiveness.12 Without this, policy-makers lack the evidence needed to support funding and resource allocation for such efforts. Only a small proportion of implementation studies to date have incorporated cost data in their reporting, with the majority focusing only on intervention costs.12 47 48 In addition to intervention costs, the cost of improving the implementation of interventions also depends on the implementation strategy used, and the service delivery setting.12 As such, the current process evaluation will incorporate measures to assess intervention and implementation costs, informing a separate cost-effectiveness study.

The current study will conduct a real-time process evaluation to gain an in-depth understanding of the factors influencing the effectiveness of the implementation approaches being tested in the HaSP trial. Specifically, we seek to achieve the following objectives:

Evaluate the implementation of the HaSP intervention trial, via assessment of Proctor’s implementation outcomes.12

Examine contextual factors influencing the effectiveness of the HaSP trial.

Identify potential mechanisms of impact to understand behavioural change and explain HaSP trial outcomes.

Document HaSP trial intervention and implementation costs, informing a separate cost-effectiveness study.

Methods and analysis

Summary of the HaSP trial

The HaSP trial commenced in June 2018 and is ongoing. A detailed protocol for the HaSP trial is available elsewhere.15 Briefly, a cluster RCT will be used to test a theoretically driven structured implementation approach against a structured implementation approach (without the explicit use of theory) for improving LS referral practices for CRC patients in eight large Australian hospital networks. At each hospital network, a locally employed healthcare professional (eg, nurse, genetic counsellor) will be appointed and trained as an ‘Implementation Lead’ to coordinate the implementation approach, with ongoing support from researchers with expertise in implementation science and behavioural change.

At each hospital network, Implementation Leads will oversee seven HaSP study phases over an 18-month period. Figure 1 shows a breakdown of the key tasks in each phase, and how they relate to process evaluation activities. A key distinction of the HaSP trial is the tailored approach used to develop targeted intervention strategies to address site-specific referral barriers. To achieve this, the LS referral pathway will be mapped in detail (‘process-mapping’) alongside audit data at each site to identify potential gaps or bottlenecks, highlighting ‘target behaviours’ for change. These will vary between sites, and may affect different stages and departments involved in the LS referral pathway (eg, improvements in pathology systems and reporting, streamlined risk assessment processes for surgeons and/or oncologists).

Figure 1

Flow chart of the HaSP trial and process evaluation. BCTs, behaviour change techniques; HaSP, Hide and Seek Project; IPSBQ, Influences on Patient Safety Behaviours Questionnaire; LS, Lynch syndrome; MDT, multidisciplinary team; TDF, Theoretical Domains Framework.

The two trial arms differ only in the methods used to determine the key barriers to performing the target behaviour(s) and develop intervention strategies (phase 4–5). In summary, the theory-driven implementation approach uses a questionnaire (Influences on Patient Safety Behaviours Questionnaire, IPSBQ) to identify site-specific barriers mapped to the domains of the TDF, allowing selection of corresponding BCTs in the design of targeted intervention strategies. In contrast, the ‘non-theory’ approach relies on the intuition and tacit knowledge of hospital staff in the identification of barriers and design of intervention strategies (ie, without reference to the TDF or BCTs). Site-specific interventions are then implemented over a 6-month period. Behavioural change will be assessed in the HaSP tria via extraction of clinical data preimplementation and postimplementation (the primary intervention outcome being the proportion of patients with risk-appropriate completion of the LS tumour testing and referral pathway).

Process evaluation design

Design framework and rationale

This process evaluation is an exploratory study that will be analysed and interpreted alongside the HaSP trial outcomes to explain how behaviour change occurs. The MRC guidance on process evaluations emphasises the importance of using both qualitative and quantitative data—analysed both separately and in combination—to enhance interpretation of intervention trial outcomes.14 A mixed-methods study design was therefore developed to assess each of the process evaluation objectives, and will be undertaken in parallel to the HaSP trial for both intervention (theory-driven implementation approach) and active control (‘non-theory’ implementation approach) arms at each of the eight Australian hospital networks.

Qualitative and quantitative data will be obtained from various sources throughout each phase of the HaSP trial to capture changes in implementation over time (see figure 1). Quantitative methods will be used to assess key HaSP process variables.14 Qualitative data will be used to capture in-depth stakeholder experiences of the intervention, generate hypotheses about causal pathways and to explain quantitative findings.14 Table 1 provides a summary of measures and time points in relation to the four key process evaluation objectives, while table 2 provides additional detail about implementation outcome measures.

Table 1

Summary of measures

Measures	Objectives*				Data collection time points†				Analysis plan			Purpose of measurement
	1	2	3	4	T0	T1	T2	T3	Data type	Analysis approach	Coding framework(s)‡
Video analysis of IL training	X				X				Qualitative	Thematic analysis	Proctor’s implementation outcomes	Process evaluation
IL post-training interviews	X	X			X				Qualitative	Thematic analysis	TDF, CFIR	Process evaluation
MDT observation	X	X				X	X		Qualitative and quantitative	Descriptive statistics and thematic analysis	TDF, CFIR, Proctor’s implementation outcomes	Process evaluation
Observation of HaSP meetings and focus groups	X	X	X			X			Qualitative	Thematic analysis	TDF, CFIR, Proctor’s implementation outcomes	HaSP trial and process evaluation
IPSBQ (barrier questionnaire)		X	X			X		X	Quantitative	Descriptive statistics and MANOVA	N/A	HaSP trial (theory-arm only) and process evaluation
LS stakeholder interviews	X	X	X			X		X	Qualitative	Thematic analysis	TDF, CFIR, Proctor’s implementation outcomes§	Process evaluation
Fidelity checklists	X						X	X	Qualitative and quantitative	Descriptive statistics and thematic analysis	Proctor’s implementation outcomes	Process evaluation
IL project logs	X			X		X	X	X	Qualitative and quantitative	Descriptive statistics	Proctor’s implementation outcomes	Process evaluation

*Objective 1—Evaluate Proctor’s implementation outcome (see table 2); Objective 2—Examine contextual factors; Objective 3—Identify mechanisms of impact; Objective 4—Collect cost data to inform an economic evaluation.

†Timepoints throughout the HaSP trial: T0=preimplementation/baseline (Phase 1 of HaSP trial); T1=early implementation (6 months, phases 2–5 of HaSP trial); T2=mid-implementation (12 months, phase 6 of HaSP trial); T3=end of implementation period (18 months, phase 7 of HaSP trial).

‡Used for qualitative analyses.

§Coding according to Proctor’s implementation outcomes applies to T3 (postimplementation) interviews only.

CFIR, Consolidated Framework for Implementation Research; HaSP, Hide and Seek Project; IL, implementation lead; IPSBQ, Influences on Patient Safety Behaviours Questionnaire; LS, Lynch syndrome; MANOVA, multivariate analysis of variance; MDT, multidisciplinary team; NA, not applicable; TDF, Theoretical Domains Framework.

Table 2

Assessing Proctor’s implementation outcomes in the HaSP trial

Implementation outcome*	Process evaluation question applied to the HaSP trial	Measures†	Implementation stage assessed
Acceptability	Do LS stakeholders perceive the HaSP implementation approaches and intervention strategies to be agreeable, palatable or satisfactory?	Semistructured interviews61 of LS stakeholders	Intervention development and implementation
		Post-training semistructured interviews12 of Implementation Leads	Implementation planning
Adoption	Are LS stakeholders willing to adopt the HaSP implementation approaches and intervention strategies?	Semistructured interviews62 of LS stakeholders	Intervention development
		Observation of HaSP meetings and focus groups12	Intervention development
		Implementation Lead project logs (administrative data)12	Intervention development and implementation
Appropriateness	Do LS stakeholders believe the HaSP implementation approaches to be relevant/ compatible for their hospital setting, their provider role and their patients?	Post-training semistructured interviews12 of Implementation Leads	Implementation planning
		Semistructured interviews12 of LS stakeholders	Intervention development and implementation
		Observation of HaSP meetings and focus groups12	Intervention development
Feasibility	Can the HaSP implementation approaches and intervention strategies be successfully carried out within each of the hospital settings?	Post-training semistructured interviews63 of Implementation Leads	Implementation planning
		Semistructured interviews63 of LS stakeholders	Intervention development and implementation
		Implementation Lead project logs64	Intervention development and implementation
Fidelity	To what extent were the HaSP implementation approaches and intervention strategies carried out as intended (according to trial arm) at each hospital network?	Video analysis of Implementation Lead training	Implementation planning
		Implementation Lead project logs	Intervention development and implementation
		Observation12 of HaSP meetings and focus groups	Intervention development and implementation
		Fidelity checklist56 65	Intervention implementation
		Observation of MDT meetings12	Intervention implementation
Implementation Cost	What was the absolute cost of carrying out the HaSP implementation approach (including implementation of intervention strategies) at each hospital network? What was the additional cost of using a theory-grounded implementation approach?	Implementation Lead project logs64	Implementation planning, intervention development and implementation
Penetration	To what degree was the HaSP implementation approach and intervention strategies integrated within each hospital networks?	Implementation Lead project logs64	Implementation planning, intervention development and implementation
		Fidelity checklist12	Intervention implementation
Sustainability	To what extent were the intervention strategies developed via the HaSP implementation approach maintained within each hospital network?	Implementation Lead project logs	Intervention implementation
		Fidelity checklist12	Intervention implementation

*See Proctor et al for implementation outcome definitions12

†All interviews and meeting/focus group observations will be conducted by a member of the research team, fidelity checklists will be completed by a member of the Implementation Team and project logs will be completed by the Implementation Leads.

HaSP, Hide and Seek Project; LS, Lynch syndrome; MDT, multidisciplinary team.

Process evaluation data will be used for summative purposes only, so as not to influence the HaSP trial outcomes.13 At trial completion, however, key process evaluation findings will be fed back to sites to enable potential intervention refinement. Given the minimal risk of harm from the HaSP trial process evaluation, a formal data monitoring committee will not be required. The process evaluation protocol has been reported according to the Standard Protocol Items: Recommendations for Interventional Trials checklist (see online supplementary additional file 1).49

Implementation outcomes

Implementation outcomes include: acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration and sustainability. To conceptualise and evaluate implementation, Proctor et al12 proposed a framework of eight ‘implementation outcomes’, which are conceptually distinct from service system and clinical outcomes, and guidance on methods by which they can be measured. The Proctor framework12 will be used to capture implementation outcomes, which have been defined and operationalised in the context of the HaSP trial (see table 2). The HaSP trial can be divided into two key stages of implementation. The first stage involves the use of a step-by-step approach to identify barriers and develop targeted, site-specific intervention strategies (‘intervention development stage’, HaSP phases 1–5). In the second stage, intervention strategies are implemented and evaluated over a 6-month period (‘intervention implementation stage’; HaSP phases 6–7). Proctor’s implementation outcomes will be assessed throughout both of these stages (see table 2).

Theoretical orientation

Use of a theoretical framework for a process evaluation offers the ability to test theoretical constructs and behavioural determinants proposed to mediate change.6 Given that the TDF4 is the underlying theoretical framework for the HaSP trial, it was also used to guide the design and analysis plan for the process evaluation. Explicit use of barrier-mapped change techniques40 41 for the development of intervention strategies in the theory-based HaSP trial arm will allow testing of prehypothesised causal pathways. In the non-theory trial arm, intuitively derived intervention strategies will be retrospectively coded to corresponding categories of BCTs.40 41 For each site, a detailed logic model will be developed to articulate assumptions about how the intervention will produce intended behaviour change effects.14 These will provide a visual representation of the interventions’ underlying theoretical constructs, contents, proposed causal pathways of behavioural change, and the conditions believed necessary for change to occur.13 50 These logic models will be developed in consultation with intervention developers (HaSP research team and Implementation Leads) and will be used to later explore and test potential mechanisms of impact.

While the TDF focuses on individual-level behavioural change, the Consolidated Framework for Implementation Research (CFIR)51 includes constructs designed to better capture organisational-level determinants of change.52 The CFIR is a taxonomy of 31 constructs (across five major domains) from multiple disciplines (eg, psychology, sociology, organisational change) that are believed to influence the implementation of complex interventions.51 53 The CFIR has been widely used in implementation studies, and has sample interview guides available online (www.cfirguide.org) from which relevant constructs can be selected by researchers.51 Together with the TDF, the CFIR will be used in the qualitative analysis to develop a broader understanding of the underlying implementation context.

Recruitment

Prior to commencing the process evaluation, eight hospital networks and Implementation Leads have been recruited via the HaSP trial. Given that Implementation Leads have been employed from within each hospital network, they will have existing contacts and the ability to identify key staff from various departments involved in the LS identification and referral pathway (eg, colorectal surgeons, oncologists, pathologists, genetic counsellors, administrators; referred to hereafter as ‘LS stakeholders’). Using snowball methods, LS stakeholders will be invited by the Implementation Lead to participate in process evaluation activities at various time points (figure 1).

Data sources

Video analysis of IL training

Training of the Implementation Leads will be video recorded for analysis. The purpose of the recording will be to assess the fidelity of the training delivery, and ensure there is no contamination in the training components delivered between the two implementation approaches.

Implementation lead post-training interviews

Following completion of a 1-day training package, each Implementation Lead will be invited to participate in a semistructured interview (online supplementary appendix 1) to understand their experiences of the training and anticipated challenges at their site. Interviews will be conducted individually by telephone by an independent member of the research team (ie, not involved in training delivery) and will be audio recorded and transcribed for thematic analysis. These will be used to assess the implementation outcomes of acceptability, appropriateness and feasibility, as per the Proctor framework.12

Influences on Patient Safety Behaviours Questionnaire

The IPSBQ is a questionnaire designed to measure psychosocial and environmental barriers (eg, knowledge, environment/resources, memory and emotion) to performing clinical practice target behaviour(s), and is theoretically underpinned by the TDF domains.4 54 The IPSBQ has been used in the context of nasogastric tube misplacement54 adapted for use in other contexts,55 and has demonstrated internal consistency, construct validity and discriminant validity.54 The IPSBQ has 34-items (each mapped to a TDF domain), takes less than 5 min to complete, and has a modifiable end statement allowing application across a range of settings and target behaviours.54 This is particularly useful in the context of the HaSP trial, given that the LS referral pathway involves multiple behaviours and decision points (eg, performing MMR IHC, initiating discussion about referral with at-risk patients, writing a referral letter) among individuals from various hospital departments (eg, pathology, surgery, oncology, genetics). Furthermore, the tool has been used successfully to identify barriers in the context of LS in the HaSP pilot study.32 The IPSBQ will be distributed by Implementation Leads to LS stakeholders at each hospital network at time points before and after the intervention implementation to assess for changes in perceived LS referral barriers. Implementation Leads will be encouraged to seek a large sample (approximately 20–40, depending on hospital network size) with representation from each of the key departments involved in the LS referral pathway to ensure correct identification of key barriers.

Interviews with health professionals involved in the LS referral pathway

Implementation Leads will invite LS stakeholders to participate individually in semi-structured telephone interviews preimplementation and postimplementation, aiming for 8–10 participants per site. These will explore current LS referral practices, perceptions about LS referral barriers and experiences of the implementation process (postimplementation). Interview questions (see online supplementary appendix 2) were developed to reflect TDF domains and relevant CFIR constructs (eg, culture, tension for change, readiness for implementation). TDF-based questions were adapted from, and expanded on, the interview guide developed by Gould et al,42 while CFIR-based questions were selected from the sample interview guide developed by Damschroder et al (available online: www.cfirguide.org). Interviews will be audio-recorded and transcribed for thematic analyses. They will also be used to assess the implementation outcomes of acceptability, adoption, appropriateness and feasibility.

Audio analysis of meetings/focus groups

Implementation leads will be responsible for audio-recording meetings and focus groups held throughout the HaSP trial, which will be structured according to the schedules prescribed in the trial protocol (see figure 1 for summary).15 These audio recordings will serve dual purposes for the HaSP trial and process evaluation, and will therefore be analysed separately according to these purposes. For the purpose of the HaSP trial, meetings and focus groups will be analysed by the Cancer Council New South Wales research team to guide the appropriate selection of target behaviours for change, barriers to behavioural change and targeted intervention strategies (see HaSP trial protocol for further detail).15 For the purpose of this process evaluation, the audio recordings will be used to gain a more in-depth understanding of barriers, the impact of the change techniques designed into the interventions, and to generate hypotheses about potential mechanisms of impact, as guided by the analysis approach described under ‘Data Analysis’. Audio recordings of meetings and focus groups will also be used to assess the implementation outcomes of adoption, appropriateness and fidelity. Implementation Leads will each aim to recruit 8–10 LS stakeholders from various departments to participate in each meeting and focus group.

Multidisciplinary team meeting observation

Where relevant and possible, a researcher will attend and observe CRC multidisciplinary team (MDT) meetings. The researcher will complete semistructured observation checklists (online supplementary appendix 3) for each CRC patient presented. Checklist items were adapted from the observation materials developed for the ‘Audit and Feedback INterventions to Increase evidence-based Transfusion practIcE’ (AFFINITIE) Programme,42 a UK-based intervention study aimed at improving evidence-based blood transfusion practices. The checklist will assess whether the following information was reported for each case presentation: personal history of polyps and/or other cancers, cancer family history, MMR and/or MSI status, LS risk and indication for FCC referral. Field notes will also be collected, documenting observed MDT processes (eg, use of proforma), group dynamics, decision-making behaviours, staff roles and interactions. These will be used to assess changes in LS referral processes preintervention and postintervention, identify cultural changes, investigate mechanisms of impact and assess the implementation outcome fidelity (for sites where MDTs are the focus of intervention strategies).

Fidelity checklists

Based on intervention packages developed at each site in phase 5 of the HaSP trial, intervention-specific ‘fidelity checklists’ will be developed in consultation with Implementation Leads to assess whether key intervention components have been delivered as planned. Fidelity checklists will differ between sites, as intervention strategies will be tailored to address context-specific barriers. Checklist development will be guided by the steps outlined by Stein et al,56 while Proctor’s ‘Prerequisites to measuring implementation strategies’57 will be used to further refine scale items to ensure measurability. Checklists will be completed by two members of the Implementation Team (enabling assessment of inter-rater reliability) every 3 weeks during the 6-month intervention implementation period (phase 6 of HaSP trial). Dichotomous scaling will be used to increase inter-rater reliability,56 58 however free-text space will enable documentation of any variations made to the intended intervention strategies. Fidelity checklists will be used to assess the implementation outcome of fidelity, penetration and sustainability.

Implementation lead project log

For each HaSP phase, Implementation Leads will complete a ‘Project Log’, using a template provided by the research team (online supplementary appendix 4). The Project Log will document the implementation tasks carried out, time taken, resources used to complete each given task, staff attendance at meetings and focus-groups, and any challenges encountered. Project Log data will be used to assess the implementation outcomes of fidelity, feasibility, adoption, implementation cost, penetration and sustainability, and will also inform a separate cost analysis.

Patient and public involvement

While the HaSP trial targets the behaviours of healthcare professionals (as determined by audit and process mapping activities) to improve LS detection, ongoing consumer input from a single cancer patient was sought for the overall study design, conduct, reporting and dissemination of the research. The consumer was sought via the Cancer Voices NSW ‘Consumer Involvement in Research Programme’. The programme trains consumers in the basics of cancer research, and matches them with an appropriate researcher. The consumer is provided with regular telephone or email study updates, through which opportunities for input are raised and discussed. Future study findings may also be presented at LS patient and other relevant consumer forums.

Data analysis

The process evaluation will be analysed using a mixed-methods approach.14 Table 1 summarises the analysis approaches in relation to study measures and objectives. To investigate context and mechanisms of impact, results from the IPSBQ will be analysed using descriptive statistics and multivariate analysis of variance to assess for differences in perceived barriers preintervention and postintervention implementation. Where possible, mediation analysis will be performed to assess the degree to which changes in specific barriers influence changes in behaviour (ie, mechanisms of impact).

De-identified interview, meeting and focus group transcripts will undergo qualitative analysis in three stages, guided by the approach used by Gould et al.42 The first stage will involve a mixed deductive and inductive thematic analysis,59 with coding of transcripts guided by TDF domains4 and CFIR constructs,51 while allowing generation of new themes outside these frameworks. Coding of postimplementation interviews will also be guided by Proctor’s implementation outcomes.12 The second stage will involve an inductive analysis approach, by which similar responses within domains and/or constructs will be grouped and theme labels generated. In the third stage, key themes, domains and constructs will be identified based on frequency (eg, elicited in >60% participants) and expressed importance.42 The first transcript will be coded simultaneously by two researchers (familiar with the TDF and CFIR) to devise a strategy for subsequent transcripts. Subsequent transcripts will be coded by a single reviewer, with 10% independently double coded by a second reviewer to determine level of agreement.60 Reviewers will meet to resolve disagreements, and the advice of a third reviewer will be sought where consensus cannot be reached. Analysis will be performed iteratively, so that themes emerging in early pre-implementation interviews can be further explored in postimplementation interviews.14

The same qualitative analysis approach will be used for both theory and non-theory trial arms. Although participants in the non-theory group will not be exposed to the TDF, analysis with reference to the TDF (and CFIR) will enable assessment of whether intuitively identified barriers and intervention strategies align with a theoretical framework of behaviour change, and inferences about potential mechanisms of impact. Results of these exploratory analyses will be assessed in conjunction with the HaSP trial outcome measures (changes in the proportion of patients with risk-appropriate completion of the LS referral pathway, proportion of patients who were referred to genetic services, proportion of referred patients who attended genetic services, and proportion of patients with missing testing and referral data) to determine which mechanisms of impact are associated with behavioural change. Qualitative data will also be used to further explain any differences observed between trial arm sites.

Fidelity checklists, Implementation Lead project logs and MDT observation forms will be analysed using both quantitative and qualitative approaches (descriptive statistics and coding according to relevant frameworks; see table 1). These data will be triangulated with quantitative data on barriers and practice change. Descriptive statistical analysis of time and resource data (drawn from the Implementation Lead ‘Project Log’) will be conducted, including reporting of costs by stage and funder. These costs will be used to inform a future cost-effectiveness study.

Discussion

Use of a theory-based, mixed-methods process evaluation alongside a complex intervention trial is the optimal approach to understanding behavioural change, and is novel in the hereditary cancer setting. Furthermore, the HaSP trial is the first to compare a theoretically driven implementation approach against a non-theoretically driven approach, while keeping all other elements of structured implementation constant. Conducting an in-depth process evaluation of the HaSP trial presents a unique opportunity to advance understanding and refine the role of theory in implementation.

Results from this study will be used to interpret HaSP trial outcomes, and optimise the design of future interventions to increase detection of patients with LS on a larger scale and across different contexts. Identifying more patients with LS can save lives by enabling access to risk management strategies aimed at cancer prevention and early detection. Such work can be applied to improve detection of other genetic conditions, and is particularly timely in the hereditary cancer setting as health systems struggle to integrate findings from the rapidly evolving field of genomic research into routine clinical practice.3 32

Beyond LS, findings from this study will contribute to implementation and behavioural science efforts more broadly by advancing understanding of (1) factors affecting the success of implementation strategies to enhance uptake of evidence-based research and (2) the role and importance of theory in intervention design. Findings from this process evaluation will help to understand what works, in what contexts, why and at what costs—hence optimising the design of future implementation strategies to promote uptake of evidence-based best practice guidelines within health systems, thereby improving patient care and outcomes.

Ethics and dissemination

Ethical approval has been granted for this study by the Royal Prince Alfred Hospital Human Research Ethics Committee (ref HREC/17/RPAH/542). Site-specific governance will be obtained for each site prior to commencing study activities, and individual consent will be obtained prior to participation in study activities. Amendments to the original submitted protocol are subject to further ethical review, and will be communicated to investigators and participants (where relevant) on approval. Results will be disseminated via publications in peer-reviewed journals and presentations at relevant conferences.