Development of an implementation and evaluation strategy for the Australian 'Zero Childhood Cancer' (Zero) Program: a study protocol.

INTRODUCTION: Effective implementation of a research Program requires an actionable plan to guide execution. To assess the actionability and success of that plan, both scientific and implementation elements must be taken into account. The aim of this study is to assess the 'Zero Childhood Cancer Personalised Medicine Program' (the Zero Program), an Australian first-ever and most comprehensive personalised medicine programme for children with high-risk or relapsed cancer, in terms of its structure, process and implementational effect. METHODS AND ANALYSIS: We will assess Program delivery mechanisms. The development of the implementation and evaluation strategy will concentrate on the work of the Zero Program as a complex whole. This includes the structure of collaborative links across stakeholder groups involved in Program development and delivery, changes to collaborative relationships over time and the impact of group working on Program outcomes. We are applying a mixed-methods design including: a rapid ethnography (observations of stakeholder interactions and informal conversations), Program professionals' completion of a rapid health implementation proforma and a social network analysis. Formative evaluations of the implementation science effects, applying feedback techniques, for example, Formative Evaluation Feedback Loops and the Zero Program professionals' feedback, will determine where Program tailoring may be needed. A repeat of the social network analysis downstream will examine network changes over time, followed by an expert panel using the expert recommendations for implementing change to assess the integration of implementation strategies into the Program structure. A summative evaluation of the Program will bring the research elements together, leading to comprehensive data triangulation and determining the sustainability and implementational effects of Program delivery. ETHICS AND DISSEMINATION: Ethical approval for this study has been granted by Hunter New England Research Ethics Committee, New South Wales, Australia (approval ref: 2019/ETH12025). Knowledge translation will be achieved through publications, reports and conference presentations to healthcare professionals, patients, families and researchers. TRIAL REGISTRATION: NCT03336931; Pre-results. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This study offers an indication of where changes that influence network sustainability are most likely to take place.

Unique insights will be gained by embedding formative and summative rapid-cycle evaluations of the Zero Program effect at key moments in study progress.

This study attempts to overcome limitations in genomics research regarding insufficient use of implementation frameworks.

A strong interdisciplinary relationship between implementation scientists and key Program members uniquely enables reflexive practices to occur.

Due to ethical constraints, this study concentrates on the views and experiences of clinical and non-clinical staff only with families’ views addressed in a substudy.

Introduction

Cancer remains a major cause of death worldwide1 with childhood cancer being a leading cause of mortality among children in the high-income countries.2 Among Australian children, cancer is also the number one cause of non-accidental death, and of the 1600 children and young adults (age <25) diagnosed with cancer in Australia annually, approximately 300 will die of the disease.3 Childhood malignancies include a variety of different tumour types that are frequently treated using conventional cytotoxic chemotherapies, radiotherapy and/or surgical intervention. Any of these treatments or combinations thereof can lead to acute and chronic side effects with up to 30% of childhood cancer survivors suffering serious chronic health conditions after successful cancer treatment.4 In addition, both acute adverse effects and long-term sequelae contribute significantly to the burden on the healthcare system, affecting resource use and service provision. Novel therapeutic approaches such as genomically guided targeted drugs and novel cellular therapies may improve survival of patients with poor-prognosis cancers and enhance quality of life through a reduction in toxicity and thus long-term side effects in childhood cancer survivors, thus reducing the burden on the long-term healthcare system.

Precision medicine, involving the development and tailoring of treatments based on genomic alterations in cancer cells that drive and maintain tumour growth, is one of the novel therapeutic approaches currently being introduced into Australian childhood cancer research and clinical practice. Paediatric cancer precision medicine is extensively tested through Programs such as the Zero Childhood Cancer Personalised Medicine Program (Zero Program) and its current embedded national clinical trial: PRecISion Medicine for Children with Cancer (PRISM).

The Zero Program, PRISM and paediatric precision medicine

The Zero Program and its embedded clinical trial, PRISM, aim to offer precision medicine to paediatric cancer patients with high-risk or relapsed malignancies (expected survival <30%). The overall goal is improved treatment options leading to improved survival outcomes. The Zero Program is unique, as it is part of a complex network of research and clinical multidisciplinary collaborators working together in real time to recommend tailored therapies for children and adolescents with high-risk cancer. The Zero Program clinical trial platform includes comprehensive genetic testing of both tumour and non-tumour material, coupled with the use of biological models such as the individual patient’s tumour cell cultures and patient-derived xenografts (‘mouse avatars’) to test drug effectiveness in living models and to ultimately inform care pathways. The information derived from this series of tests is then assessed by Zero’s national Multidisciplinary Tumour Board (MTB) for clinical relevance. Clinically relevant information is fed back to the treating clinicians, who can choose, in collaboration with the child’s parents, whether to tailor treatment based on this personalised information or conduct care as usual following previously defined pathways for these high-risk children.

As Zero is a novel precision medicine model of healthcare delivery, it is as yet unknown how clinicians’ behaviours and practices (individual and team) will be affected. However, we are aware of the exponential speed with which this application of advanced technologies is evolving5–7 with extensive new research being published daily on novel pathogenic genetic variants relevant to clinical practice. Given the rapid pace at which precision medicine is growing,8 it is important to use evidence-based, theoretical approaches (according to implementation of science-driven theories, frameworks and models) to plan, predict, guide and evaluate the process of implementation of precision medicine programmes.9 However, a recent review demonstrated that fewer than 2% of studies aiming to translate genomic research into clinical practice explicitly made use of implementation science frameworks.7 In addition, there was a noted lack of research developing and evaluating evidence-based strategies for enhancing the implementation of genomic medicine.7 The study advanced in this protocol aims to address these gaps in an evidence-based manner by (a) proposing an augmented implementation science framework tailored for the Zero Program and (b) developing a rapid-cycle evaluation of a novel, evidence-based implementation strategy and enhancing stakeholder collaboration and involvement. The purpose of this study is to assess the Zero Program, an Australian first-ever and most comprehensive personalised medicine Program for children with high-risk or relapsed cancer, in terms of its structure, process and implementational effect.

Implementation science frameworks

Traditionally, implementation theories, models and frameworks have been designed to underpin rigorous implementation of interventions or new ideas in scientific settings.10 11 They have been presented by Lynch et al9 as including: the Consolidated Framework for Implementation Research (CFIR)12; the Reach, Effectiveness, Adoption, Implementation and Maintenance Framework13; the Promoting Action on Research Implementation in Health Services framework14 and the Theoretical Domains Framework.15 The low uptake of implementation frameworks in genomics research7 may be attributed to the large number of existing implementation frameworks that propose ‘what works’, but ignore ‘where, how and why it works’, thus overlooking the essential pragmatic interplay between individuals, groups and systems at work.12 Some implementation frameworks provide constructs directed at individual-level change but do not adequately elaborate system-level change.9 CFIR, unlike other frameworks discussed by Lynch et al,9 elaborates system-level constructs as a comprehensive taxonomy that is operationally defined. At the same time, it recognises that these constructs are inextricably linked to individual action and behaviour and it has been proven to influence the implementation of complex programmes by embracing, consolidating and unifying key constructs from published implementation theories.12

An augmented implementation science framework for implementing paediatric precision medicine

We planned to develop a tailored implementation science framework (The CFIR Framework)12 according to data from (a) a rapid ethnography (observations and informal conversations) and (b) a social network analysis (SNA) which will be undertaken on two occasions to examine relationships between professional groups involved in the Program and changes to networks and individual relationships over time. The CFIR Framework12 will ensure that individual and whole of system aspects of the Zero Program are taken into consideration and will help to identify working practices, Program context, relationships, activities and the complexities surrounding the implementation of precision medicine, including barriers and facilitators to implementation. The CFIR will be able to offer a more predictable means to ensure effective implementation of precision medicine practices which could be broadened out to other fields in the longer term. The CFIR Framework12 comprises five major domains with eight subcomponents. These relate to the intervention itself; the outer context (such as patient need and resources); the inner setting (such as culture and leadership); individual characteristics and processes (such as planning and reflection). Inner setting, outer context and characteristics of individuals also closely relate to contextual domains.12 These domains interact in rich and complex ways to influence implementation effectiveness. By using CFIR, with different data capture elements supporting rigorous triangulation of results, we aim to explore healthcare system-level effects (macro level), take account of microsystems at play (healthcare professionals as a defined population in their unique environments) and the different settings in which healthcare professionals work (meso-level elements). With the long-lasting impact most likely to be achieved by considering all levels simultaneously,16 17 we plan to explore the interplay between individuals and systems, leading to the generation of a tailored implementation strategy to address barriers to Program embedment and longitudinal effect.

Rapid-cycle evaluation to generate an evidence-based implementation strategy for implementing paediatric precision medicine

To support the assessment of this evidence-based implementation strategy, we will employ a rapid-cycle evaluation. A suite of different types of evaluations are on offer that could be conducted during implementation assessment phases as reported in the literature.18 Medical Research Council guidance also offers practical applications for testing complex interventions using process evaluation19 but without specific guidelines around applications in a rapid-cycle context. In light of the exponential speed that precision medicine is evolving within Australia, we aim to apply the rapid-cycle Formative Evaluation Feedback Loop (FEFL) process model, adapted from Braithwaite et al’s20 original design, to examine eight success factors of implementation: (1) preparing for change, (2) capacity for implementation—people, (3) capacity for implementation—setting, (4) types of implementation, (5) resources, (6) leverage, (7) desirable implementation enabling features and (8) sustainability.

FEFL models include formative and summative evaluations embedded in study phases to (1) identify potential and actual influences on the progress and effectiveness of implementation efforts21 and (2) generate strategies to overcome any barriers recognised during study stages to increase implementation effectiveness.22 FEFL applications will be augmented by expert recommendations for implementing change (ERIC),23–25 whereby expert panels provide conceptual clarity with regards to implementation strategies likely to be effective. Insights gained from the FEFL and ERIC data will support the future development of the Zero Program, with the FEFLs identifying potential gaps between evidence and practice in precision medicine and opportunities to examine new implementation strategies for long-term sustainability.

Study objectives

Against this background, this study aims to:

Assess the barriers and facilitators to the integration of a genomics precision medicine model of care in childhood cancer across clinical settings.

Map the social networks involved in the Zero Program’s development and management and how networks coalesce over time to inform implementation successes.

Test a new methodology in this context underpinned by an augmented implementation science framework and rapid-cycle evaluation to evaluate system-level implementational effect.

Disclose new opportunities for a coordinated national implementation initiative that will be key to influencing policy for translation and integration into childhood cancer treatment.

Fulfilling these aims will benefit members of the Zero Program and members of the implementation science research team, with scope to provide evidence that may benefit wider population groups and other precision medicine models of care. Building in feedback loops at key study time points through formative and summative FEFLs will ensure that study findings are disseminated in a timely manner, while including the views of a range of stakeholders adds rigour to reporting.

Methods and analysis

Study design

Alongside the Health Implementation Scientists, the Program Leader, a lead Zero Program clinician from the central site and two operations managers are also investigators on the health implementation science study, while a key implementation scientist is a chief investigator on the Genomics Program. This 5-year study (2019–2023) will be conducted via a mixed-method study design (see table 1 for type and sequence of activities). Mixed-method approaches provide both rich and detailed information and will facilitate a tailored implementation strategy for the Zero Program, while the augmented implementation science framework will incorporate: (1) the use of CFIR; (2) a rapid ethnography (including observations, informal conversations and healthcare professional proformas), (3) two rounds of SNA alongside an in-depth SNA data appraisal, (4) FEFLs as part of a rapid-cycle evaluation integrating quantitative and qualitative data augmented by ERIC to help identify and develop implementation strategies and (5) data triangulation to inform the final evaluation and achieve a comprehensive understanding of Zero’s processes and impact. It was not appropriate or possible to involve patients or the public in the design, or conduct, or reporting, or dissemination of our research.

Table 1

Representation of mixed-methods triangulation design

Phase	Stage	Augmented framework	Data collection	Data analysis
Phase 1(identifying barriers and facilitators to Program implementation)	1	Demographic survey	Quantitative data collection (online survey)	Quantitative data (descriptive statistics)
	2	Rapid ethnographyCarry out observationsInvolve stakeholders to identify individuals who influence or who are influenced by Zero	Qualitative data collection(observations 100 hours)	Qualitative data analysis(thematic analysis/content analysis)
	3	RHIPInvolve stakeholders in item development to tailor CFIR items to create RHIP and ensure clinical and theoretical relevance of RHIP which is designed and administered to assess implementation barriers and facilitators in precision medicine	RHIP (online survey 100 participants) or 10 interviews	Quantitative data(descriptive statistics)and qualitative data analysis(thematic analysis/content analysis)
	4	SNA part IDescribe and map members of the network, analyse the structure of the network using network analysis	Quantitative data collection(online survey (~100 participants) (SNA part I))	Graph theoryquantitative data analysis(SNA part I)
Rapid-cycle evaluation		First formative evaluation
Phase 2(identifying strategies to overcome barriers and enhance facilitators to Program implementation)	1	SNA part IIRepeat of SNA part 1 to map networks over time	Quantitative data collection(online survey (~100 participants) (SNA part II))	Graph theoryquantitative data analysis(SNA part II)
	2	Expert panelGenerate implementation strategies to overcome barriers (based on ERIC)	Qualitative data collection(expert panel with 8–10 participants)	Qualitative data analysis(thematic analysis/content analysis)
Rapid-cycle evaluation		Second formative evaluation and summative evaluation

CFIR, Consolidated Framework for Implementation Research; ERIC, expert recommendations for implementing change; RHIP, rapid health implementation proforma; SNA, social network analysis.

Study population

This is a multicentre study currently working with eight paediatric oncology hospitals involved in the Zero Program across Australia. The study population comprises a range of the Zero Program clinical and non-clinical members, with each constituent group sampled to ensure a purposive sample representative of a range of clinical, research, operational, administrative and leadership staff working on the Program, according to participant inclusion criteria (see table 2).

Table 2

Sample and inclusion/exclusion criteria

Sample: (1) Clinicians, (2) administrators, (3) program managers and operations managers, (4) clinical research assistants/clinical trial managers/coordinators, (5) laboratory research scientists, (6) research team members, (7) staff involved in the ethics coordination process, (10) allied healthcare professionals
Inclusion criteria	Exclusion criteria
Participant is willing and able to give informed consent for participation in the study	Participants (clinical, non-clinical, stakeholders) unwilling or unable to give informed consent or unwilling or unable to take part in the study
Is a healthcare professional, committee or board member or a manager related to the PRISM trial or the Zero Program, or is a health professional on the expert panel for the PRISM trial or the Zero Program
Is a non-clinical professional or allied healthcare professional, related to the PRISM trial or The Zero Program, or is a non-clinical professional or allied healthcare professional on the expert panel for the PRISM trial or the Zero Program
Is a stakeholder (service-user representative or similar) within the PRISM trial or the Zero Program or sitting on an expert panel related to the Zero Program

PRISM, PRecISion Medicine for Children with Cancer.

Patient and public involvement

Patients and the public were not involved in the design or planning of the study.

Data collection

Stages of the data collection’s procedure and analysis are shown in table 1. Qualitative data collection activities will be conducted in situ at Program sites (face-to-face) and via telephone, email or Skype to ensure optimal involvement of busy clinicians. Face-to-face and in-situ activities (such as observations of everyday working practices, informal conversations with Program members and expert panel meetings to inform ERIC) will be conducted at the central site in Sydney, while further observation work (as required) will be undertaken with research scientists, laboratory scientists, genomic scientists and Program managers. A link to the online SNA survey and rapid health implementation proforma (RHIP) survey will be emailed to participants. In addition, hard copy versions will be available from the researcher (JS) on site. The survey is held on the Qualtrics platform which is hosted on the server of the lead project site.

Phase 1: identifying barriers and facilitators to implementing paediatric precision medicine

Demographic questionnaire

All participants involved in data collection will complete a demographic questionnaire about their roles and responsibilities, seniority, gender and age. Demographic data will complement rich, nuanced data capture across the study. The views and experiences of a range of healthcare professionals and other stakeholders involved in the Zero Program and PRISM trial will be included, with demographic details informing the evaluation of stakeholder characteristics. An online version will also be available to accompany the RHIP surveys and SNA work.

Rapid ethnography

Rapid ethnography is a qualitative research method that allows an independent observer to work in situ in the field, focusing on activities of importance undertaken and relevant interactions.26 Rapid ethnography increases the likelihood of close observations of clinical and non-clinical behaviours in naturalistic settings. The methodology foregrounds informal conversations with those delivering or supporting the delivery of care. As data (observations and informal conversations) are collected ‘on the hoof’,27 it enables a dedicated study researcher (in this instance JS) to examine what is happening in the clinical setting, people’s everyday behaviours and how conversations unfold. Being observant of, or taking part in, informal conversations when working in situ, rather than organising a formal, static interview (as is more commonly the case with qualitative data) allows for close observations of clinical adjustments, behaviours and interactions in real time.

The study population involved in this part of the study will comprise a range of the Zero Program clinical and non-clinical members of staff, with each constituent group represented (with the support of the operations and Program managers) from each of the eight study sites. Operations and Program managers will also support JS in identifying key players in Program activity, and as a result they act as a bridge between the implementation science team and broader Zero workforce. Key informants will be involved in informal conversations, as they go about their daily activities to examine a wide range of topics that may include: (1) working practices, (2) Program functioning, (3) collaborative and independent working, (4) related activities and (5) barriers and facilitators to the implementation of precision medicine, in the care of children with cancer.

Observing and interacting with people in situ during the rapid ethnography phase of this study will offer insights into who is involved in the Program and how work and relationships affect Program development. Up to 100 observational episodes (some of which will include informal conversations) are planned over a 6-month period (100 hours or until no new themes are emerging and data saturation is achieved). Observations will take place at different times of the day to ensure the observation of more naturalistic behaviours. Researcher field notes and an observation sheet will form the basis of the qualitative data set and will be completed continuously throughout field visits.

Rapid health implementation proforma

Key players, including healthcare professionals who are the subjects of the observations, will complete a RHIP complementing data acquired through the rapid ethnography. The RHIP will explore: (1) what the Zero Program is achieving; (2) individual and group needs and characteristics; (3) Program facilitators and barriers and (4) macro-level, meso-level and micro-level system functioning. The RHIP will be administered to approximately 100 participants according to a purposive sample of different professional groups across settings and sites (eg, managers, oncologists, surgeons, consultants, researchers and laboratory technicians identified through Program and operations managers). RHIPs are predominantly qualitative and text-dependent, containing a limited number of questions. However, for the current study, the RHIP will be adapted to include both qualitative and quantitative items to examine response frequencies alongside text responses. Proformas are commonly incorporated into a study to support other data collection methods28 as they encourage a more detailed understanding of a Program’s development and delivery than single methods can achieve.

Themes and categories will be elicited from an analysis of the qualitative data from demographic questionnaires, observations and the RHIP using NVivoV.12 plus as a data management tool that the researchers will derive a thematic qualitative data analysis framework.29 Group work will help the team derive the framework and support the emergence of key themes within the data. All data will be discussed by at least two analysts until consensus is achieved to ensure that data findings are rigorous. Once data have been thematised, findings will support the next phase of the study, the SNA.

Social network analysis part 1

Approximately 100 healthcare professionals working with the Zero Program will be invited to participate in a whole network, online SNA survey.30 31 Invited participants will come from different sectors, organisations and settings and together make up the respective parts of the Zero Program network. A name generator design will be used.32 Participants will be asked to nominate up to five collaborative contacts that they ‘have consulted, sought advice from, or discussed some issue of the project with (eg, specimen processing, interpretation of results, management of a child’s disease)’. For each person nominated, participants are asked to say whether they were a regular, occasional or new contact, allowing us to assess the impact of the Zero Program on existing networks. A second question asks participants to nominate contacts from outside the Zero Program who play a supporting role. Participants’ answers will be aggregated to build a directed sociogram of the whole Zero network, and of the wider links outside the programme. Data will be analysed using UCInet V.633 and sociograms of the relationships will be constructed using NetDraw software.34 Setting up the SNA once the Program has been established enables us to examine the relationships and networks that are under development. The SNA survey will help clarify the effect of newly configured networks on relationships and Program activities.35 Problem areas may be diagnosed from this analysis, for example showing poorly connected sites or isolated individuals requiring more practical support in their work.35–37 These barriers will inform the FEFLs.

Phase 2: identifying strategies to overcome barriers to implementing paediatric precision medicine

Social network analysis part 2

Phase 2 of data collection will help to identify strategies to overcome the barriers to successful implementation of paediatric precision medicine derived from phase 1. Phase 2 SNA data collection will be repeated with the same participants towards the end of the study to capture a comprehensive data set and enable the tracking of change to social networks over time (see table 1). The repeat survey will use the same questions as in SNA part 1. This second iteration of the SNA will enable the investigation of any changes to networks. SNA parts 1 and 2 SNA surveys concentrate on interactions with healthcare professionals within and outside the Program and may include MTB Meeting and Curation Meeting members.

All individuals providing information about their networks will be assured of anonymity and data confidentiality. The survey will be piloted on stakeholders in a similar network before being rolled out, and questions will be refined as necessary to ensure contextual clarity. Apart from relationship questions, the SNA survey will include additional items examining key player roles and daily work activities. Analysis of network parameters such as density of the network and brokerage potential of individuals will be computed, again informing the design of implementation strategies. Sociograms and network parameters can not only diagnose the strength of a network, but also point to risks to its efficient operation (see table 1). It can also highlight differences between different sites (clusters) within the whole network. There are risks involved in repeating surveys within a relatively short time frame, such as answer recall or research fatigue.35 However, we will mitigate against these by ensuring that Zero Program managers alert participants to the repeat survey 2 weeks preceding the email invitation to participate.

Expert panel

Expert panels are a useful approach to compile both individual and group views on practices and activities, while assessment of group activity takes into account group dynamics and interactions.38 The process will involve consensus-building activities24 towards agreement on an appropriate tailored implementation strategy, with strong researcher facilitation of the expert panel work. The expert panel will be held towards the end of the project (see table 1) to meet the needs of the implementation science project design. The input of an expert panel in this study will support the development of implementation strategies and approaches and will involve Program members in active participation. Presenting data to the expert panel at the end of the study will lend itself to eliciting views about how to successfully address barriers to implementation. Expert panel members will be identified by the Zero Program Management Team and they will be presented with an overview of all data to be discussed in detail. The panel will consist of participants chosen pragmatically and purposively as knowledgeable agents are able to help take the implementation strategies forward. The expert panel discussion will be recorded, anonymised and analysed (via thematic analysis) to assess implementation processes and mechanism and an appropriate implementation strategy (supported by ERIC).24 The study team will report back at the conclusion of the project to the Program leader and key players across the Zero Program through an end of study report and executive summary.

Formative and summative evaluation feedback loops

Ongoing evaluation over the course of a Program such as Zero is important, and as a result the study team has built in two rapid-cycle evaluations at two time points (after phase 1 and at the end of phase 2, see table 1). These are formative and summative feedback sessions, sharing data through preliminary findings with others in the Program and ensuring that teams of researchers, study developers and site participants are all involved in reflective discussions about clinical work and how the precision medicine model is achieved. Rapid-cycle feedback loops will support the evaluation process and ensure that any necessary changes to study design or implementation strategy are driven by project participants. Collaboratively, while informing study stages, formative and summative feedback sessions will ensure usable results. Building in rapid-cycle feedback loops at key time points through formative evaluations will also ensure that study findings are both timely and tailored to the Zero Program to ensure rigorous implementation planning around outcomes. The FEFLs model will help guide formative and summative evaluations. The formative evaluation will involve assessing and presenting the degree to which Program interventions are reaching intended recipients. This will help highlight recipient engagement and interactions, while indicating how to maximise effectiveness, including how to overcome barriers and enhance enablers influencing effectiveness. The summative evaluation (during phase 2) will repeat the FEFL analysis and provide further in-depth information about ongoing engagement, interaction and effectiveness for feedback to Program participants. FEFLs will be recorded and a comparison of formative and summative evaluations and participant responses will be undertaken. Formative evaluation will enable an analysis of data as discrete, individual units, before all data sets are triangulated.28 Summative evaluation will precede data reporting, while data will be presented according to full agreement (data convergence), partial agreement (complementarity between data), conflicting findings (discord) or silence (findings identified in only one data source and no additional sources).39–41 By utilising multiple sources of information, we will gain a richer understanding of what is taking place in the Zero Program. By triangulating data, we will indicate corroboration and divergence, and as a result provide a more comprehensive report, increasing credibility and offering a plausible representation of the issues under review.42

Ethics and dissemination

Knowledge translation will be achieved through publications, reports and conference presentations to healthcare professionals, patients, families and researchers. Following full ethical approval and site governance, informed consent was supplied for each of the individual phases of the study. In addition, for the rapid ethnography work we also included an opt-out process.

Significance and study impact

There are many challenges to effective implementation of a precision medicine model of care, such as the one being trialled by the Zero Program through PRISM. The implementation of Program outputs as the research develops (the translational phase of Program development) while maintaining relevance in a rapidly evolving discipline of paediatric precision medicine is another considerable challenge. These challenges are further complicated by the fact that healthcare systems are complex adaptive systems.43 44 Healthcare systems comprise clusters of multiple, interacting and interdependent parts whose interconnections and activities in real time and over time alter the context, outcomes and behaviours of those directly and indirectly involved in a healthcare system.43 44 As indicated, the Zero Program is a complex, multilevel, multicomponent intervention and therefore precision medicine in paediatric cancer is susceptible to variation in implementation across different sites. If the Program is not implemented effectively, it can neither produce the expected effects nor can it achieve its aims and objectives.

Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Conclusion

In this project, we considered a wide range of stakeholder views, including health professionals and non-clinical professionals involved in the Zero Program’s PRISM trial, regarding the structures and process that they work with to make a long-term implementation strategy as sound and successful as possible. We are researching the relationships and collaborative links across the Zero Program network and tracking changes in the constitution of the network over time. Close working relationships with Program management, and regular feedback will ensure successful implementation and tailoring of the Program. Building in feedback loops is a significant study addition and it will ensure that any potential pitfalls in extending Program implementation in the health system can be recognised, and where possible negated.