Protocol for establishing a core outcome set for evaluation in studies of pulmonary exacerbations in people with cystic fibrosis.

INTRODUCTION: Pulmonary exacerbations are associated with increased morbidity and mortality in people with cystic fibrosis (CF). There is no consensus about which outcomes should be evaluated in studies of pulmonary exacerbations or how these outcomes should be measured. Outcomes of importance to people with lived experience of the disease are frequently omitted or inconsistently reported in studies, which limits the value of such studies for informing practice and policy. To better standardise outcome reporting and measurement, we aim to develop a core outcome set for studies of pulmonary exacerbations in people with CF (COS-PEX) and consensus recommendations for measurement of core outcomes. METHODS AND ANALYSIS: Preliminary work for development of COS-PEX has been reported, including (1) systematic reviews of outcomes and methods for measurement reported in existing studies of pulmonary exacerbations; (2) workshops with people affected by CF within Australia; and (3) a Bayesian knowledge expert elicitation workshop with health professionals to ascertain outcomes of importance. Here we describe a protocol for the additional stages required for COS-PEX development and consensus methods for measurement of core outcomes. These include (1) an international two-round online Delphi survey and (2) consensus workshops to review and endorse the proposed COS-PEX and to agree with methods for measurement. ETHICS AND DISSEMINATION: National mutual ethics scheme approval has been provided by the Child and Adolescent Health Service Human Research Ethics Committee (RGS 4926). Results will be disseminated via consumer and research networks and peer-reviewed publications. This study is registered with the Core Outcome Measures in Effectiveness Trials database. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This will be the first core outcome set (COS) for studies of pulmonary exacerbations in people with cystic fibrosis.

The protocol for generation of this COS has been adapted from the Core Outcome Measures in Effectiveness Trials framework.

Two rounds of online Delphi (eDelphi) surveys and a series of stakeholder workshops will be conducted to develop a COS and consensus methods for measurement of prioritised outcomes.

An international steering committee comprising diverse representatives from varied locations globally will oversee the development of this COS.

Stakeholders involved in COS development are likely to reside in high-income countries; consequently, this COS may not be applicable to research conducted in low to middle-income countries.

Introduction

Cystic fibrosis (CF) is a life-limiting disease characterised by episodic pulmonary exacerbations which are thought to drive progressive lung damage.1 Treatment for these episodes is complex and generally involves a combination of antimicrobials, therapies to improve airway clearance (including chest physiotherapy and mucoactive agents), optimisation of nutrition, psychosocial counselling and possibly anti-inflammatories.1–5 Of the 10 Cochrane reviews evaluating trials of treatment strategies for pulmonary exacerbations, most were inconclusive and many controversies about treatment remain.1–10 Synthesis of data is impeded by inconsistency in the selection of outcomes in these studies and how they have been assessed, and by the reporting of outcomes that might not be meaningful to people living with disease.

A core outcome set (COS) represents a minimum set of agreed outcomes derived from broad stakeholder consensus for measurement and reporting in all trials for a specific condition.11 Trialists can add other outcomes relevant to the trial. The Core Outcome Measures in Effectiveness Trials (COMET) initiative was established in 2010 to improve consistency in the selection of meaningful outcomes when designing studies, to facilitate collaboration, avoid duplication and to improve the value of the research that is conducted.12

We have reviewed the range of outcomes previously reported in studies of pulmonary exacerbations in people with CF13 and the methods used for measuring these outcomes.14 We have engaged Australian stakeholders including clinicians, people 13 years and above with CF and carers of children less than 18 years of age with CF to identify outcomes of importance to them. The top 10 outcomes capturing symptoms or functional capacity from the perspective of people affected by CF were difficulty/painful breathing, sputum production and clearance, fatigue, appetite, pain, motivation/demoralisation, fevers/night sweats, treatment burden, inability to meet personal, school or work goals and avoidance of gastrointestinal symptoms (constipation, bloating and flatulence).15 Our group has also conducted a Bayesian expert knowledge elicitation workshop to elicit outcomes of importance to healthcare professionals (Snelling, unpublished). It is unknown, however, if these priorities are shared by stakeholders outside Australia.

We have therefore established an international steering committee to oversee the development of COS. The primary aim of this international collaborative group will be to oversee the development of a COS for evaluation in studies of pulmonary exacerbations in people with CF and consensus recommendations for the measurement of core outcomes based on the shared priorities of key stakeholders. The steering committee comprises a range of subject matter experts from different countries including people with CF from diverse backgrounds, healthcare professionals and healthcare commissioners, researchers, people affected by disease and people involved in the dissemination and translation of research findings into practice and policy. It is expected that this research will aid the selection of meaningful outcomes for evaluation in relevant studies in order to optimise the value of the research and to minimise research waste.16

Methods and analysis

The core outcome set for studies of pulmonary exacerbations in people with CF (COS-PEX) is registered in the COMET database.17 The protocol for generation of this COS has been adapted from the COMET framework.17 In addition to the work described above, development of the COS-PEX and consensus methods for measurement of core outcomes will involve three additional steps; these are presented in figure 1. This project will be conducted between April 2022 and December 2023. The project will be conducted in accordance with the Core Outcome Set Standards for Development.18 Results will be reported according to the Core Outcome Set Standards for Reporting.19

Figure 1

Method for development of core outcome set for studies of pulmonary exacerbations in people with CF (COS-PEX).

Patient and public involvement

Two investigators affected by CF, including one person living with disease and a mother of twins with CF, are included as investigators on this study. They were both involved in contributing to protocol development and meet the ICMJE requirements for authorship. Previously, we have conducted two rounds of workshops and online surveys to elicit outcomes of importance to people affected by CF for evaluation in the online Delphi (eDelphi) surveys described in this study. This involved 95 responses from people >13 years living with disease and carers of children with CF less than 18 years of age.

Step 1: eDelphi surveys

Outcomes will be defined according to the taxonomy proposed by COMET.17 20 An outcome will be defined as a measurement or observation used to capture and assess the effect of treatment such as an assessment of side effects (risk) or effectiveness (benefits). Outcome domains (such as lung function) will be defined as an aspect of health that is likely to be impacted by a healthcare intervention. Outcomes (such as forced expiratory volume in 1 s) identified from the systematic review13 and preliminary workshops21 will be mapped to domains by members of the steering committee in order to structure a list for evaluation in the eDelphi survey.

An international two-round eDelphi will be conducted to generate consensus about outcome domains of importance to key stakeholders (figure 1). This will involve two rounds of 20–30 min surveys answered anonymously; participants will be asked to rate approximately 30 outcomes randomised by block allocation according to their perceived importance. There will be two versions of the survey, including (1) for children with CF aged between 8 and 18 years; (2) for adults with CF and other stakeholders. Each version will evaluate identical outcomes, although the wording will be tailored to the targeted population’s age and role, and the paediatric version will include a picture illustrating each outcome. Both versions will include plain language definitions for the outcomes presented. The version for children will be pitched at grade 5 reading level according to the Flesch-Kincaid Index (10 years of age). Both versions will be piloted to a minimum of five people and feedback will be incorporated prior to finalisation.

The eDelphi method has been validated as a reliable approach for achieving consensus on COS for various health conditions.22 23 This method involves participants contributing subject matter knowledge independently, and then having the opportunity to revise their responses based on the feedback and opinions offered by other respondents. Participants do not interact directly with each other, thereby avoiding domination of the discussion by few contributors. Data will be reported according to the checklist recommended by Sinha et al24; this will include a discussion of the size and composition of the panel, the Delphi method and the results.

Participants and recruitment

Subject matter experts including people with CF from diverse backgrounds, carers of children less than 18 years of age, healthcare professionals, researchers, journal editors, policy makers and regulatory and pharmaceutical authority representatives will be eligible to participate. We will employ multiple recruitment strategies to ensure inclusivity and diversity of representation, including recruitment through (1) participating medical facilities; (2) consumer and patient networks, including, but not limited to, advertising via email and social media including Facebook and Twitter; and (3) recruitment by investigators with lived experience of disease. We will also use snowballing strategies enabling participants to extend an invitation to other relevant stakeholders to participate. Individuals will also be able to access the survey directly at https://www.beatcf.org.au. Monetary remuneration may be offered to participants as compensation for their time to promote the representativeness of the sample.

There are no recommendations available to guide the determination of a minimal sample size for Delphi surveys for the purposes of developing COS.25 Our target sample size will be a minimum of 250 respondents, which is just above the lower participant limit used to develop COS reported in the literature; we will aim to include at least 200 people with CF, 25 carers of children with CF and 25 health professionals and members from industry and regulatory bodies.

Survey administration

A link will be provided to access the online survey. The surveys will be developed in research software Qualtrics.26 Responses will be anonymous, and all data will be non-identifiable. Participant information and consent will be included online. Individual consent will be required for participation, and consent from guardians of children between 8 and 18 years old will also be required for children to participate. Individuals will be able to exit the online surveys at any time prior to submission of their responses. After this time, anonymisation will make it impossible to withdraw their responses.

Data collection: round 1

Each participant will rate the outcome domains according to the 9-point Likert scale suggested by Grading of Recommendations Assessment, Development and Evaluation27; ratings from 1 to 3 reflect outcomes of ‘limited importance’, ratings from 4 to 6 include outcomes that are ‘important but not critical’ and ratings between 7 and 9 indicate outcomes of ‘critical importance’. An ‘unable to score’ option will also be available. A best worst evaluation exercise will also be included in the survey; this is an established method that can be used to calculate the relative importance of juxtaposed outcomes.28 Participants will be given the opportunity to provide feedback including a rationale for their answers and to suggest additional outcome domains of importance not included in the round 1 survey.

Basic demographic information including sex, age and stakeholder group will be requested. Clinicians will be asked how many full-time equivalent years of clinical experience they have had in caring for people with CF, and people with CF will be asked several questions to help categorise their severity of disease (such as number of hospitalisations over the past 12 months and lung transplant status).

Responses will be considered separately for the following groups: (1) children and their carers, and (2) adults with CF and other stakeholders. Outcomes with a mean and median score greater than 7 based on responses from at least 70% of respondents will be included in round 2, as well as new outcome domains that are suggested by more than 10% of participants.

Data collection: round 2

In the round 2 survey, participants will be presented with a graph of the distribution of scores for each outcome for the respective groups. An explanation to aid interpretation of the graph will be provided, including an animated explanation for the paediatric survey. Comments made by individual participants in the round 1 survey will be supplied to the individual who provided those responses to allow them to compare their own responses to those of the group. Individual responses will therefore remain anonymous to the remainder of the group. Participants will be asked to repeat the rating exercise using the same method described for round 1. A best worst exercise will also be repeated.

Prior to completion of the survey, participants will be invited to register their interest in participating in the consensus workshop(s) described in step 2 by supplying their name and email address.

Data analysis

Quantitative analysis will be conducted using STATA V.13. The analysis will involve calculating the distribution of scores and the mean, median and proportion of scores ranked for each Likert category, and the overall ranking of the outcome domains according to the responses from the two groups. Results for the paediatric and adult survey, as well as the differences in responses between people with CF compared with other stakeholders, will be compared. The criteria for inclusion in each COS are based on the recommendations specified by Outcome Measures in Rheumatology Clinical Trials.29 Domains with a median and mean of more than 7 based on responses from 70% or more from people with CF/carers and health professionals/policy makers for the respective surveys will be included as ‘middle tier’ outcomes at a minimum; the top 3–5 core outcome domains will be selected based on means, medians and proportions (figure 2). If the thresholds for inclusion are modified post hoc, these will be reported to ensure transparency. The utility function of outcomes examined in the best worst exercise will be calculated for all outcomes using conditional logit regression analyses.

Figure 2

Core outcome set for studies of pulmonary exacerbations in people with CF (COS-PEX).

Step 2: consensus stakeholder workshops

Two consensus workshops chaired by members of the steering committee will be conducted to review the proposed COS based on the results obtained from the two-stage eDelphi. The first workshop will focus on outcomes for adults with CF and the second workshop will focus on outcomes for children with CF. The workshops will be up to 2 hours in duration and will occur via videoconference, owing to infection control restrictions which preclude mixing of people with CF. All attendees will be invited to participate as investigators rather than research subjects; consent will therefore not be required.

Recruitment methods will be the same as those employed for the eDelphi. Anyone who is interested and meets eligibility criteria will be welcome to attend one or more workshops. Individuals who register their interest in attending will be provided with a copy of the written results of the relevant eDelphi. Those attending the paediatric COS workshop will also be provided with a link to an online animation to explain the results.

Data collection

Run sheets will be developed for the COS-PEX workshops. An assigned member of the investigator group will record notes on the group dynamics and interaction between participants. The anticipated workshop format will involve (1) welcome and presentation of results from step 1, (2) breakout discussion in groups comprising approximately 10 participants facilitated by a moderator to discuss the differences in results between groups and any identified issues, to resolve any uncertainties and to agree on the proposed scope of the COS (including the specific CF population(s), the setting and the type of intervention(s) for which the COS is likely to be relevant), (3) a summary of each group discussion will be reported back to the larger group, and (4) participants will be asked to endorse the final agreed COS. All participants will be given the opportunity to ask questions and discuss any differences or similarities in opinion.

Data analysis and reporting

Transcripts of the breakout discussions will be entered into the HyperRESEARCH software. CM will code these data and use thematic analysis to explore the range of perspectives for core outcome domains and will report key recommendations and anticipated challenges for implementation for the COS.

A preliminary plain language report will be disseminated to all workshop participants and relevant stakeholder groups for the COS-PEX and posted via the https://www.beatcf.org.au website to invite public comment for a period of 2 weeks. The steering committee will then finalise and endorse the final COS prior to dissemination of the results by peer-reviewed publication.

Step 3: consensus methods for measurement of the core outcome domains

Up to three online workshops will be convened by members of the steering committee to develop consensus recommendations regarding the measurement of the core outcomes identified for each COS based on the methods previously identified from a systematic review of tests and tools used to measure outcomes in trials of CF.11 All attendees will be invited to participate as investigators; consent will therefore not be required.

Recruitment methods will be the same as those employed for steps 1 and 2.

The anticipated workshop format will involve (1) presentation of available tests and tools for measurement of the core outcomes included in each COS identified by a recent systematic review and feasibility considerations, (2) breakout discussion in groups comprising approximately 10 participants facilitated by a moderator to discuss the utility of different tests and tools for measurement and to agree on the favoured tests or tools and the scope for use (including the specific CF population(s) and setting), (3) a summary of each group discussion will be reported back to the larger group, and (4) participants will be asked to endorse the final agreed tests of tools for measurement. All participants will be given the opportunity to ask questions and discuss any differences or similarities in opinion.

Workshops will be audio recorded. Data will be imported to the HyperRESEARCH software. These data will be qualitatively evaluated using thematic analysis to identify themes and recommendations. These results will be discussed among members of the steering committee.

A preliminary report will be drafted and disseminated to all workshop participants and relevant stakeholders and posted via the https://www.beatcf.org.au website to invite public comment for 2 weeks. The steering committee will review and endorse the final recommendations regarding consensus methods for measurement of core outcomes prior to dissemination of the results via peer-reviewed publication.

Ethics and dissemination

Ethics approval for this research has been granted by the Child and Adolescent Health Service Human Research Ethics Committee (RGS 4926). Deviations from this protocol will not occur without prior approval. All data will be reported in such a way that it will not be possible to identify study participants.

Members of the steering committee include a range of diverse representatives from around the world who are recognised contributors to the field of CF research and who are situated to promote implementation of COS-PEX in future studies. Results will be disseminated via consumer and research networks and peer-reviewed publications.

Development of COS-PEX is expected to improve the value of research in this field and minimise waste by ensuring that the outcomes evaluated and reported are meaningful to all relevant stakeholders, and most importantly people with CF. It is also expected that this work will improve the transparency of outcome reporting and assessment of studies, optimise research engagement of people affected by disease, improve the acceptance and translation of research findings and help facilitate comparison of data and synthesis of evidence across studies.