Discrete choice experiment to evaluate preferences of patients with cystic fibrosis among alternative treatment-related health outcomes: a protocol.

INTRODUCTION: Clinical decision-making is a complex process. Patient preference information regarding desirable health states should inform treatment and is critical to agreeing on goals of therapy. Cystic fibrosis (CF) is a common, inheritable multisystem disorder for which the major manifestation is progressive, chronic lung disease. Intermittent pulmonary exacerbations are a hallmark of disease and these drive lung damage that results in premature death. We suspect that clinicians make assumptions, most likely implicit assumptions, about outcomes that are desired by patients who are treated for pulmonary exacerbations. The aim of this study is to identify and quantify the preferences of patients with cystic fibrosis regarding treatment outcomes. METHODS AND ANALYSIS: We will develop a discrete choice experiment (DCE) in collaboration with people with CF and their carers, and evaluate how patients make trade-offs between different aspects of health-related status when considering treatment options. ETHICS AND DISSEMINATION: Ethics approval for all aspects of this study was granted by the Western Australia Child and Adolescent Health Service Human Research Ethics Committee [RGS903]. Weighted preference information from the DCE will be used to develop a multiattribute utility instrument as a measure of treatment success in the upcoming Bayesian Evidence-Adaptive Trial to optimise management of CF. Dissemination of results will also occur through peer-reviewed publications and presentations to relevant stakeholders and research networks. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This will be the first discrete choice experiment (DCE) conducted in patients with cystic fibrosis (CF) to examine preferred health outcome states.

Attributes selected for inclusion in the DCE will be determined in consultation with people affected by CF (patients and carers).

The DCE presents individuals with hypothetical choice tasks, asking patients to choose between different treatment options and alternative health states, and in doing so make trade-offs between different aspects of health-related status.

Recruitment for this study will predominantly occur within Australia, which may limit the generalisability of findings to other CF populations.

Weighted preference information is intended to be incorporated into a multiattribute utility instrument for use as a measure of treatment success in trials of treatments for CF pulmonary exacerbations.

Introduction

Medical decision-making is a complex process. In the clinical setting, this should be a shared, iterative process between clinicians and patients (and their carers if appropriate). Each group brings differing needs and perspectives. Understanding patient preferences regarding health outcomes is critical to informing treatment choices and agreeing to goals of therapy.1 In addition to being desired by patients, these goals must also be considered achievable by clinicians.

Cystic fibrosis (CF) occurs in 1:2000 to 1:3500 births and is an inheritable multisystem disorder for which the major manifestation is progressive, chronic lung disease.2 Survival improved dramatically during the latter part of the 20th century but has more recently slowed with average survival approximately 50 years.3–5 The disease is characterised by intermittent pulmonary exacerbations which drive lung damage. Minimising the decline in lung function that accompanies pulmonary exacerbations (one in four patients do not recover their baseline function) is thought to be key to improving survival and quality of life.6 Management of pulmonary exacerbations generally involves a combination of antimicrobial, anti-inflammatory and mucolytic agents, physiotherapy and optimisation of nutrition.7–12 However, there is no consensus between centres regarding a standardised approach due to the paucity of evidence available to guide therapy.12 The James Lind Alliance, in partnership with healthcare providers and people with CF from 23 countries, has recognised treatment of pulmonary exacerbations as a research priority13 and specific knowledge gaps in this area have been recently identified.14

Determining the value of different treatment options depends on the value patients place on the consequences of each treatment decision. A variety of methods exist to elicit patient preference information. These include revealed preference and stated preference methods.15 Revealed preferences are based on observed choices made by individuals in real-life scenarios. Stated preferences are derived from decisions made by individuals when confronted with realistic, hypothetical choice scenarios, such as in a discrete choice experiment (DCE).16

DCE involves administration of a choice-based questionnaire that presents clinical vignettes and asks respondents to make trade-offs between different aspects of health-related status.17 The core theory informing DCE design is that the value of an option depends on the value of its attributes.16 18 Attributes are characteristics of treatments or services that may be processes (factors related to the delivery of care), structures (such as the setting in which healthcare occurs) and/or health outcomes,17 19 which are defined as the effects to health resulting from a treatment intervention.20

DCEs differ from other stated preference methods because, by assigning different levels to attributes (contrasting results for each characteristic), they force participants to weigh their relative importance (such as drug benefit vs toxicity), compared with other techniques which simply rank or rate them.16 DCEs can also capture some of the dependencies between attributes.21 These dependencies can be investigated more thoroughly when coupled with a causal model.22 The major limitation of DCEs is that to maximise statistical and response efficiency,23 the number of attributes selected for evaluation must be restricted, meaning the most important attributes must be known prior to construction. The range of attributes reported in the literature ranges from 2 to >10, with a mean of 5.

Guidance about the best approach for structuring DCE is provided by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR).23–25 Since DCE’s vary in terms of their objective(s), no single design will uniformly yield optimal results.23 Accordingly, DCEs will vary with respect to their statistical efficiency and response burden.26

This study presents a protocol for DCE designed to quantify how people with CF make trade-offs between different aspects of health status. In this case, attributes will be outcomes associated with treatment of pulmonary exacerbations that capture how a person feels, functions or survives, which are considered most important to people living with disease. Weighted patient preference information from the DCE will be incorporated into a multiattribute utility instrument (MAUI), which will generate a score as a measure of success in pulmonary exacerbation trials, including the planned Bayesian Evidence-Adaptive Trial to optimise management of CF (BEAT-CF). We expect recruitment for this study will largely occur within Australia, which may limit the generalisability of findings to other CF populations.

Aims

The aims of this study are (1) to identify and prioritise health outcomes of importance to people affected by CF, (2) to map these outcomes to consensus-derived causal models of CF pulmonary exacerbations and (3) to examine how patients make trade-offs between different aspects of health-related status when considering treatment decisions.

Methods and analysis

Overview of approach and consumer involvement

Consumer involvement is critical to this work which will comprise four stages (figure 1): (1) key health outcome elicitation and prioritisation from the perspective of people affected by CF, (2) mapping of these outcomes to a consensus-derived causal model of the disease processes, (3) selection of outcomes for inclusion in the DCE taking into consideration orthogonality and (4) development and administration of DCE to weigh the relative importance of outcomes from the perspective of patients. This study commenced in October 2018 and completion is expected in May 2020. Study progression at each stage is contingent on completion of the preceding research stage.

Figure 1

Research stages. CF, cystic fibrosis; DCE, discrete choice experiment.

Key health outcome elicitation by CF consumers

Elicitation of key health outcomes from consumers will occur using two methods: (1) preliminary consumer workshops and (2) online health outcomes surveys. Patient preference information is expected to vary between individuals but also according to age and stage of disease. To help elucidate these differences, and because young people may be less inclined to contribute in a group where older participants are present, workshops will be conducted separately for the following groups: young people with CF (13–25 years), adults with CF (>25 years) and persons who identify as carers for people with CF (including parents).

Workshops for patients will occur via teleconference, owing to infection control restrictions which preclude direct contact among this patient population. Carer workshops will be conducted in-person at the Telethon Kids Institute (Perth, Australia) with teleconference dial-in facilities available if requested.

Follow-up workshops

Outcomes identified through the consumer engagement activities detailed above will be collated with any additional potentially important health outcomes identified from review of the literature. Prioritisation of outcomes will occur during a series of follow-up workshops with each of the consumer groups. A combined workshop will also be conducted to derive a consensus list of prioritised outcomes relating to treatment of pulmonary exacerbations from the perspective of patients >13 years.

Consensus causal diagram

A consensus causal model (in the form of a Bayesian network) which links outcomes to causal disease processes for pulmonary exacerbations will be developed by a group of clinicians and other subject domain experts and people with lived experience of the disease. This process will be moderated by external facilitators using expert knowledge elicitation methods.27 28 The purpose of this is to guide selection of outcomes for inclusion in the DCE by choosing those that are likely to be important while minimising the inclusion of multiple attributes that measure the same outcome. The causal model will also aid in identifying probable combinations of attributes to ensure they are covered by the DCE, as well as helping to rule out improbable attribute combinations. Finally, the model will identify dependencies between attributes that need to be controlled for or otherwise handled during the analysis.

DCE design

The first step in designing the DCE is the identification of the important attributes (characteristics) for evaluation, and the assignment of possible levels to these attributes.16 29

Attributes and levels

Attributes and levels will be selected according to guidance provided by ISPOR.24 Only attributes that are identified as important to people with CF that map to causal disease pathways will be considered for inclusion. Levels (which may be categorical, continuous or probabilities) will be assigned in consultation with consumer representatives based on those that patients can relate to and consider meaningful which best represent the spectrum of possibilities that are clinically encountered.

Code to generate design

An experimental design will be constructed chiefly by RN in Ngene, software widely used in DCE development.23 The principles underpinning our design is that it will (1) consist of a pool of choice tasks, divided into blocks to which respondents will be randomly allocated, (2) maximise efficiency in terms of the precision of the coefficients (ie, D-efficiency)30 and (3) account for the ordered nature of the parameters under consideration by employing small non-zero priors in Ngene. As described below, the design may be updated following qualitative review of the initial design.

DCE questionnaire

The questionnaire will contain background information explaining the study rationale and potential risks and benefits of participating. Attributes and levels will be clearly defined. Sociodemographic data (age, sex, postcode) will be collected to assess if these factors influence stated preferences.

The draft DCE will be administered to a convenience sample of consumers (see figure 2 for DCE choice task example). If the tasks are too difficult or present implausible combinations of levels, we will define a candidate set of acceptable choice sets, and regenerate the design with a fixed amount of level overlap.31 32 Feedback about other design elements, including the length, layout, specific wording and comprehensibility will also be obtained. Suggestions for improvement will be considered and the final model agreed by consensus.

Figure 2

Discrete choice experiment choice task example. CF, cystic fibrosis

Sampling and recruitment strategy

There are 3422 people registered on the National CF database in Australia.33 Our research population comprises patients >13 years with CF and individuals who identify as carers for person(s) living with CF.

Recruitment from the sampling pool for stages 1 and 3 of this study will occur through a variety of means including through outpatient clinics and inpatient wards at Sir Charles Gairdner Hospital (adult tertiary hospital facility) and Perth Children’s Hospital (children’s tertiary hospital facility) and by advertising through consumer and research networks, including the Western Australia CF consumer reference group, CF Australia and CF Western Australia, including through electronic and social media bulletins and communiqués. Interested persons will contact a member of the study team by phone or email to register their interest. Patient information and consent forms will be provided for the workshop (electronically via email attachment or wet signature for in-person workshop attendees) and survey participants (online). Participants aged between 13 and 18 years will additionally require guardian consent. Links for the online CF-related health outcomes survey and DCE questionnaire will be sent via email once consent forms are received.

Workshops will proceed if two or more consumers register to attend. For the combined workshop, we aim to recruit a minimum of two young people and two adults with CF.

There is no consensus regarding DCE sample size requirements for applications in healthcare. ISPOR guidance remarks that statistical precision increases at sample sizes above 150 and levels out over 300 observations.23 Lancsar et al suggests a minimum of 20 observations per choice set is required to achieve a reliable model,18 while Marshall provides a rough rule of thumb based on the number of tasks, alternatives per choice set and levels.34 DCE will remain open until 200 responses are received and 4 months have elapsed since commencement. This target sample size represents a compromise between the desire for an accurate tool (one that reflects the average preferences for consumers) and the practical consideration that, at most, we aspire for roughly 1 in 10 patients to contribute from the sample pool of approximately 2000 people >13 years with CF.33

Given our recruitment strategy, participants are expected to predominantly reside in Australia, although it is possible that some participants living overseas may participate, depending on the reach of our consumer and research networks. As a robustness check, analyses will be conducted with and without any non-Australia-based respondents.

Participant reimbursement

Participants will not be paid to take part in any aspect of this study. Parking reimbursement for those who attend the in-person caregiver workshops will be provided.

Patient and public involvement

BEAT-CF will focus on evaluating optimal treatment(s) for pulmonary exacerbations, which has been identified by the James Lind Alliance as a research priority for people affected by CF.13 Consumer advocates have been involved in elements of trial design, and patients will be involved at all stages of the research process. Patients are not officially involved in participant recruitment, although promotion of research activities is expected to occur by word-of-mouth. Results will be disseminated to participants involved in this study and broadly via peer-reviewed presentations and by consumer research networks and CF advocacy organisations.

Data collection

Workshops will be approximately 2 hours in duration. Outcome elicitation (preliminary workshops) will occur using nominal group technique.35 Key aspects of this approach include clarification of the purpose of the session, allowing time for participants to formulate individual responses, and then asking participants to present one idea aloud, in turn to the group until saturation occurs, that is, until no new outcomes are identified.35 Results for these sessions will be collated on Excel spreadsheets and remain visible to participants throughout the session. Discussion of individual ideas will be permitted to allow clarification, rather than to resolve differences. A facilitator will ensure discussion is equally balanced among all ideas and between individuals. Prioritisation of outcomes (follow-up workshops) will occur by collating results from participants asked to rank outcomes at the follow-up workshops.

The online CF-related health outcomes survey will present consumers with the same two open-ended questions as those posed at the preliminary workshops (Appendix 1). This is being performed to ensure broad capture of CF-related health outcomes. The survey will be advertised and remain open for a 4-week period from commencement.

Data collection instruments and technologies

Workshops will be audio-recorded to enable playback, which is necessary to ensure the validity of data by minimising investigator recall bias.

CF-related health outcomes survey will be built using a REDCap online database, which will be hosted on a secure server at the Telethon Kids Institute. DCE will be built by a commercial provider. Both surveys will be conducted anonymously and will collect non-identifiable data only. Participants can exit from the online surveys at any time prior to submission of their responses. After this time, it will not be possible to withdraw their responses, as all items are non-identifiable.

Data processing

Workshop and causal diagram data files will be stored as password protected Excel or word documents. The non-identifiable CF-related health outcomes results data set will be downloaded from REDcap. The non-identifiable DCE data set will be sent as a password protected file by the commercial provider.

All data files will be stored securely on a password protected computer, which will be backed up on the Telethon Kids Institute server. Hard copy consent forms will be stored securely in a fireproof, locked filing cabinet at Telethon Kids Institute. The Institute is protected by high-level security and requires swipe card access for entry to the building and individual work areas. Data and research records will be retained for a minimum of 5 years after the date of last publication or until the youngest subject turns 25 years of age (whichever occurs later).

Data analysis

Analysis for the DCE will be performed in STATA V.13 using a range of regression approaches. For initial analysis, we will conduct a conditional logit. This will be used to understand the treatment preferences and trade-offs made by patients when considering outcomes relating to treatment of pulmonary exacerbations. For conditional logit analysis, the functional form is specified as:

which represents the utility of option j in choice set s for survey respondent i, where is a vector of dummy variables representing the levels of the health state presented in option j, β is a vector of utility weights associated with each level and is the error term.36

Second, we will use a mixed logit model to evaluate preference heterogeneity among respondents:

where β represents population mean preferences and n is the individual deviation around those mean preferences.36

Additionally, we will run exploratory analyses using a generalised multinomial logit, which considers both scale and preference heterogeneity.37 38 However, this will not be the prespecified primary outcome as there is concern about its ability to converge with a relatively small sample size. An exploratory analysis on DCE responses will also be conducted using causal Bayesian networks.39 Causal Bayesian networks are a generalisation of the path models of structural equation modelling,22 which have been recently applied in DCE analysis to provide greater insight into choice processes.21

Ethics and dissemination

Ethics approval for all aspects of this study was granted. Deviations from this protocol will not occur without prior approval. This study will be conducted in accordance with the International Council for Harmonisation of technical requirements for pharmaceuticals for human use (ICH) guidelines for Good Clinical Practice.40

Participant information sheets will be provided to workshop and survey participants. Asking consumers to consider health-related outcomes may result in distress. Participants will be warned about this risk, and patients will be recommended to contact their general practitioner, CF clinic or Lifeline if this occurs.

Data obtained from workshop sessions or survey responses will remain confidential. Data will be reported in such a way that it will not be possible to identify individuals or their contributions.

Dissemination will occur through peer-reviewed publications and presentations to relevant stakeholders and research networks. DCE results will be reported according to the Guidance for Reporting Involvement of Patients and the Public checklist.41 This is a consensus reference document agreed by international representatives, which provides guidance about how to report patient and public involvement in health-related and social research.