Acceptability of patient reported outcome measures (PROMs) in a cystic fibrosis data registry.

INTRODUCTION: Improvements in the treatment of cystic fibrosis (CF) have resulted in longer survival and an increased focus on optimising daily functioning with the condition. Patient-reported outcome measures (PROMs) are valuable tools in evaluating the health-related quality of life of persons with chronic diseases. PROMs may be incorporated into clinical registries to assess and provide feedback regarding the health-related quality of life of the affected population. This study uses qualitative methodology to describe the views of patients with CF, caregivers and clinicians on the usefulness and practicality of incorporating a PROM in the Australian Cystic Fibrosis Data Registry (ACFDR).
METHODS: We conducted semistructured interviews with a convenience sample of patients with CF (n=5), caregivers (n=7) and clinicians (n=13) on their opinions on incorporating the Cystic Fibrosis Questionnaire-Revised or the Cystic Fibrosis Quality of Life Questionnaire into the ACFDR. We analysed data into topics and subtopics using conventional content analysis.
RESULTS: Participants believed that PROMs could generate useful aggregate health-related quality of life data to support better understanding of the experiences of the modern CF population. Participants emphasised that implementation must be supported by processes to feedback data to patients and clinicians. Most participants preferred electronic PROMs administration for easy integration into existing systems and the potential to support feedback.
CONCLUSION: Patients, caregivers and clinicians in this study generally supported the usefulness and practicality of PROM implementation in the ACFDR. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

What are the perspectives of patients with cystic fibrosis (CF), caregivers and clinicians towards incorporating a patient-reported outcome measure (PROM) in a clinical registry?

Participants believe PROMs’ inclusion in the CF registry will be useful and practicable, however emphasise the need for data feedback and follow-up systems.

This study provides insight into the views of key stakeholders on the incorporation of a PROM in a chronic disease registry.

Introduction

Cystic fibrosis (CF) is a life-shortening, genetic disorder primarily characterised by progressive pulmonary disease. It causes a range of complications including diabetes, gastrointestinal malabsorption and infertility.1 2 CF is managed through comprehensive medical check-ups in multidisciplinary clinics, which include pulmonologists, dieticians, social workers, physiotherapists and psychologists. Routine visits occur every 3–6 months for adults.3 Currently, over 3400 Australians live with CF and 53.8% are adults.4

Although CF remains incurable, recent advancements in treatment have dramatically improved the life expectancy and daily functioning of people living with CF.1 4 In response, researchers have increased their focus on the social and psychological impacts of the disease through measurement of health-related quality of life (HRQOL).5 HRQOL encompasses the domains of psychological health, social networks and relationships, physical health and functional capacity.5

Patient-reported outcome measures (PROMs) are structured questionnaires completed by patients or a proxy which provide a standardised method of evaluating HRQOL.6 In CF, PROMs are used as an outcome measure in clinical trials, in observational research and in the clinical setting to aid communication and decision-making.2 In 2016, the International Committee for Mental Health in CF recommended annual screening of depression and anxiety in patients with CF and caregivers using two mental health PROMs.7

PROM incorporation in non-CF clinical registries has facilitated HRQOL research, informed quality improvement8 and, where patients can access PROMs data, empowered patients’ management of their health.9 The UK CF Registry has recently incorporated a CF-specific PROM, the Cystic Fibrosis Questionnaire-Revised (CFQ-R) into their data collection to evaluate the HRQOL benefit of genetic therapies.10 The UK CF Registry will develop an online portal ‘My CF Registry’ which will allow patients to directly enter and access their PROM data.11 The Cystic Fibrosis Foundation patient registry in the US uses mental health PROMs in some of its CF clinics to report the prevalence of depression and anxiety in their patient populations.12

The Australian Cystic Fibrosis Data Registry (ACFDR) collects clinical and demographic information from Australian adults and children with CF on an opt-in basis.13 Data collected includes physical health measures, treatments, hospitalisations and social functioning.4 Patients complete surveys annually on paper during clinic visits which are entered into a centralised database by clinic administrative staff. The data are centrally collated and cleaned by Monash University. ACFDR data are used in research, benchmarking14 and clinicians can access data on their patients to evaluate patient progress and communicate information.13 The ACFDR is considering implementing a PROM which patients would complete alongside other ACFDR surveys at clinic visits. This would provide aggregate HRQOL data, be used in research, and enable clinicians to access patients’ individual HRQOL data to monitor or discuss during the clinical consult.

We conducted a systematic review of CF PROMs evaluating HRQOL to assess suitability for the ACFDR.15 A search of six databases for studies published between February 2009 and February 2019 describing PROMs used for CF adults and children yielded 27 PROMs. The results are detailed elsewhere.15 We found the CFQ-R and Cystic Fibrosis Quality of Life Questionnaire (CFQoL) provided comprehensive assessment of defined domains of HRQOL and evidence of robust psychometric properties in English-speaking populations.15

The CFQ-R is a validated CF-specific PROM with adult, adolescent and child versions. The adult version includes 48 questions around 11 domains: physical functioning, vitality, emotion, social, role/school, body image, treatment burden, health perceptions, weight, respiratory and digestion.16 CFQoL is another widely used and well-validated CF-specific PROM with only an adult version. It includes 52 questions and 9 subscales: physical, social, treatment, emotional, relationships, career, future, chest symptoms, body image. Both PROMs are reported to require approximately 10 min to complete.17 18

We wished to understand the views of key stakeholders on national PROM administration through the ACFDR. Therefore, we conducted a qualitative study with the aim of assessing the perspectives of patients with CF, caregivers and clinicians on the usefulness and practicality of incorporating a CF-specific PROM into the ACFDR.

Methods

Study design

This study used qualitative descriptive methodology to summarise data with minimal interpretation.19 We proposed a sample size of 10–15 each of clinicians, patients and caregivers.20 Clinicians were defined as respiratory specialists or allied health clinicians who worked primarily with patients with CF in an inpatient or outpatient setting. Patients were defined as individuals of any age who had been diagnosed with CF. Caregivers were defined as a primary caregiver for a child under 16 years who had been diagnosed with CF.

Patient and public involvement

We conducted semistructured telephone interviews with patients, caregivers and clinicians. A convenience sample of patients with CF and caregivers were recruited through advertisements posted on the Australian Cystic Fibrosis website and Facebook group. The advertisement included a short explanation of the project and a monetary incentive of a US$20 gift card was offered. Snowball sampling methods were used to recruit CF clinicians with an email sent to clinicians involved in the ACFDR with an explanation of the project and instructions to forward the email to anyone who would be interested. All clinicians who expressed interest in participating during the recruitment period were included in phone interviews.

Interested participants contacted researchers through email or telephone. All participants were sent an explanatory statement and consent form. Where written consent could not be obtained, verbal consent was recorded before the interview. Participants were also emailed adult versions of the CFQ-R and CFQoL, with instructions to read or complete the two questionnaires, taking note of any shortcomings or areas that were particularly pertinent.

Results of this study will be sent to participants on request.

Interviews

Researchers developed semistructured interview guides which used open-ended questions to assess participant views on the usefulness of incorporating the CFQ-R or the CFQoL within the clinical registry and the preferred method and frequency of administration for adults and children. We also explored the clarity, relevance and suitability for the ACFDR of the CFQ-R and the CFQoL, which will be described in a separate paper. The interview guide has been included as online supplemental material. To develop the interview guide, researchers drew on issues in PROM administration in CF identified through the prior systematic review, including time burden on participants, frequency of administration required to capture an accurate result, and reliability of parent proxy reporting.15 Telephone interviews were conducted between June and August 2019 by RR and IR. RR is a trained qualitive researcher who has conducted prior research in PROMs and manages the ACFDR. IR is an Honours student who was trained in qualitative research methods for this study. The interviewers had no relationship with participants. Interviews ranged between 16 and 40 min, with an average of 23.5 min.

Analysis

Using NVivo software, we analysed data in three phases using conventional content analysis.21 One researcher began by coding phrases in the transcripts under key descriptive terms to create initial codes. These were combined into larger categories, which largely followed the interview guide; however, some novel categories emerged directly from the data. These categories were combined to form topics and subtopics, which have been defined and presented.

To ensure coding rigour, a second researcher individually coded 10% of transcripts. Discrepancies between the two coders were resolved with discussion. Data saturation was determined when no new information was generated from successive interviews. Data saturation was considered for patients and caregivers as one group, and clinicians as another group. Two further interviews were conducted after data saturation was determined to be reached in each group.

Results

The final number of participants included 5 adult patients, 7 caregivers and 13 clinicians. Adult participants included 3 women and 2 men aged between 21 and 35 years old. Parent caregivers were all mothers, with children with CF aged between 5 months to 14 years old. Table 1 describes clinician roles and practice settings.

Table 1

Clinician characteristics

Clinician	Role	Patient age group	Practice setting
Clinician 1	Nurse	Paediatric	Both
Clinician 2	Doctor	Adult	Both
Clinician 3	Psychologist	Paediatric	Inpatient
Clinician 4	Physiotherapist	Adult	Outpatient
Clinician 5	Research coordinator	Paediatric	Unknown
Clinician 6	Psychologist	Adult	Both
Clinician 7	Nurse	Adult	Both
Clinician 8	Clinic coordinator	Adult	Both
Clinician 9	Nurse	Paediatric	Outpatient
Clinician 10	Doctor	Adult	Both
Clinician 11	Clinic coordinator	Paediatric	Outpatient
Clinician 12	Doctor	Paediatric	Both
Clinician 13	Doctor	Paediatric	Both

Major topics and subtopics which emerged from the data analysis are summarised in table 2 with illustrative quotes.

Table 2

Summary of major topics and exemplary quotes

Major topic	Subtopics	Exemplary quote
Usefulness of HRQOL information	Benefits of capturing population level HRQOL data	“It would allow more sort of psychosocial projects to happen because we’ve got a better overall view of what’s impacted people. And at the moment we don’t really have that data stored at all. And also maybe correlations between the hard numbers and the way people are feeling.” (Caregiver 6)
	Benefits to individual care	“Even if its like a standard ‘you fill out how you’re feeling’ [questionnaire] it can really help I think” (Patient 1)
	Necessity of follow-up processes	“Following up with a phone call or an SMS the CF clinic coordinator or the doctor or whoever just say hey just gone through the results of that questionnaire you did yesterday or two days ago in clinic a couple of you know red flags for me. How are you going.” (Patient 4)
Practicality of PROM implementation	Preferred method of administration	“I’m always happy to do it electronically.” (Caregiver 4)
	Preferred frequency of administration	“I think it’s a bit of a balance between overloading them with too much to do but also the relevance” (Clinician 3)
	Considerations surrounding proxy reporting	“You don’t truly know what your child is thinking and I guess I’ve seen situations where parents have come and said ‘My child is really sad or lethargic’ and then the child will say ‘I’m really not’” (Clinician 9)

HRQoL, health-related quality of life; PROMs, patient-reported outcome measures.

Usefulness of HRQOL information

Benefits of capturing population level HRQOL data

Participants from all groups identified that the ability to collate aggregate HRQOL data and determine trends in patient experiences would be a major benefit of PROM implementation in the ACFDR. Patients and caregivers believed learning about others’ CF experience would normalise their own and enable them to feel connected.

“[It] would probably give other CFers a gauge of how we’re all feeling and whether or not we’re alone in the ways we deal.” [Patient 5].

All groups identified increased insight for CF clinicians on the priorities of patients with CF and families as another benefit of establishing aggregate HRQOL data. Patients and caregivers described discrepancies between clinicians’ understanding of their needs and their actual concerns.

“Sometimes [clinicians] don’t realize the biggest things that are affecting us.” [Caregiver 5].

Patients and clinicians expressed that generating population-level HRQOL data for Australians with CF will give valuable information on living with CF. Some clinicians felt that aggregate PROM data has the potential to modify provision of care and influence psychosocial research in CF.

“I guess [a reason] for collecting this patient reported data is it gives us an idea from the patients themselves about the importance and prevalence of problems as they see them. So I think that that would provide some very useful data to how we tweak our services and research.” [Clinician 13]

Benefits and challenges of PROMs in patient care

Currently, some clinics use longitudinal ACFDR physical health data for individual patients to monitor patient progress and support communication. Participants were divided on this use for PROMs data. Patients believed that completing PROMs may enable recognition of challenges or aspects of the disease that they may otherwise not have noted.

“[PROM completion] is reflective I guess in terms of realising the impact of the disease and the potential progression and impact on his life and our family life” [Caregiver 3]

Patients and caregivers believed that PROMs could provide reminders during consults. Some adult patients also felt that the questionnaires would enable a safe way for them to initiate uncomfortable topics.

“It’s awkward like bringing these things up and discussing them. But… just filling out a questionnaire then it takes away that kind of awkwardness and… the social worker or doctor whoever they can get all the information.” [Patient 2]

Similarly, some clinicians felt that responses to PROMs can identify new information about patients. One clinician described this when using CFQ-R as a screening tool in their outpatient clinic.

“Surveys pick up areas that we may not necessarily pick up in a clinic. So, we have a lot of patients that report that everything’s fine and that all [is] good with treatments and stuff. But the CFQ-R will show that they actually have a high treatment burden.” [Clinician 11]

Clinicians envisioned this information would enable them to link patients and families with the appropriate services:

“Depending on where things fall, we can go okay something actually needs to be addressed there. So then you can have a clinical pathway referral, social work referrals and mental health care plan something like that.” [Clinician 4]

However, some participants from each group felt that communication during consultations was already effective and clinicians understood problems affecting patients and families. Some clinicians and patients also identified that these questionnaires could consume valuable time during the consultation.

“Their quality of life is not necessarily measured by any instrument. It’s measured by the things that they want to do… I find it much better to talk to them about what they’re doing in their life, what they want to achieve.” [Clinician 5]

Necessity of follow-up processes

Many participants from all groups expressed that for PROMs to be useful at a population or individual level, effective follow-up systems must be in place to report aggregate data back to patients and clinicians and to identify red flags. It was important to some patients and caregivers that they could see the outcome of collecting the data.

“I’ve often wondered… what happened with this data… I’ve never had anyone in clinic come back to me and say you know the results were used to go into this study.” [Patient 4]

Clinicians frequently emphasised their perceived duty of care to follow-up on concerning responses identified in PROMs.

“[Do we] just enter the data and ignore the messages that are in it or do we have a threshold for where we really feel clinically responsible for some of the answers that are there.” [Clinician 5]

Clinicians identified several possible follow-up processes. Some suggested the ACFDR could create a system to notify red flags. Others indicated clinic staff could be assigned to monitor patient PROM responses. When asked whether their clinics would have the resources to follow-up on concerning patient responses and provide adequate support, clinicians were divided. Some clinicians were confident they could refer patients to psychologists or social workers. Others described they would have limited access to these resources, demonstrating a significant barrier to PROM implementation.

“In terms of resourcing that’s one of the gaps in our service.” [Clinician 8]

Several clinicians and caregivers felt it was essential to make sure patients and families were able to discuss their emotional reaction to the PROM following completion. This was particularly important for newly diagnosed children and families who may learn aspects of their condition through completing PROMs.

“It might upset the parents of newly diagnosed kids.” [Caregiver 3]

Clinicians suggested administering PROMs in clinic would be essential to ensure that patients and families are supported during PROM completion.

“Making sure they’ve got an appropriate appointment afterwards to discuss what those questions actually mean for them” [Clinician 10]

Practicality of PROM implementation

Preferred method of administration

Most participants from all groups preferred electronic administration. They identified the major advantage of electronic administration over paper is avoiding the need for data entry. Clinicians suggested methods for electronic implementation including using apps on patient phones, links on email or computers in clinic. Clinicians envisioned data could be sent directly to them and imported into the registry.

“An Ipad… in the clinic that we can use [with an] electronic submission form… and the submit button that goes directly through a central database and that bypass[es] the need of having a lot of paperwork collected” [Clinician 2]

Adult patients and caregivers identified that sending an SMS or email reminder would assist with electronic administration.

“If you email it to me if I had time to spare at clinic I could just pull it up on my phone… sending out a reminder might help” [Patient 3]

A barrier to electronic administration identified by clinicians was the difference in electronic devices available to clinics. Some clinics had access to iPads, but others did not have computers.

“Having the resources to do it electronically would be a challenge” [Clinician 1]

Preferred frequency of administration

Participants expressed a range of opinions on the most effective frequency of administration. Many participants noted that CF can be affected by external factors such as the season or school holidays. Patients’ perception of their disease may even change arbitrarily based on mood. Therefore, patients and caregivers generally preferred frequent administration twice to four times a year to limit the effect of external factors on the accuracy of HRQOL data captured.

“Once a year might not give a clear snapshot because the once you did… was a time that environmentally no one was unwell… Whereas if you did it four times a year… It would definitely give you more of an overview.” [Caregiver 5]

Conversely, clinicians generally felt it was not practical to collect data more than yearly and favoured less frequent administration; yearly, 2 yearly or at important stages in life. They believed that frequent administration could be impractical and onerous for patients and families. Annual administration was suggested by some participants as a practical compromise, with the annual review clinic proposed as an opportune time to complete the questionnaire.

“I think on a on a practical level it will be difficult to collect this data more than once a year.” [Clinician 13]

Considerations surrounding proxy reporting

Participant groups also diverged in their opinions on the effectiveness of proxy reporting by caregivers. Most parent caregivers were confident in their ability to complete the PROM accurately on behalf of their children. However, adult patients generally felt there would be a discrepancy between parent and child views, especially the emotional aspects.

“I probably keep things bottled up a lot. I know I probably wouldn’t let my parents know what I’m going through.” [Patient 5]

Participants from all groups agreed that the ability for parents to accurately represent children’s concerns depended on the parents’ relationship with their child, the sickness of the child and the family understanding of CF. Participants stated that children would vary in their ability to complete a PROM, depending on their concentration and literacy. Children’s lack of awareness of their condition was raised as a major barrier to answering questions themselves. Some caregivers and one paediatric doctor were concerned about how PROMs would affect children.

“There are those [children] that would be really negatively affected by that because they’ve been very well their entire lives and now you’re giving them a questionnaire that might suggest that they shouldn’t be so well.” [Clinician 12]

One caregiver and one clinician emphasised the importance of having the parent look at the PROM and decide whether the child was able to complete it.

“The parents are the ones that know their children well enough to decide. Well, I think you’re old enough to do this yourself or I still need to do this for you because you’re not quite capable.” [Caregiver 7]

Discussion

This is the first Australian qualitative study assessing perspectives of patients with CF, caregivers and clinicians towards PROMs incorporation in a CF registry. PROMs in CF registries have previously been recognised as a potential facilitator of longitudinal research.22 Non-CF registries found PROMs highly beneficial to registry-related research,23 where PROMs have been used to demonstrate trends in HRQOL,24 in comparative effectiveness research, and to develop algorithms to predict outcomes of treatment.23 To ensure PROMs generate useful data, PROM users advise widespread dissemination of PROMs data,25 26 encouraging stakeholder feedback,25 and undertaking routine monitoring of the quality of data and how it is used.27

ACFDR results are currently used by some clinicians to monitor patient progress or to aid discussion during the clinic visit. Participants in this study were divided on this use for PROMs. The non-CF PROMs literature describes that PROMs can enhance individual care by enabling longitudinal assessment of changes in health, focusing clinician’s attention on issues most concerning to patients,24 28 29 and supporting patients to better describe their problems.12 24 Barriers identified by participants and in the literature include adding time to the consultation, taking time away from more relevant aspects of care,30 and intruding on the clinician-patient relationship.27

Some clinicians felt they were obligated to follow-up concerning results or discuss emotional responses to PROMs. Patients felt that discussing PROMs during consults would make PROMs more meaningful to them. Other qualitative studies of PROM users have identified that using PROMs in individual care is vital for patient compliance.23 31 Real time reporting of clinically meaningful PROMs data to clinicians has been identified by experts as ‘difficult’ to implement,32 but essential for up-to-date PROMs data to be used during the consultation.23 30 Strategies to follow-up PROMs data include generating reports which demonstrate trends and highlights areas of concern,13 33 which can be emailed to patients and clinicians or shared via a dashboard.32 In addition to reporting, PROM experts also advise developing decision-support protocols for concerning PROM results, based on the resources available in the clinic.34

Most participants believed electronic PROM administration would be efficient and easy to integrate with ACFDR data. Non-CF PROMs users have found that electronic administration simplifies data collection12 and facilitates real time analysis and feedback.12 13 31 The ACFDR currently uses the Research Electronic Data Capture (REDCap) system, which other registries have used effectively to collect PROMs.23 35 Benefits of using REDCap include automatic reminders to patient emails, secure data storage and inability to submit forms with missing data.35 This study identified that a barrier to electronic PROM administration will be the varying electronic data entry systems available in clinics. Patients’ own mobile phones may be a common resource to all clinics, as patients with CF and families are younger and likely to own a mobile phone.

Patients and caregivers described that HRQOL can change frequently in CF, therefore were generally willing to complete PROMs more frequently, the majority choosing once or twice a year, but some up to six times. However, clinicians were concerned frequent administration would be administratively impractical and burdensome for patients. This conflict between burden and accuracy has been described by PROMs users in non-CF contexts.27 Most participants in this study selected annual administration as the least burdensome frequency that could still demonstrate population level trends or be used in individual clinical care. However, further research is required to assess whether more frequent PROM administration will avoid external factors affecting the reliability of longitudinal PROMs data.

In this study, participants differed on whether parents could accurately represent the experiences of their children when reporting HRQOL. Evaluations of the accuracy of parent proxy compared with child PROM completion in CF have shown considerable variation.36 37 Parent proxy reporting in CF is more reliable for observable symptoms and for younger children.38–40 The Federal Drug Administration has recommended that children complete PROMs whenever possible41 and parents complete observer-reported outcomes which only assess observable behaviours or signs.41 In a scoping review to the UK National Health Service, Morris et al42 proposed that generally children could self-report without assistance over 8 years old and with assistance between five to 8 years old. Participants in this study recognised the variability in children’s ability to report for themselves would depend not only on age, but also literacy, concentration, and understanding of CF.

Limitations

The CFQ-R and CFQoL are both limited by their length and age over 20 years, which may limit their acceptability and relevance to the modern CF population.15 This was addressed in interview questions and participants found the PROMs comprehensive, clear, of acceptable length and generally applicable to their experience. These results will be reported in detail in another paper.

The sample size of participants, in particular patients, was smaller than initially proposed. A small group of young adults with CF is included in the study with no children, teenagers or older patients included. Patients with CF were less responsive to recruitment than caregivers and clinicians and those who responded were unavailable for interviews during the interview period. Data saturation was achieved when grouping patient and caregiver groups together, however would not be achieved for the patient group alone. Future studies in this area would benefit from a longer recruitment and interview period, which would enable inclusion of a larger and more diverse population and provide a wider perspective toward PROM acceptability.

Participants self-selected for the study based on an advertisement or email about PROMs. This may have captured a non-representative sample of patients with CF and caregivers who are involved in research and more willing to complete a questionnaire, and CF clinicians who are enthusiastic about PROMs. The diversity of opinion in CF may be better captured by quantitative surveys on the efficacy of PROMs conducted on a larger, representative sample. During analysis, a second researcher used a guide to code transcripts which were then checked, rather than generating codes independently.

Conclusion

Participants supported PROM incorporation in the ACFDR for the potential to provide useful aggregate HRQOL data, facilitate research and useful in individual care. However, participants emphasised that implementation must be supported by processes to feedback data to patients and clinicians, follow-up on red flags and opportunities to discuss potential areas of concern which arise following PROM completion. Most participants preferred electronic administration for easy integration with existing systems and potential to support feedback. This study will be followed by a single-centre pilot study of PROM administration.