Durable usage of patient-reported outcome measures in clinical practice to monitor health-related quality of life in head and neck cancer patients.

PURPOSE: To investigate the long-term follow-up (5 years) of implementing patient-reported outcome measures (PROMs) in clinical practice to monitor health-related quality of life (HRQOL) in head and neck cancer (HNC) patients.
METHODS: A mixed method design was used. The usage rate of OncoQuest (a touch screen computer system to monitor HRQOL) and the subsequent nurse consultation was calculated among HNC patients who visited the outpatient clinic for regular follow-up, as well as differences between ever users and never users (sociodemographic and clinical characteristics). The content of the nurse consultation was investigated. Reasons for not using (barriers) or using (facilitators) OncoQuest and the nurse consultation were explored from the perspective of HNC patients, and of head and neck surgeons.
RESULTS: Usage rate of OncoQuest was 67% and of the nurse consultation 79%. Usage of OncoQuest was significantly related to tumor subsite and tumor stage. Topics most frequently (>40%) discussed during the nurse consultation were global quality of life (97%), head and neck cancer related symptoms (82%), other physical symptoms such as pain (61%), and psychological problems such as anxiety (44%). Several barriers and facilitators to implement PROMs in clinical practice were reported by both patients and head and neck surgeons.
CONCLUSION: Usage of PROMs in clinical practice and a nurse consultation is durable, even 5 years after the introduction. This study contributes to better insight into long-term follow-up of implementation, thereby guiding future research and projects that aim to implement PROMs in clinical practice to monitor HRQOL among (head and neck) cancer patients.

Introduction

In clinical practice, monitoring health-related quality of life (HRQOL) can be helpful to identify cancer patients with problems, concerns, and needs [1-3]. The use of patient-reported outcome measures (PROMs) to monitor HRQOL is feasible and acceptable [4]. Using PROMs allows patients to actively participate in their own care by providing the information they know best and helps clinicians to identify patients’ most bothersome issues [3, 5–7]. However, because monitoring alone does not improve patients’ HRQOL [8, 9], this needs to be followed by providing individual feedback to the patient and the healthcare provider (HCP) so they can discuss the need for supportive care [10-16]. In addition, collecting HRQOL data by means of PROMS in daily clinical practice is of high importance for research purposes also. PROMs provide data which are unique in detaining the patient’s own view on the impact of having cancer, and its treatment, on their HRQOL.

Based on the existing evidence, several evidence-based national guidelines recommend routine HRQOL assessment in clinical practice [e.g., 17–21]. Although the importance of monitoring HRQOL in clinical practice is clearly recognized, it remains difficult to implement PROMs in clinical practice. Previous studies showed that following the implementation, usage rates vary from 40 to 73% [22-24]. These results are based on short-term follow-up studies investigating the success of implementation. Knowledge on the long-term implementation is lacking [4, 7].

In 2006, we introduced a computer-assisted PROMs system (OncoQuest) in our clinical practice to monitor HRQOL among head and neck cancer (HNC) patients. Usage rate of OncoQuest in the first year after implementation was 67% [22]. To increase usage rate and the relevance of monitoring HRQOL for the individual patient, from 2008 onwards, we have been using OncoQuest in combination with a nurse consultation.

The overall aim of the present study was to investigate the long-term follow-up of OncoQuest and the nurse consultation, 5 years after the implementation. We investigated the current usage rate of OncoQuest and the nurse consultation among HNC patients, as well as differences between ever users and never users (sociodemographic and clinical characteristics) (aim A). Furthermore, we investigated the content of the nurse consultation (aim B). Also, we explored reasons for not using (barriers) or using (facilitators) OncoQuest and the nurse consultation from the perspective of HNC patients (aim C) and from the perspective of the head and neck surgeons (aim D). This study contributes to better insight into long-term follow-up of implementation.

Methods

A mixed methods design was used, with quantitative and qualitative research methods. The study was conducted at the Department of Otolaryngology – Head and Neck Surgery, VU University Medical Center in Amsterdam, The Netherlands, between April 2013 and September 2013. According to the Dutch Medical Research Involving Human Subjects Act, ethical approval was not necessary, because this study evaluated regular care and patients were not subjected to procedures or required to follow rules of behavior.

Description of OncoQuest and the nurse consultation

All HNC patients who are treated with curative intent are offered to use OncoQuest to monitor HRQOL via the EORTC QLQ-C30 and QLQ-H&N35 questionnaires and the Hospital Anxiety and Depression Scale (HADS) (in total 79 items), and to discuss the results by consulting an oncology nurse during follow-up visits after cancer treatment. Patients are also asked permission to use their HRQOL data in OncoQuest for research purposes. OncoQuest is available on dedicated computers with a touch screen in a separate consultation room. Due to logistic reasons, patients fill in OncoQuest after the follow-up visit with their surgeon. When a patient has completed OncoQuest, he or she can discuss the results with the nurse. The results of OncoQuest are available in real time in clear graphics on a computer screen by means of a stand-alone application (OncoQuest viewer), which is linked to the hospital information system. Based on the results, the nurse can identify and support HNC patients with problems regarding HRQOL or psychological distress. In this nurse consultation, the nurse provides information and advice, and, if needed, supportive care options are discussed. Usage is on a voluntary basis.

Aim A: usage rate of OncoQuest and nurse consultation, and differences between ever users and never users

To determine the usage rate of OncoQuest, the number of patients who ever used OncoQuest was divided by the number of all eligible HNC patients. The number of eligible HNC patients (denominator) was counted by reviewing data of patients (all subsites, all stages) who were treated with curative intent (all treatment modalities) at least 3 months earlier and who attended the outpatient clinic during the study period (April–September 2013) through the hospital information system. Patients’ usage data—if they used OncoQuest and the number of times they used OncoQuest—was consequently derived from the OncoQuest database. We defined patients as an ever user of OncoQuest if they had used OncoQuest during at least one follow-up visit. We did not limit the usage to the study period, since patients visiting the outpatient clinic are at different stages of their follow-up and could therefore have used OncoQuest in the past.

To determine the usage rate of the nurse consultation, the number of patients who used the nurse consultation during the study period was divided by the number of all patients who used OncoQuest during the study period (current users). We limited the usage of the nurse consultation to the study period, since information on usage of the nurse consultation was not collected earlier.

To gain insight into the characteristics of ever users, current users, and never users of OncoQuest and the nurse consultation, sociodemographic (age, sex) and clinical variables (tumor location, tumor stage, and type of treatment) were obtained from the medical records.

Aim B: content of the nurse consultation

To gain insight into the content of the nurse consultation following the use of OncoQuest, a researcher observed all nurse consultations during the study period with patients’ consent and filled out a study-specific report form. In this report form, the researcher noted the length of the consultation, the topics that were discussed, and the advice and supportive care options provided by the nurse.

Aim C: barriers and facilitators

To gain insight into barriers to use OncoQuest and the nurse consultation, a random sample of patients who did not use OncoQuest and the nurse consultation during the current follow-up visit (nonusers), were asked about their reasons. A researcher posted at the consultation room of one of the six head and neck surgeons (based on a circulation scheme). Patients who did not proceed to OncoQuest were approached by the researcher for a short structured interview with predefined answer categories. The researcher asked patients for their reasons for nonuse of OncoQuest and the nurse consultation, as well as for their suggestions on improvement. The researcher noted the answer as provided by the patient. In addition, the researcher noted the respondents’ sociodemographic variables sex and date of birth. Information regarding the number of times patients used OncoQuest ever before the current follow-up visit was derived from the OncoQuest database.

To gain insight into facilitators to use OncoQuest and the nurse consultation, we asked all patients who used OncoQuest and the nurse consultation during the study period (current users) to complete and return a study-specific paper-and-pencil questionnaire. The questionnaire included two questions on age and sex, and four items with predefined answer options (comparable to the questions asked in the interviews with nonusers) regarding reasons to use and added value of OncoQuest and the nurse consultation. Patients were able to choose multiple answers and were invited to elaborate by means of a free-text response. Information regarding the number of times patients had used OncoQuest before the current follow-up visit was derived from the OncoQuest database.

To evaluate patients’ satisfaction with the nurse consultation, the questionnaire contained three questions on satisfaction with this consultation. General satisfaction was measured with the Net Promotor Score (NPS) with the question “How likely is it that you would recommend the nurse’s consultation to other cancer patients (11-point rating scale: 0 (not likely) to 10 (very likely)). The NPS is calculated by subtracting the percentage of detractors (that score 0–6) from the percentage of promotors (that score 9–10). The percentages “passives” that score 7–8 is not included in calculating the NPS. The range of the NPS lies between −100 and +100. A positive score is considered good [25]. Additionally, patients were asked how satisfied they were with respect to the knowledge of the nurse regarding supportive care options, and with respect to her listening skills (11-point rating scales).

Aim D: exploring the perspective of the head and neck surgeons

Semistructured interviews were conducted with the head and neck surgeons involved in follow-up visits (N = 6). Interviews included questions concerning the added value and barriers of OncoQuest and the nurse consultation, and recommendations for improvement.

Data analyses

All statistical analyses were performed using the IBM Statistical Package for the Social Sciences (SPSS) version 22 (IBM Corp. Armonk, NY, USA). For aim A, descriptive statistics were used to describe sociodemographic and clinical characteristics of ever users and never users, as well as the number of times OncoQuest was used. We conducted chi-square tests, independent sample t-tests, and multiple logistic regression analyses to study demographic (age, sex) and clinical variables (tumor subsite, stage, and treatment modality) in relation to usage of OncoQuest and the nurse consultation. A multiple logistic regression model, with a stepwise forward selection procedure, was applied. Variables were added one by one to the multiple regression model, with p value for entry p < 0.1. p values <0.05 were considered statistically significant.

For aim B, descriptive statistics were used to describe the characteristics of the nurse’s consultation. For aim C, the answers provided by the patients during the interviews on nonusage were categorized into one of the predefined answer categories. If the answer did not match one of the predefined answer categories, these reasons for nonusage and suggestions for improvement were added. Descriptive statistics were used to summarize reasons, suggestions for improvement, and the number of times these patients had used OncoQuest in the past, and to summarize the facilitators identified in the questionnaire and patients’ satisfaction with the consultation.

For aim D, the semistructured interviews were analyzed by thematic analyses.

Results

Aim A: usage rate

During the study period of 6 months, 449 patients who visited the outpatient clinic for their follow-up visits fulfilled the inclusion and exclusion criteria (all eligible HNC patients). The characteristics of these patients are shown in Table 1. Of these patients, 303 were defined as an ever user of OncoQuest. To calculate the usage rate of OncoQuest, the number of OncoQuest ever users was divided by all eligible HNC patients (303/449). The usage rate of OncoQuest was 67%, meaning that 67% of the patients used OncoQuest at least once post-treatment. OncoQuest ever users used it between 1 and 11 times after treatment (median 2, interquartile range (IQR) 1–3).

Table 1

Characteristics of the total study population (N = 449) and differences between ever users and never users of OncoQuest

	Total, N = 449	Ever users, N = 303	Never users, N = 146
Sex (N, %)
Male Female	312137	205 (68%)98 (32%)	107 (73%)39 (27%)
Age in years
Mean (SD)	63.5 (10.4)	62.9 (10.3)	64.9 (10.5)
Tumor site (N, %)
Pharynx Larynx Oral cavity Other	16514111726	121 (40%)77 (25%)85 (28%)20 (7%)	44 (30%)64 (44%)32 (22%)6 (4%)
Tumor stage (N, %)a
1 2 3 4	1396554149	75 (27%)49 (17%)43 (15%)114 (41%)	64 (51%)16 (13%)11 (9%)35 (27%)
Type of treatment (N, %)b
Surgery (Chemo)radiation Surgery and (chemo)radiation	12823883	70 (23%)169 (56%)64 (21%)	58 (40%)69 (48%)19 (13%)

aData on tumor stage is missing in 42 patients

bData on type of treatment is missing in 2 patients

Multivariate analyses showed that tumor stage and subsite were significantly associated with the use of OncoQuest. Patients treated for a tumor stage >I used OncoQuest relatively more often, and patients treated for a tumor stage I used OncoQuest relatively less often. Patients with laryngeal cancer used OncoQuest relatively less often compared to other head and neck subsites (Table 2).

Table 2

Multiple logistic regression analysis of significant variables of usage of OncoQuest in the total study population (N = 407)

	OR	95% CI	p value
Tumor stage
Stage 1	1.0		p = 0.001
Stage 2	2.6	1.3–5.2
Stage 3	3.3	1.5–7.0
Stage 4	2.4	1.4–4.3
Tumor site
Larynx	1.0		p = 0.013
Pharynx	1.9	1.0–3.3
Oral cavity	2.1	1.2–3.7
Other	3.3	1.1–9.7

Age, sex, and treatment modality were not significantly associated with usage rate, and therefore not included in these analyses. Number of patients in these analyses is smaller (N = 407) than the total study population (N = 449) because of missing data on tumor stage

During the study period of 6 months, 147 patients used OncoQuest at the time of their follow-up visit (current users), of which 116 (out of 147) also used the nurse consultation (usage rate 79%) (Table 3). Age, sex, tumor subsite, tumor stage, and treatment modality were not significantly related to the use of the nurse consultation.

Table 3

Characteristics of current users (patients who used OncoQuest during the study period) (N = 147), and differences between users and nonusers of the nurse consultation

	Total, N = 147	Users, N = 116	Nonusers, N = 31
Sex (N, %)
Male Female	9552	76 (66%)40 (34%)	19 (61%)12 (39%)
Age in years
Mean (SD)	62.9 (10.4)	62.2 (10.2)	65.8 (11.1)
Tumor site (N, %)
Pharynx Larynx Oral cavity Other	53303232	40 (34%)22 (19%)28 (24%)26 (22%)	13 (42%)8 (26%)4 (13%)6 (19%)
Tumor stage (N, %)a
1 2 3 4	28322747	25 (23%)25 (23%)22 (21%)35 (33%)	3 (11%)7 (26%)5 (19%)12 (44%)
Type of treatment (N, %)
Surgery (Chemo)radiation Surgery and (chemo)radiation	346251	26 (22%)45 (39%)45 (39%)	8 (26%)17 (55%)6 (19%)

aData on tumor stage is missing in 13 patients

Reports of the nurse consultations were available for 113/116 patients. In three cases, there was no report due to logistic reasons. The consultations lasted 2 to 30 min (median 10 min, IQR 7–13 min). Topics most frequently (>40%) discussed during the consultation included global quality of life (97%); head and neck cancer related symptoms such as speech, swallowing, and oral dysfunction (82%); other physical symptoms such as fatigue and pain (61%); anxiety (44%); and depression (41%). The nurse provided practical advice in 23% of the consultations. This advice comprised information on how to cope with current physical or psychological sequelae of cancer and its treatment. In 11% of the consultations, patients received information on, e.g., the website of the Dutch Cancer Society, relevant books, or brochures. Also, 11% of patients were referred to peer support, including patient societies or a community center for cancer patients.

Aim C: barriers and facilitators to use OncoQuest and the nurse consultation

Barriers

In total, 84 HNC patients who did not use OncoQuest at the current follow-up visit were asked to participate in a short structured interview. Two patients did not want to participate, and one was mentally challenged and unable to answer questions. Ten patients indicated that they were not interested in OncoQuest with no further explanation. The majority of the remaining 71 patients were male (N = 52, 73%), and their mean age was 66 years old (range 42–87, SD 9.9). Of these 71 patients, 42 (59%) had used OncoQuest at, least one follow-up visit in the past (ever users), and 29 patients (41%) were never users. Most mentioned reasons for not using OncoQuest were lack of time (N = 29, 41%) and that patients did not have supportive care related questions (N = 14, 20%) (Table 4). Suggestions to improve OncoQuest that were mentioned during the interviews related mostly to the user-friendliness, e.g., tailor the questions to the individual patient and limit the number of questions. Patients also mentioned that it would be helpful to have a better explanation of OncoQuest at the start of treatment, and it would be nice to receive feedback during their consultation with the head and neck surgeon.

Table 4

Barriers for using OncoQuest (N = 71)

	Total (N = 71), N (%)	Ever usersa (N = 42), N (%)	Never usersb (N = 29), N (%)
Lack of time	29 (41)	15 (36)	14 (48)
No questions regarding supportive care	14 (20)	10 (24)	4 (14)
No change in symptoms	10 (14)	9 (21)	1 (3)
No need	10 (14)	4 (11)	6 (21)
No added value	7 (10)	5 (12)	2 (7)
Emotional problems	6 (8)	5 (12)	1 (3)
Participation in other scientific research	4 (6)	2 (5)	2 (7)
Filled out a similar questionnaire at another department	2 (3)	1 (2)	1 (3)
Parking fee is getting too high/ticket expires	2 (3)	2 (5)	–
Not wanting to use a computer	2 (3)	–	2 (7)
Miscellaneous	13 (18)	8 (19)	5 (17)

Seventy-one patients did not use OncoQuest at the current follow-up visit (multiple answers possible)

aForty-two patients who used OncoQuest at a previous follow-up visit, but not at the current follow-up visit

bTwenty-nine patients who did not use OncoQuest at the present or a previous follow-up visit

Facilitators

In total, 112 (out of 116) patients who used OncoQuest and the nurse consultation during the study period received a study-specific questionnaire, of which 83 completed the questionnaire. The majority was male (N = 57, 69%), and the mean age was 62 years old (SD 10.2, range 24–87). Most frequent reasons to use OncoQuest were that patients were asked by their HCP (N = 71, 86%), and that patients wanted to contribute to scientific research (N = 34, 41%) (Table 5). Most frequently mentioned reasons regarding the added value of OncoQuest were that changes in symptoms and HRQOL over time are monitored (N = 61, 73%), and that symptoms are adequately addressed (N = 39, 47%). Also, some mentioned that by using OncoQuest, they had a clearer overview of what they wanted to ask their HCP (N = 4, 5%). Free text responses yielded no additional reasons to use OncoQuest.

Table 5

Facilitators to use OncoQuest and the nurse consultation (N = 83)

	Number	Percent
OncoQuest
Being asked by staff	71	86
Contributing to scientific research	34	41
Monitoring changes in quality of life	6	7
Expecting to benefit from using OncoQuest	5	6
Nurse consultation
Being asked by staff	71	86
Appreciating the conversation with the nurse	21	25
Wanting to discuss results OncoQuest	10	12
Nurse has more time than physician	9	11
Wanting to share their story with the nurse	3	4
Having a question for the nurse	1	1

Eighty-three patients who did use OncoQuest and the nurse consultation at the current follow-up visit (multiple answers possible)

With respect to the nurse consultation, the most frequent reasons for usage was being asked to do so (N = 71, 86%) and the opportunity to talk with the nurse (N = 21, 25%) (Table 5). Free text responses yielded additional reasons including seeing the nurse’s consultation as an automatic consequence following the use of OncoQuest, and a personal and friendly way to discuss the results. As added value of the nurse consultation, patients mentioned the amount of time the consulting nurse had to discuss the OncoQuest results (N = 61, 74%), the personal conversation and advice (N = 29, 35%), and obtaining answers to questions regarding their disease, HRQOL, and symptoms (N = 13, 16%). Overall, patients were satisfied with the nurse consultation, with a Net Promotor Score of 25. The majority highly valued the listening skills of the nurse (11-point scale: median 9, IQR 8–10) and the knowledge of available supportive care (11-point scale: median 8, IQR 8–10).

Aim D: perspective of head and neck surgeons

All six interviewed head and neck surgeons stressed the importance of OncoQuest and the nurse consultation for patients during the follow-up period after treatment. Several themes were mentioned concerning the added value of OncoQuest in clinical practice (Table 6). Also, several barriers were mentioned, like a lack of feedback from the surgeon to the patient on the results of OncoQuest may lead to a decrease in patient’s motivation to use OncoQuest, and using OncoQuest takes too much time according to their patients. Finally, head and neck surgeons mentioned several improvements that could possibly eliminate these barriers, for example, the provision of feedback to the physician within the electronic patient file (instead of via a standalone application) and enhanced accessibility of OncoQuest for patients (e.g., ability for patients to access OncoQuest online from home) (Table 6).

Table 6

Facilitators, barriers, and suggestions for improvement according to head and neck surgeons (N = 6)

Interview topics	Themes
Facilitators of OncoQuest and the nurse consultation	- An enhanced insight into patients’ HRQOL- A signaling function for symptoms missed in follow-up consultations- Attention for a wide range of symptoms and concerns (including psychosocial issues)- The possibility to improve (timely) referral to supportive care tailored to individual patients’ needs- The opportunity to offer increased (profound) attention to patients- Offering patients the opportunity to have their story heard
Barriers of OncoQuest and the nurse consultation	- Not all patients have a need for OncoQuest (at each follow-up consultation)- The value of repeated use is not clear to patients- Using OncoQuest takes too much time according to their patients- The availability of OncoQuest outcomes during consultation is hampered due to difficulties to trace OncoQuest results at their computer screen- Patients use OncoQuest after the follow-up visit (due to logistic reasons)- A lack of feedback from the surgeon to the patient on the results of OncoQuest may lead to a decrease in patient’s motivation to use OncoQuest- Patients’ usage of OncoQuest after the follow-up visit to the surgeon (due to logistic reasons) leads to a lack of (accurate) feedback from the surgeon to the patient and to a possible delay in referral to supportive care- OncoQuest may address unsolvable problems (e.g. dry mouth, swallowing problems) possibly leading to a decrease in patients’ motivation to use OncoQuest- A lack of structural feedback to the surgeon from what is discussed during the nurse consultation
Suggestions for improvement of OncoQuest and the nurse consultation	- Provision of feedback to the physician within the electronic patient file (instead of via a standalone application)- The provision of extra reminders for physicians during consultation hours so they remember to discuss patients’ results- Enhanced accessibility of OncoQuest for patients (e.g., ability for patients to access OncoQuest online from home)

Discussion

The main aim of this study was to investigate the usage of PROMS in clinical practice through a computer-assisted PROMs system “OncoQuest” combined with a nurse consultation, 5 years after the introduction. Study results are discussed within the scope of the RE-AIM framework. The RE-AIM framework is designed to enhance the quality, speed, and public health impact of efforts to translate research into practice and can be used to study the reach, effectiveness, adoption, implementation, and maintenance of healthcare innovations [26].

In this study, we investigated the reach of OncoQuest and the nurse consultation by comparing ever users and never users. The usage rate of OncoQuest remained 67%, the same as at the time of the introduction [22]. This high usage rate, even in the long term, might be explained due to the fact that OncoQuest is combined with a nurse consultation. Among the patients who used OncoQuest, the majority also used the nurse consultation (79%). Other explanations based on our study data might be that the HCPs at the outpatient clinic personally invite patients to make use of OncoQuest and patients’ awareness of usage of OncoQuest data for research purposes also. This suggests that trust, therapeutic alliance, and altruism, as in helping other patients by participating in scientific research, might increase durable participation in monitoring HRQOL by means of PROMS in clinical practice. Usage rate of OncoQuest was significantly related to tumor stage. Patients treated for a tumor stage >I used OncoQuest relatively more often and patients with a tumor stage I used OncoQuest relatively less often. It may be that the never users have a lower need for supportive care and therefore are not likely to use OncoQuest and the nurse consultation.

Although we did not perform a randomized controlled trial on the effectiveness, study results showed that current users were very satisfied and most would recommend OncoQuest and the nurse consultation. HRQOL following cancer treatment was monitored via OncoQuest, and important issues to patients were discussed with the nurse. Patients felt that the nurse truly listened to their concerns and indicated to have faith in the referrals and advice given. These findings confirm earlier studies on the value of using PROMs in clinical practice to improve communication between patients and HCPs [4, 6], to identify cancer patients’ most bothersome issues [3], to monitor the course of symptoms as swallowing problems [27, 28], and to facilitate screening for psychological distress [2].

With respect to the adoption by care providers, all head and neck surgeons were satisfied with the use of PROMs in clinical practice and the nurse consultation. This high adoption rate by surgeons was important also for patients since our study results showed that one of the facilitators for patients to use OncoQuest and the nurse consultation was to be referred by their HCP. Adoption by HCPs is a key issue in implementing new interventions in cancer care [4, 29].

In this study, we explored barriers and facilitators of using OncoQuest and the nurse consultation. Finding the optimal way to implement PROMs in clinical practice is difficult and often creates logistical challenges [15, 30, 31]. In line with previous studies [15], patients in our study mentioned several barriers, such as no change in symptoms since the previous follow-up visit, and no need for supportive care. To overcome these barriers, we decided 5 years ago that using OncoQuest and the nurse consultation is on a voluntary basis, so that patients can decide themselves whether or not to use this service. This decision in favor of user-friendliness, does, however, hamper us to use the HRQOL data for research purposes.

Finally, this study provided insight into the maintenance of OncoQuest and the nurse consultation among HNC patients. Although the results are positive and usage rate was high, maintenance remains a challenge and needs regular reevaluation. For instance, in our hospital, a new electronic hospital information system was introduced last year, and we are still busy incorporating OncoQuest into this new system.

A strength of this study is that we involved all stakeholders to study the long-term follow-up of OncoQuest and nurse consultation. A weakness is that we did not use the RE-AIM framework upfront to define outcome measures as well as process indicators to study the long-term implementation. In addition, there is a potential bias in comparing the usage rate of OncoQuest as determined in the current study, compared to the usage rate as determined in the first year after implementation of OncoQuest, resulting from differences in study cohorts. Future research is needed to obtain more insights into the uptake of supportive care services after using OncoQuest and the nurse consultation. The likelihood of supportive care uptake by patients is increased when there is a good relationship between the patient and the team of HCPs [32]. Implementing PROMs in clinical practice facilitates this good relationship. OncoQuest is now used in follow-up care and not yet in patients during treatment. The reason is that patients during treatment already have frequent contact with their HCPs. However, an adapted version of OncoQuest specifically targeting patients during treatment may have added value as well, but further research is needed to explore the needs of both patients and HCPs for OncoQuest during treatment. Another area of further research is to provide patients the opportunity to fill out PROMs at home before their follow-up visit. Via an online system, the uptake of using PROMs may be increased. The online use of PROMs at home seems to be feasible for a large percentage of cancer patients [33, 34]. This could lead to a better preparation for clinical consultations and a better adherence to medical advice [35], possibly including a better uptake of supportive care services.

Conclusion

The majority of HNC patients use PROMs and a nurse consultation in clinical practice, also in the long term. This study contributes to better insight into durable usage of PROMs, thereby guiding future research and other projects that aim to implement PROMs in clinical practice to monitor HRQOL among (head and neck) cancer patients.