Feasibility of an eHealth application "OncoKompas" to improve personalized survivorship cancer care.

PURPOSE: The purpose of this study was to investigate the feasibility of an online self-management application (OncoKompas) among cancer survivors. In OncoKompas, cancer survivors can monitor their quality of life (QOL) via participant reported outcomes (PROs) ("Measure"), which is followed by automatically generated individually tailored feedback ("Learn") and personalized advice on supportive care services ("Act").
METHODS: A pretest-posttest design was used, conducting a survey before providing access to OncoKompas, and 2 weeks after, followed by an interview by a nurse. Adoption was defined as the percentage of cancer survivors that agreed to participate in the study and returned the T0 questionnaire. Implementation was defined as the percentage of participants that actually used OncoKompas as intended (T1). General satisfaction was assessed based on the mean score of three study-specific questions: (1) general impression of OncoKompas, (2) the user-friendliness, and (3) the ability to use OncoKompas without assistance (10-point Likert scales). Furthermore, satisfaction was measured with the Net Promotor Scale (NPS).
RESULTS: OncoKompas was feasible with an adoption grade of 64 %, an implementation grade of 75-91 %, a mean satisfaction score of 7.3, and a positive NPS (1.9). Sociodemographic and clinical factors and QOL were not associated with satisfaction. Several facilitators and barriers related to the feasibility of OncoKompas were identified.
CONCLUSION: OncoKompas is considered feasible, but has to be further improved. In order to enhance feasibility and increase satisfaction, we have to balance the time it takes to use OncoKompas, measurement precision, and tailoring towards personalized advices.

Introduction

Cancer and cancer treatment have a large impact on quality of life (QOL). Head and neck cancer (HNC) has a specific impact on survivors. In addition to symptoms such as fatigue, HNC survivors are confronted with oral dysfunction, speech and swallowing problems, and related social withdrawal and psychological distress. All of these symptoms can deteriorate quality of life [1, 2] and increase survivors’ needs for supportive care services. Several recent papers report on the need for improving survivorship cancer care [3, 4] to enhance quality of life and diminish societal discrimination.

It is essential that cancer survivors have access to optimal supportive care services including self-management options. Access to supportive care may be hampered by current changes in the health care system, e.g., limited time of health care providers and centralization of care [5]. To improve accessibility, cancer survivors are expected to adopt an active role in managing their own care. Self-management is defined by McCorkle et al. [6] as “those tasks that individuals undertake to deal with the medical, role, and emotional management of their health condition(s).” Alongside usual care, self-management options can be (cost-)effective and improve quality of life [3]. Although there is evidence that supportive cancer care can be effective, referral rates are low and many survivors have unmet needs. Innovating supportive cancer care includes incorporation of self-management and eHealth, implementation of evidence-based approaches to monitor QOL [7, 8], and redesign of the organization of supportive care according to participant centered models of care (e.g., the chronic care model, disease management, and stepped care) [3, 7, 9].

We developed an eHealth application “OncoKompas” with the aim to facilitate and innovate the access to supportive cancer care. In OncoKompas, cancer survivors can monitor their QOL by means of participant reported outcomes (PROs) (“Measure”), which is followed by automatically generated tailored feedback (“Learn”) and personalized advice on supportive care services (“Act”). To ensure sustainable implementation of OncoKompas, we followed participatory design principles. Cancer survivors and health care professionals (HCPs) were involved in each step of the development process [10-12] (Fig. 1). A qualitative assessment of needs among cancer survivors and HCPs (step 1) showed that cancer survivors are interested in an eHealth application that targets personalized access to supportive care and that HCPs expect that an eHealth application could optimize survivorship care [13]. A prototype of the eHealth application, “OncoKompas”, was developed. Existing applications were used as examples to build the application [14, 15]. OncoKompas was developed together with mixed teams consisting of cancer survivors and medical specialists as well as allied health professionals. In step 2, the usability of a prototype was tested among both cancer survivors and HCPs, targeting system quality (ease of use), content quality (usefulness and relevance), and service quality (the process of care provided) [16, 17]. HCPs raised several points for improvement regarding the ease of use and usefulness of the application, resulting in improved persuasiveness and improved tailoring. Usability tests among cancer survivors identified some weaknesses in the user interface that resulted in adjustments, e.g., clearer user instructions [18]. Based on these findings, the prototype of OncoKompas was optimized. The next step (step 3) in the developmental cycle of OncoKompas is to investigate the feasibility in clinical practice.

Fig. 1

Flow chart development process OncoKompas based on participatory design principles

The purpose of the present study was to investigate the feasibility of OncoKompas: adoption (intent to use OncoKompas), usage (actual use of OncoKompas), and satisfaction with OncoKompas among cancer survivors. Secondary aims were to investigate which sociodemographic and clinical factors are associated with the feasibility of OncoKompas and to obtain insight in possible barriers and facilitators of the feasibility of OncoKompas.

This feasibility study will provide insight into factors that contribute to the development and usage of eHealth applications among head and neck cancer survivors.

Methods

A pretest-posttest design was used, conducting a survey before providing cancer survivors access to OncoKompas (T0) and 2 weeks after (T1). After participants completed the T1 survey, they were interviewed by a nurse specialized in oncology to obtain more in-depth insight into the feasibility of OncoKompas.

Study sample and procedures

Between January and July 2013, all eligible cancer survivors from the Departments of Otolaryngology-Head and Neck Surgery from VU University Medical Center, Amsterdam, University Medical Center Leiden, and University Medical Center Maastricht, The Netherlands were invited by an oncology nurse or head and neck surgeon. Participants were eligible if they (1) were treated for head and neck cancer with curative intent with a maximum of 2 years prior, (2) were 18 years or older, (3) were able to write, read, and speak Dutch, (4) had some Internet experience, and (5) had access to the Internet at home.

If participants agreed to participate, they were asked to complete and return a paper-and-pencil questionnaire (T0). Subsequently, participants were contacted to provide them with a login code for OncoKompas at home. An interview was scheduled with an oncology nurse from their hospital 2 weeks later. Prior to their appointment with the nurse, cancer survivors were asked to fill in the post-test questionnaire (T1). During the consultation with the nurse, attention was paid to perceived usefulness of the tailored advice and personalized referral to supportive care services, as provided by OncoKompas. Oncology nurses made standardized interview reports. The scheme consisted of two main components: part A for survivors who completed OncoKompas and part B for survivors who did not complete Oncokompas. Key questions included in part A comprise the following: (1) “How would the survivor describe their experience with OncoKompas?”, (2) “Did the survivor view their personalized advice and supportive care options? If not, why?”, and (3) “Did the survivor find the results applicable to their personal situation?”. Key questions included in part B comprise the following: (1) Why did the survivor not complete OncoKompas?”, (2) “What should be changed in the application to enable the survivor to complete OncoKompas?”, and (3) “What aspects did the survivor miss in OncoKompas that prevented the survivor to complete the application?”. Nurses completed the scheme during the consultation with the survivors and supplemented these following the consultation.

Technical support was offered by two researchers (SL and FJ) when problems occurred with the access or use of OncoKompas. The researchers recorded an entry in a logbook with each technical problem.

The study was conducted according to regular procedures of the local ethical committee of the VU University Medical Center, Amsterdam. All participants signed informed consent.

Intervention “OncoKompas”

OncoKompas can be considered as both a screening and a monitoring tool and consists of three components: (1) measure, (2) learn, and (3) act. In the “Measure” component, cancer survivors can independently complete PROs targeting the following QOL domains: physical functioning, psychological functioning, social functioning, healthy lifestyle, and existential issues. Besides these domains for cancer survivors in general, a specific domain, containing topics for head and neck cancer patients, is allocated (Table 1). Specific PROs were selected by the project team in collaboration with teams of experts, based on Dutch practice guidelines and literature searches. Data from the “Measure” component are processed in real-time and linked to tailored feedback to the cancer survivor in the “Learn” component. All algorithm calculations are based on available cut-off scores, or they are defined based on Dutch practice guidelines, literature searches, and/or consensus by teams of experts. In the “Learn” component, a compass metaphor is used to summarize overall well-being. Following this, feedback is provided to the participant on the level of the topics (e.g., depression and fatigue) by means of a three-color system: green (no elevated well-being risks), orange (elevated well-being risks), and red (seriously elevated well-being risks) (first-degree algorithms). Cancer survivors receive elaborated personalized information on the outcomes, e.g., on the topic depression, information is provided on the symptoms of depression and the proportion of cancer survivors who suffer from depressive symptoms.

Table 1

Overview of topics OncoKompas

Psychological quality of life	Physical quality of life	Social quality of life	Healthy lifestyle	Life questions	Head and neck cancer
Anxiety and depression	General everyday life	Social life	Alcohol	Life questions	Swallowing
Fear of recurrence	Pain	Relationship with partner	Physical activity		Speech
Subjective cognitive functioning	Sexuality	Relationship with children	Dietary intake		Oral function
Stress	Sleep quality	Financial circumstances	Weight		Neck and shoulder function
	Body image	Patient–physician communication	Smoking		Loss of smell and taste
	Fatigue	Return to work			Head and neck cancer specific lymphedema
	Diarrhea				Nutritional drink/Tube feeding
	Lack of appetite
	Dyspnea
	Nausea or vomiting
	Constipation
	Hearing and tinnitus

Special attention is paid to clusters of interrelated symptoms. For example, feedback on the association between depression and fatigue is provided, if a participant has an orange or a red score on depression as well as on fatigue. The feedback in the “Learn” component concludes with a comprehensive self-care advice (tips and tools). All these advices are tailored to the individual cancer survivor (second-degree algorithms).

In the “Act” component, survivors are provided with personalized supportive care options, based on their PRO scores and expressed preferences (e.g., preference for individual therapy versus group therapy) (third-degree algorithms). If a participant has elevated well-being risks (orange score), the feedback includes suggestions for self-help interventions. If a participant has “seriously elevated well-being risks” (red score), the feedback includes an advice to contact their own medical specialist or general practitioner.

In appendix 1, a worked example of OncoKompas using a case study is presented. For a clickable demo of the application (in Dutch) or an animation video (in Dutch and English), please visit www.oncokompas.nl.

Outcome measures

A study-specific survey was composed with items on sociodemographic and clinical factors, a QOL questionnaire (assessed at baseline (T0)), and items on usage and satisfaction (assessed at follow-up (T1)).

Adoption, usage, and satisfaction

Adoption was defined as the percentage of cancer survivors that agreed to participate in the study and returned the T0 questionnaire and informed consent.

Usage was defined as the percentage of participating cancer survivors that actually used OncoKompas as intended based on the item “Did you fill out and use OncoKompas?” (T1).

General satisfaction was assessed based on the mean score of three study-specific questions: general impression of OncoKompas, the user-friendliness, and the ability to use OncoKompas without assistance (10-point Likert scales: 0 (poor) to 10 (good)). Furthermore, satisfaction was measured with the Net Promoter Score (NPS) with the question “How likely it is that you would recommend OncoKompas to other cancer survivors (10-point Likert scale: 0 (not likely) to 10 (very likely)). The NPS was calculated by dividing the percentage of promoters (who score 9–10) minus the percentage of detractors (who score 0–6). The percentage “passives” (who score 7–8) is not included in calculating NPS. The NPS ranges between −100 and +100. A positive score is considered good [19].

Additionally, the satisfaction of participants on the three components of OncoKompas was assessed (measure, learn, and act).

Moderating factors

Sociodemographic (age and gender) and clinical variables (tumor location, tumor stage, type of treatment, and comorbidity) were drafted by a physician from the medical records. Comorbidity was assessed by the use of the Adult Comorbidity Evaluation 27 (ACE-27), a validated chart built instrument, resulting in a total comorbidity score of none, mild, moderate, or severe [20].

The 30-item EORTC QLQ-C30 (version 3.0) includes a global health-related quality of life (HRQOL) scale (two items) and five functional scales: physical functioning, role functioning, emotional functioning, cognitive functioning, and social functioning. There are three symptom scales (nausea and vomiting, fatigue, and pain) and six single items relating to dyspnea, insomnia, loss of appetite, constipation, diarrhea, and financial difficulties. In the present study, the HRQOL scale was used. The scores of the QLQ-C30 are linearly transformed to a scale of 0–100, with a higher score indicating a higher level of HRQOL [21].

Facilitators and barriers

After the participants completed the T1 survey, they were interviewed by an oncology nurse to obtain more in-depth insight into barriers and facilitators of the feasibility of OncoKompas.

To evaluate technical issues interfering with the feasibility of OncoKompas, the entries in the helpdesk logbook were evaluated on the type of problem encountered, and if and how the problem was solved.

Data analyses

Descriptive statistics were used to summarize the adoption, usage, and satisfaction. OncoKompas was defined feasible in case of an adoption and usage grade of more than 50 %, a mean satisfaction score of at least 7, and a positive Net Promoter Score. This definition of feasibility is based on adoption and usage rates reported in previous studies on eHealth applications [22].

Correlations between satisfaction with OncoKompas and gender (male vs. female), age (<65 vs. >64 years), comorbidity (none/mild vs. moderate/severe), tumor subsite (oral cavity/oropharynx vs. hypopharynx/larynx vs. other), tumor stage (stage I/II vs. stage III/IV), and treatment modality (surgery alone vs. surgery plus (chemo)radiation vs. (chemo)radiation) were examined using chi-square tests. The outcome variable satisfaction with OncoKompas was not normally distributed and was therefore dichotomised into two categories: a score from 0 to 6 and a score of 7 and higher. The association between satisfaction with OncoKompas and QOL was analyzed using Mann-Whitney U tests. Statistical analyses were performed using the Statistical Package for the Social Sciences (IBM SPSS Statistics 20). For all analyses, p values <0.05 (two-tailed) were used as criterion for statistical significance.

The structured interview reports by the oncology nurses were analyzed by thematic analyses. Barriers and facilitators towards feasibility of OncoKompas were extracted from the reports and those that were mentioned at least five times are reported. The entries in the logbook with technical difficulties were categorized by type of problem that was encountered. The number of unique problems was counted.

Results

Adoption and usage

In total, 106 HNC survivors were asked to participate in the study. The adoption grade was 64 %: 68 out of 106 intended to use OncoKompas and gave informed consent and returned the T0 survey. Reasons for non-consent included a lack of time of the patient and no willingness to travel to the hospital for the nurses’ consultation.

In total, 12 participants dropped out during the study (17.6 %), leaving a study cohort of 56 participants (Table 2). The reasons for dropout included cancer recurrence (n = 2), entering palliative care (n = 2), family circumstances (n = 1), comorbid illness (n = 1), tiredness due to which OncoKompas could not be used (n = 1), evaluating the questions in OncoKompas as too confronting (n = 1), insufficient Internet skills according to the participant (n = 1), and not able to reach participants by telephone (n = 3). Of the 56 participants (56 out of 68) who completed the study, 51 survivors filled out OncoKompas completely and as intended (usage grade 91 %). In total, five of these participants indicated that they received assistance while filling out OncoKompas (with the help of my spouse (N = 4) or with the help of my daughter (N = 1)). Two survivors indicated to have used OncoKompas partially (3.6 %). They mentioned that they only filled out the ‘Measure’ component in OncoKompas. Three other survivors (5.4 %) indicated not to have used OncoKompas due to technical reasons; one participant indicated to be hindered by a bug when using the application, while the other two participants indicated that because of a bug in the application they could not continue. Of the participants who encountered technical problems, one survivor contacted the helpdesk. Despite provision of assistance, this participant was not able to complete OncoKompas. Usage grade thus lies between 75 % (51 out of 68 participants (including dropouts)) and 91 % (51 out of 56 participants (excluding dropouts)) who used OncoKompas as intended.

Table 2

Demographic and health characteristics of the participating participants (N = 56)

	n	%
Sex (n, %)
Female	22	39.3
Male	34	60.7
Age in years
Mean (SD)	59.05 (9.85)
Minimum	25
Maximum	77
Tumor site (n, %)
Oral cavity and oropharynx	30	53.6
Hypopharynx and larynx	12	21.4
Other	14	25.0
Comorbidity (ACE-27) (n, %)
None	16	28.6
Mild	17	30.4
Moderate	18	32.1
Severe	5	8.9
Type of treatment (n, %)
(Chemo) radiation therapy ((C)RT)	27	48.2
Surgery	13	23.2
Surgery + (C)RT	16	28.6
Time since treatment (in months)
Mean (SD)	12.32 (6.5)
Minimum	0
Maximum	24
Quality of Life (EORTC QLQ C-30)
Mean (SD)	76.33 (16.49)
Minimum	33.33
Maximum	100

Satisfaction

Most of the participants were satisfied with OncoKompas in general (60.4 %, mean score 6.8, SD 1.2). Participants evaluated OncoKompas as user-friendly (76.0 %, mean score 7.1, SD 1.6). Participants were able to use OncoKompas without assistance (90.6 %, mean score 7.8, SD 1.7). The mean satisfaction score was 7.3 (SD 1.5). The Net Promoter Score of OncoKompas was positive 1.9, consisting of 21 % promoters, 19 % detractors, and 60 % passives.

Regarding the feasibility of the “Measure” component, almost all participants answered all PROs (98 %). For some participants, the PROs were intrusive (21 %), confusing (29 %), or difficult to answer (37 %). Confusing and difficult questions mentioned by survivors included questions related to God and religion. Questions about sexuality were found intrusive. Almost all participants (94 %) viewed their well-being profile in the “Learn” part of OncoKompas. To most participants, the description of their results was clear and understandable (84 %) and easy to find in the application (81 %). More than half of the participants evaluated the information as applicable to themselves (61 %), but less than half evaluated the information of added value for their own health status (43 %). More than half of the participants (61 %) indicated that the overall picture regarding their results (the compass metaphor) did not add much. Most participants viewed their personalized advices (71 %), and these advices were evaluated as clear (85 %) and complete (68 %). The amount of supportive care options provided in the “Act” component was considered to be exactly right to most participants (71 %) or too much (23 %). More than half of the participants (57 %) indicated to be interested in one or more of the offered supportive care options and almost a third of these participants subsequently did take action accordingly (29 %). The majority indicated to return to OncoKompas in the future to view their personalized advices and actions once again (71 %).

The helpdesk was contacted for a total of 21 unique problems. The problems mainly consisted of difficulties logging on to the application: due to a browser problem (n = 2), loss of password (n = 2), expiration of security certificate of the application (n = 3), use of a tablet (n = 2), blockage by firewall (n = 2), no compliance of the computer with application requirements (n = 1), and other reasons (n = 5). Furthermore, an error message appeared (n = 2) and the button to print the results was not visible (n = 2).

Factors associated with satisfaction with OncoKompas

Satisfaction with OncoKompas was not significantly associated with age (X 2 (1, N = 53) = 0.26, p = 0.61), gender (X 2 (1, N = 53) = 0.58, p = 0.45), tumor location (X 2 (2, N = 53) = 5.49, p = 0.06), tumor stage (X 2 (1, N = 53) = 0.00, p = 0.97), type of treatment (X 2 (2, N = 53) = 3.38, p = 0.19), comorbidity (X 2 (1, N = 53) = 0.034, p = 0.85), and HRQOL (p = 0.35).

Barriers and facilitators

In total, seven barriers towards the feasibility of OncoKompas were mentioned at least five times (Table 3): (1) The application did not fully take into account other diseases that participants suffered from, (2) the amount of information in the application was too much, (3) items regarding existential issues were difficult to answer and too much oriented towards religion, (4) participants did not find the results completely applicable to their personal situation (they experienced their symptoms in a different way), (5) participants found a lapsed time of 2 years since treatment to introduce the application too long (these participants often already found a solution to the experienced problems or learned to live with them), (6) the description of participant’s overall well-being was suboptimal, either considered confronting or meaningless, and (7) participants found some items in the application confusing making it difficult to answer them truthfully.

Table 3

Barriers and facilitators (mentioned at least five times)

Barriers	No. of times mentioned	Facilitators	No. of times mentioned
The application did not fully take into account other diseases that participants suffered from	11	The user-friendliness of the application	13
The amount of information in the application was too much	9	The informative nature of the applications	7
Items regarding existential issues were difficult to answer and too much oriented towards religion	9	The provision of a clear overview to the participant of their personal situation and options	6
Participants did not find the results completely applicable to their personal situation.	8	The clarity of the items in the questionnaire.	6
Participants found a lapsed time of 2 years since treatment to introduce the application too long	7	The usefulness of the application in general	5
The description of participant’s overall well-being was suboptimal	6	The particular usefulness of the application for participants who are very ill or experience many symptoms	5
Participants found some items in the application confusing making it difficult to answer them truthfully.	6

Six facilitators were mentioned at least five times (Table 3): (1) the user-friendliness of the application, (2) it’s informative nature, (3) the provision of a clear overview to the participant of their personal situation and options, (4) the clarity of the items in the questionnaire, (5) the usefulness of the application in general, and (6) the particular usefulness of the application for participants who are very ill or experience many symptoms.

Discussion

The aim of this study was to investigate the feasibility of the eHealth application OncoKompas aiming to facilitate and innovate cancer survivorship care. Our results show that OncoKompas is feasible with an adoption grade of 64 %, a usage grade between 75 and 91 %, a mean satisfaction score of 7.3, and a positive Net Promoter Score. Almost all participants were able to use OncoKompas. The few participants that were not able to use OncoKompas seemed to be hindered by insufficient eHealth skills. These findings are in line with previous studies that examined the feasibility of eHealth applications in clinical practice showing that eHealth applications are acceptable to many participants but not to all [23-26].

Our study showed no associations with sociodemographic factors. Also in previous studies, no significant associations between use of eHealth and gender were found [25, 27, 28]. Previous studies revealed mixed results concerning the association between age and use of eHealth applications [25-29]. Increased age seems to be associated with less use of the Internet, although this association has become less strong in recent years, probably due to the availability and increased familiarity of Internet [26]. A review by Or and Karsh revealed that most studies did not show significant relationships between eHealth acceptance and age [25]. Our study results match these findings.

In our study, there were no significant associations between satisfaction with OncoKompas and clinical factors or QOL. Previous studies that focused on health and treatment factors showed that participants who use eHealth are healthier than participants who do not use eHealth [30, 31]. A better health status seems to lead to a better acceptance of Internet applications [32]. However, in other studies, the opposite was found [33, 34].

Other factors than sociodemographic and clinical factors and QOL may be of more interest in investigating why participants use (or not use) eHealth applications. Adoption rate of eHealth can be predicted by the way an eHealth application is rated in terms of usefulness and ease of use, and the self-efficacy of participants regarding information technology [25, 35]. In the present study, only participants with sufficient (self-reported) computer skills were included. Facilitators associated with the feasibility of OncoKompas included the user-friendliness of the application and its informative nature. Ease of use was also determined in previous studies as an important factor for the acceptance of eHealth applications [25]. Important barriers included the feeling that the results did not completely reflect the personal situation, the large amount of information in the application, and difficulties answering some of the items. The barrier regarding the time investment required to complete the application has been reported in previous studies as well. Length and information overload have been found to be important reasons to quit using an online application [36-38]. Individualized feedback has been found to be related to sustained intervention use and less dropout [39, 40].

The strength of our study is that we used mixed methods, providing in-depth insight into the feasibility of eHealth applications in clinical practice.

In the present study, only participants were included with access to the Internet at home. Therefore, we do not have good insight into the representativeness of our study sample. The positive attitude of participants towards OncoKompas might not be generalizable to all HNC survivors. Another limitation concerns the small sample size, which may have hampered testing the associations. Finally, in this study, participants had access to the eHealth application for only 2 weeks. Further research is needed on the feasibility in the longer term.

Conclusions

OncoKompas is considered feasible, but our results also show that improvements can be made to enhance the feasibility and increase the satisfaction among cancer survivors. The PROs can be further investigated and possibly be reduced. It is also important to look at the phrasing of individual items, because of the barrier that answering some of the items was difficult to participants. However, in order to ensure accuracy of the individualized feedback, we have to balance the time it takes to use OncoKompas, measurement precision, and tailoring towards a personalized tool. It is clear that any eHealth application will not be suitable for all participants, due to different needs, preferences, and coping styles of cancer survivors. It is worthwhile to obtain more insight into how further tailoring of eHealth applications and more sophisticated marketing strategies can be applied leading to applications that are attractive to more participants and hereby increase adoption and usage.