Development and evaluation of a computerized clinical outcome assessment tool for head and neck cancer patients.

Paper-based clinical outcome data collection methods have practical limitations when used in clinical settings, as the data are often not summarized in time to facilitate patient-physician communications and therefore cannot be used in clinical decision making. This study aimed to develop a computerized clinical outcome assessment tool (COAT) and evaluate its acceptability, feasibility, and potential clinical applications during clinical encounters for patients with head and neck cancer (HNC).The traditional Chinese (TChi) character version of the Functional Assessment of Cancer Therapy-Head and Neck (FACT-H&N) questionnaire was first transcribed and implemented into a touch-screen computerized administration and reporting system (COAT-HNC for short). Each HNC patient was invited to complete the COAT-HNC during their scheduled clinic visits as part of their clinical care. Upon completion, a structured summary report was generated, and subsequently used for treatment evaluation and planning.A cohort of 385 HNC patients were enrolled. Each scale of the computerized TChi FACT-H&N questionnaire demonstrated acceptable internal consistency, with Cronbach coefficient alpha ranging from 0.74 to 0.90. The touch-screen-based and audio-capable COAT-HNC was reported to be easy to use. Patients and physicians were able to utilize the summary report during their clinical encounters to discuss treatment progress and to plan care.It is practically feasible to design, develop, and implement the COAT-HNC system in routine HNC care. The COAT-HNC has the potential to become a valuable tool for data collection and management of clinical outcomes, and appears useful for HNC patients. However, larger studies to demonstrate its clinical usefulness are still needed.

Introduction

Head and neck cancer (HNC) (especially cancer of the oral cavity and nasopharyngeal cancer) is one of the most prevalent forms of cancer in Taiwan.[ Its incidence rate is gradually increasing, from 25.1 per hundred thousand in 2008 to 29.7 in 2011, and its mortality rate is the fifth leading cause of cancer-related deaths.[ Middle-aged men in particular are at high-risk of HNC.[ Although the survival rate fortunately continues to rise,[ and the availability of medical information, diagnostic technology, and modes of therapeutic intervention increase, due to the exponential increase in incidence, its human and economic impacts on patients, their families, and society remain enormous.

As this disease progresses, patients experience an increasing deterioration of their health-related quality of life (HRQOL), a great impairment in their ability to work, and a declining participation in social and physical activities. For example, radiation-induced damage to normal tissue caused by radiotherapy may result in body image defects, as well as problems with eating, speaking, and hearing. It is therefore important to collect clinical information from the patients’ perspective to assess and confirm their own function and well-being.[ This information can also be used to facilitate patient–physician communication and enhance decision making during clinical encounters. Additionally, it allows clinicians to identify patients’ clinical issues in combination with the laboratory data, as well as to systematically track the changing needs of care resulting from treatment modification over time.[

Systematic clinical outcome assessment is valuable in oncology practices, as it can potentially increase the awareness of a wide range of issues, increase the detection of psychological morbidity, social problems, and changes in physical status, and improves care, and its outcomes.[ To accurately and efficiently measure Patient-Reported Outcomes (PROs) in clinical research and patient care, the US National Institute of Health funded the Patient-Reported Outcomes Measurement Information System (PROMIS). Similarly, a National Health Service and National Resource was created in England and Wales.[ Wu et al[ initiated a study, supported by standardized and systematic instruments, to collect cancer patients’ treatment experiences and perspectives. The aim here was to provide a complete picture of treatment and disease impact, to better understand patients’ health status and care needs, and to help clinicians make better, effective decisions to provide high-quality patient-centered care.[

The recent advances in information technology have created an ideal mechanism for collecting, storing, analyzing, and reporting clinical outcome data, including HRQOL data, in a more standardized and systematic manner, to facilitate patient–physician communication during clinical encounters. PROs data, including patients’ physical and mental state, social functionality, and symptoms, can be routinely collected, stored within the electronic health records (EHR) system and linked to other patient health records. The EHR system can then serve as a clinical caring tool to timely screen patients’ conditions and needs, and monitor their changes in functional status and HRQOL over time, to improve case management in clinical oncology practice.[ Results of a study by Snyder et al[ provide evidence that the incorporation of PROs data in the EHR system could improve patient–clinician communication, patient treatment outcomes, and care quality.

Although use of touch-screen technology (TST) on desktop computers and mobile devices is not uncommon in either our daily life or in the field of medicine,[ TST-assisted clinical outcome data collection and reporting in routine clinical practices has been limited, and has yet to reach its desired potential, due to substantial logistical constraints.[ It is therefore important and necessary to carefully design, develop, and deploy a user-friendly data capture and reliable reporting system, similar to those used in clinical decision support systems, to improve clinical outcomes and quality of care in cancer patients.

Aim of the study

The main objectives of this study were to: design, develop, and deploy a computerized Clinical Outcomes Assessment Tool (COAT) for HNC patients; and examine the potential use of the COAT-HNC to facilitate patient–physician discussion and communication regarding clinical decision making and treatment outcomes during clinical encounters.

Methods

Participants

Convenience sampling of patients with HNC, including nasopharynx, oral cavity, oropharynx, hypopharynx, and larynx, was carried out at the Lin-Kou Medical Center of Chang Gung Memorial Hospital (CGMH), a community-based medical center in northwestern Taiwan. The participant inclusion criteria were relatively broad to include HNC patients treated at CGMH with diverse clinical and demographic characteristics to increase the generalizability of the study results.

Procedures and ethics

This study was reviewed and approved by the Human Research Ethics Committee of Chang Gung Memorial Hospital Institutional Review Board (IRB reference number: 96-0730B). Approved recruiting procedures were followed. Eligible HNC patients were approached by 1 of 2 trained research assistants during their scheduled clinic visits. Patients who agreed to participate in the study signed informed consent forms and were instructed to complete the computerized TChi FACT-H&N questionnaire and provide other demographic and clinical information (see Fig. 1). A research assistant was available on-site to assist the patient participants as needed. The entire administration process for each patient took about 15 to 20 minutes.

Figure 1

Computerized assessment. This is a situational example. After the patient agreed to participate in this study, he was instructed and encouraged to self-complete the computerized TChi FACT-H&N assessment on the touch-screen computer monitor. TChi FACT-H&N = the traditional Chinese character version of the Functional Assessment of Cancer Therapy-Head and Neck.

To evaluate the feasibility and acceptability of the use of the COAT-HNC during office visits, a debriefing interview was conducted by a research assistant after the computerized survey was completed. Interview questions included coverage of any difficulty experienced in understanding the questions or distress caused by any questions. Patients were also given the opportunity to suggest any additional questions for the survey and to comment on the overall design of the COAT-HNC system.

Measurement

FACT-H&N. The traditional Chinese character version of the Functional Assessment of Cancer Therapy-Head and Neck (TChi FACT-H&N) health-related quality of life (HRQOL) questionnaire [ was used. This has 27 core items assessing patient well-being in 4 domains: physical (7 items), social/family (7 items), emotional (6 items), and functional (7 Items) well-being. It also has 11 specific items to assess H&N related symptoms. Each item is rated on a 0 (“not at all”) to 4 (“very much”) Likert type scale, with a higher score indicating better HRQOL or functionality.

Design of the COAT-HNC system

To make the COAT-HNC system for data collection, analysis, and reporting as user-friendly as possible, particularly for those with limited computer experience, we applied a stepwise approach to ensure every step was properly designed and executed. The TChi FACT-H&N was transcribed into a touch-screen computer administration program, written in Microsoft Access, for ease of administration and self-reporting. Each question and their corresponding response choices were displayed 1 item at a time on the screen. Proper type faces and font sizes were used uniformly to improve ease of readability and to avoid any confusion or distraction. Questions related to basic demographic and clinical characteristics were also included.

A unique audio read-out feature was also developed to improve patient comprehension of the questions. This allowed the computer to read the question and associated response categories in Mandarin Chinese to the patient when chosen. That is, a patient would have the option to click an “ear-like” icon to hear a specific question prerecorded by real human voice when needed or preferred. This was to address the limited health literacy issue and to ensure each question was understood by the study participant as intended.

To facilitate patient–physician discussion and communication on the data being collected, a dashboard-like data visualization feature with organized information was also added to allow the patient and the physician to view the results together on the monitor. The option to print out a hard copy of the summary report (“Patient Health Profile”) was also added (see Fig. 2 for example). This structured report was designed to organize the longitudinal data with clinically relevant information, such as HRQOL, in an easy-to-understand manner, to initiate conversations between patients and physicians.

Figure 2

Patient's health profile. This is an example of patient's health profile including basic demographic and clinical information, etc. This output report was the result of the computerized clinical outcomes assessment tool (COAT) system when a patient completed the computerized TChi FACT-H&N assessment. TChi FACT-H&N = the traditional Chinese character version of the Functional Assessment of Cancer Therapy-Head and Neck.

Statistical analysis

Descriptive statistics and psychometric properties of the computerized TChi FACT-H&N were examined. Means, standard deviations, and Cronbach alpha coefficients were calculated for each subscale of the FACT-H&N using the SPSS statistical analysis software package (version 21; SPSS, Inc, Chicago, IL). Brief open-ended interview data on Post-COAT-HNC administration were also reviewed and summarized.

Results

Study sample characteristics

A total of 385 HNC patients were enrolled. Most patients were men (n = 339, 88.1%). The median year of education was 9 years, ranging from 0 to 20 years. The median age was 52.5 years (ranging from 23 to 81 years). All patients exhibited good performance status ratings (Karnofsky score ≥ 70).[ The most common type of cancer was nasopharyngeal carcinoma (NPC) followed by pharynx (including oropharynx and hypopharynx) cancer, then oral cavity cancer. All patients reported had completed radical treatment for at least 1 year without evidence of disease recurrence. Detailed patient demographic and clinical characteristics can be found in Table 1.

Table 1

Patient characteristics (N = 385).

Reliability analysis

The computerized TChi FACT-H&N Total Scale demonstrated good reliability (α = 0.90), according to Terwee criteria[ (0.70< α < 0.95). The Cronbach's of each of the 5 TChi FACT-H&N subscales were acceptable; the scores for Physical Well-Being, Social/Family Well-Being, Emotional Well-Being, Functional Well-Being, and H&N concern were 0.81, 0.76, 0.74, 0.85, 0.87, and 0.76, respectively.

Feedback on the COAT-HNC system

Those patients who provided feedback generally had minimal or no difficulty in answering all the COAT-HNC questions using a touch-screen computer. A limited number of patients expressed some difficulty for reasons including: the translations of negatively worded items made questions challenging to understand; the long time lapse between the 2 COAT-HNC assessments required some efforts to recall how to use it; personal characteristics such as limited literacy, older age and lower educated, constrained their reading ability, and such patients consequently had to demand assistance each time; and physical limitations, such as blurred vision or fatigue, interfered with their ability to respond. Overall, difficulty in responding to the COAT-HNC questions often occurred during the first encounter as patients tried to comprehend the questions. Some patients still asked for assistance to ensure their full understanding of the questions even after using the COAT-HNC system several times. Some also commented that they were able to utilize the read-out option when needed. The respondents were also satisfied with the amount of time required to complete the COAT-HNC questions without jeopardizing their scheduled clinic visits.

Feedback on the summary report

Both patients and physicians commented that the structured summary report (e.g., the Patient Health Profile; see Fig. 2) gave them the opportunity to review and discuss the collected information together. Specifically, physicians noted that the organization of the data in sections allowed them to easily identify the specific clinical areas in which the patient needed additional attention. The trend chart showing the longitudinal data collected at multiple time points also helped them to evaluate treatment progress. Patients also expressed that they were more engaged in the discussions and could ask relevant questions regarding their treatment options and expected outcomes. Patients and physicians also suggested that color-coding of data to indicate progress (improved in green; no change in yellow; worsened in red) could help to draw their attention to specific care needs.

Discussion

The self-reported HRQOL data collected using the COAT-HNC system appears to be valid and reliable. The psychometric properties of the TChi FACT-H&N are similar to its original English version, which is an important first step in ensuring that all collected data are reliable and useful for their clinical applications in Chinese-speaking patients. This study also addresses the significant and important need for reliable, valid, and readily available patient-reported outcome data for use in an oncology setting. Asking patients to provide their clinically relevant data before or during the in-person clinic visit also sends the message to the patients that physicians care very much about their health status and treatment outcomes. The use of simple and inexpensive technology appears to be practically feasible, and can be easily applied in H&N oncology clinics and other settings.

The collected evidence also supports the COAT system as a potentially valuable tool for clinical outcome assessment data management in treating HNC patients. The results demonstrated that we had successfully designed, developed, and deployed the COAT system in HNC care with minimal cost and promising success. Patients of varying educational backgrounds, even those of limited literacy, were able to complete all the COAT questions by themselves with no or minimal assistance. It should be noted that some elderly patients were reluctant to use the system at first, due to discomfort using the computer; this was especially prevalent in patients over 65-years-old who had limited or no experience using a computer in their life. However, after a research assistant's careful instruction and assistance, elderly patients were eventually able to complete the COAT-HNC questions.

The minimal demands on staff and patient time in the data collection process would reduce burden and increase the use of the COAT system in busy clinics. This system also presents the opportunity to allow patients to share more specific information with their physicians with minimal effort. Another point to note is that computerized equipment, internet connectivity, and additional skills training for healthcare professionals were all necessary to assist or encourage patients using the COAT-HNC system. These staffing and training requirements all have cost implications that need to be considered.

The COAT-HNC also facilitated physicians in the discussion of HRQOL issues with patients during clinical encounters. The structured summary report was readily available to facilitate patient-physician review of results during the same clinical visit. This could help initiate and improve patient-physician discussions and communication regarding patients’ symptoms, treatment outcomes/side effects, and care quality.[ This study also suggests that it is helpful for patients to know and understand the information regarding their current health status and the progress they have made, so that they can make more informed treatment choices and decisions.

The PROs data, when routinely collected by the standardized procedures and tools on the Website and linked to other clinical records, could support the delivery of high-quality and patient-centered care, and may be cost-effective to monitor patients’ functional status and HRQOL over time.[ Our study results are very similar to those of Snyder et al study which evaluated the use of Web technology to collect PROs data, which is then linked to the EHR to provide plain-text and graphical table reports.[ Our study also supports the idea that providing patients with the information they need would improve both the patient-physician relationship and patient satisfaction with their care. The study findings also suggest the COAT system to be reliable and cost-effective for collecting and monitoring HNC patients’ long-term HRQOL in clinical oncology practice.

Conclusion

The COAT-HNC system has proven to be a viable mechanism for routine clinical outcome data collection in oncology settings for HNC patients. It is relatively easy to design, develop, and deploy. It is also inexpensive, cost efficient, and can be easily integrated with any EHR systems. The data are reliable and can be easily captured. The audio read-out feature also provides a plausible solution to address any patient literacy issues, by improving patient understanding of the questions and related answers. The structured summary report offers a unique opportunity for patients and physicians to discuss relevant and important clinical issues and to develop practical, feasible treatment plans together. Developing systems like the COAT-HNC system for collecting and reporting patient outcomes can help healthcare providers improve the quality of care they deliver. It has the potential to be applied to meet large-scale data collection needs, and it provides a mechanism to facilitate patient-physician communication regarding clinical outcomes.

In terms of clinical utility and research applications, the COAT-HNC system has demonstrated good reliability, user acceptability and usability. Its implementation reduces the data entry error rate compared with paper forms. The individual question response data from the patients and their analysis results were recorded and securely stored electronically to decrease use of paper. It was also beneficial for the healthcare professionals to have timely access to these additional data to better understand and communicate HRQOL issues with their patients. In summary, the COAT-HNC system has great potential and further investigations are merited to evaluate its clinical applications to improve cancer care across different clinical settings.

Author contributions

Designed this Study: JTCC, C-HC. Carried out the data collection: K-HF, C-YL, T-MH, B-SH, EY-CC, C-JK, S-FH. Performed statistical analysis, interpretation of findings, and discussion of results: C-RL, JT-C C, C-HC. Drafted the manuscript preparation and revisions: C-HC, C-RL, T-MH, JT-CC. All authors read and approved the final manuscript.