Development and Validation of Ovarian Symptom Index-18 and Neurotoxicity-4 for Korean Patients with Ovarian, Fallopian Tube, or Primary Peritoneal Cancer.

PURPOSE: The purpose of this study was to develop Korean versions of the National Comprehensive Cancer Network/Functional Assessment of Cancer Therapy (NCCN-FACT) Ovarian Symptom Index-18 (NFOSI-18) and FACT/Gynecologic Oncology Group (FACT-GOG) Neurotoxicity 4-item (NTX-4), evaluating their reliability and reproducibility.
MATERIALS AND METHODS: In converting NFOSI-18 and NTX-4, the following steps were performed: forward translation, backward translation, expert review, pretest of preliminary format, and finalization of Korean versions (K-NFOSI-18 and K-NTX-4). Patients were enrolled from six institutions where each had completed chemotherapy for ovarian, tubal, or peritoneal cancer at least 1 month earlier. In addition to demographics obtained by questionnaire, all subjects were assessed via K-NFOSI-18, K-NTX-4, and a Korean version of the EuroQoL-5 Dimension. Internal structural validity and reliability were evaluated using item internal consistency, item discriminant validity, and Cronbach's α. To evaluate test-retest reliability, K-NFOSI-18 and K-NTX-4 were readministered after 7-21 days, and intraclass correlation coefficients (ICCs) were calculated.
RESULTS: Of the 250 women enrolled during the 3-month recruitment period, 13 withdrew or did not respond, leaving 237 (94.8%) for the analyses. Mean patient age was 54.3±10.8 years. Re-testing was performed in 190 patients (80.2%). The total K-NFOSI-18 and K-NTX-4 scores were 49 (range, 20 to 72) and 9 (range, 0 to 16), respectively, with high reliability (Cronbach's α=0.84 and 0.89, respectively) and reproducibility (ICC=0.77 and 0.84, respectively) achieved in retesting.
CONCLUSION: Both NFOSI-18 and NTX-4 were successfully developed in Korean with minimal modification. Each Korean version showed high internal consistency and reproducibility.

Introduction

In the United States alone, approximately 22,440 new cases of ovarian cancer and 14,080 related deaths were recorded for 2015 [1]. The incidence of ovarian cancer, which is now 1.5% annually in Korea [2], continues to climb. Ovarian cancer is typically diagnosed at an advanced stage of the disease (in more than 70% of patients). Relapse is common after firstline treatment, and prolonged therapeutic intervention is usually required [3,4]. Quality of life (QoL) is thus included with survival as an endpoint in clinical trials. However, there is scant evidence from the numerous studies conducted to date that palliative chemotherapeutic regimens administered for platinum-resistant ovarian cancer actually improve symptom control [5].

The National Comprehensive Cancer Network (NCCN)–Functional Assessment of Cancer Therapy (FACT) Ovarian Symptom Index-18 (NFOSI-18), developed in the United States in 2011, helps assess symptoms that patients with advanced ovarian cancer may experience [6]. It is based on three subscales: disease-related symptoms, treatment-related symptoms, and general function/well-being Preliminary reliability indicators suggest good internal consistency, with a Cronbach's α of 0.80. NFOSI-18 and its subscales have demonstrated significant positive associations with performance status. In a recent study validating the content of NFOSI-18, approximately 90% of participants rated each item as clear and understandable [7]. However, NFOSI-18 lacks a neurotoxicity subscale, and 18% of patients rendered spontaneous reports of neuropathy during open-ended questioning regarding symptoms. The FACT/Gynecology Oncology Group (GOG) Neurotoxicity 4-item (NTX-4) survey is a validated tool for assessing platinum/paclitaxel-induced neurologic symptoms [6,7].

The Korean Gynecology Oncology Group (KGOG) is considering participating in international GOG collaborative trials that use NFOSI-18 and NTX-4. As there are currently no validated Korean versions of these questionnaires, development and evaluation of Korean versions (K-NFOSI-18 and K-NTX-4) are required. Accordingly, the goals of this study were to adapt the existing NFOSI-18 and NTX-4 for Korean women and to evaluate the reliability and reproducibility of these adaptations.

Materials and Methods

1. Study population

This multicenter study, involving six institutions (Seoul National University Hospital, National Cancer Center, Seoul National University Bundang Hospital, Asan Medical Center, Samsung Medical Center, and Gangnam Severance Hospital), was conducted between August 2016 and October 2016. Participants were recruited by the respective Departments of Obstetrics and Gynecology and were consecutively enrolled. Inclusion criteria were as follows: (1) age 18 years or older; (2) diagnosis of ovarian, fallopian tube, or primary peritoneal cancer; (3) prior treatment with adjuvant chemotherapy or radiotherapy; and (4) ability to speak and understand the Korean language. Chemotherapeutic treatment within the 1 month prior to study enrolment, inability to understand the questionnaires, and refusal to participate were the criteria for exclusion.

The collected patient data included demographic variables (age, marital status, educational level, state of employment, family income, smoking/drinking habits, exercise activity, menopausal status, height, and weight) and disease characteristics (cancer stage, grade, duration of treatment, and recurrence status). To assess test-retest reliability, the questionnaires initially administered to patients were repeated in the same subjects 7-21 days later. All patients were asked to participate in this retesting. The Korean version of the Euro-QoL-5 Dimension (K-EQ5D) questionnaire was also administered at the same time as the initial administration of K-NFOSI-18 and K-NTX-4 [8].

2. NFOSI-18 and NTX-4 questionnaires

Following baseline interviews, patients completed a Korean version of NFOSI-18 (K-NFOSI-18). NFOSI-18 consists of 18 items with four dimensions: disease-related symptoms, physical (nine items); disease-related symptoms, emotional (one item); treatment side effects (five items); and function/well-being (three items). Each item is scored from 0 (“not at all”) to 4 (“very much”) [6].

Following baseline interviews, patients also completed a Korean version of NTX-4 (K-NTX-4) [9]. NTX-4 consists of these four items: numbness or tingling in the feet, numbness or tingling in the hands, discomfort in the feet, and discomfort in the hands. Each item is scored from 0 (“not at all”) to 4 (“very much”).

K-NFOSI-18 and K-NTX-4 were developed through forward translation, backward translation, and discussion. Two independent researchers translated K-NFOSI-18 and K-NTX-4 into Korean, and a bilingual (English and Korean) researcher performed backward translation. The first Korean version was reviewed by a scholar of Korean literature. In September 2016, ten patients with ovarian cancer participated in a pilot test of both surveys at the Seoul National University Hospital. Based on the pilot results, item 13 (“I am bothered by side effects of treatment”) was modified because the question was incomprehensible in the Korean language for two patients. Once revised, the finalized versions were administered to study participants at each hospital as interviewer-assisted self-report questionnaires.

3. Data analysis

Categorical variables were expressed as frequency (%) and continuous variables as mean±standard deviation or median (range). To test the validity and reliability of questionnaires, item-level and dimension-level analyses were performed. In item-level evaluations, item internal consistency (IIC), and item discriminant validity (IDV) served as indices of internal structural validity. IIC reflects correlation between an item and all companion items in the same dimension, whereas IDV measures how well an item correlates with items in the same dimension, compared with items in other dimensions. In dimension-level evaluations, reliability was evaluated via Cronbach's α, based on the total score or dimension score (in dimensions with more than two items). Reproducibility was evaluated through test-retest reliability and indicated by the intraclass correlation coefficient (ICC). The NFOSI-18 scoring guidelines ver. 2 (S1 Fig.) and NTX scoring guidelines ver. 4 (S2 Fig.) were used to calculate dimension scores. Responses were inverse-coded in K-NFOSI-18 and K-NTX-4, except for five questions in K-NFOSI-18 (C3, GF5, BMT5, GF3, and GF7), so that higher scores would represent better performance in both scores. Spearman’s correlation coefficient was calculated to evaluate the external validity of K-NFOSI-18, K-NTX-4, and K-EQ5D scores. Differences in K-NFOSI-18 and K-NTX-4 scores according to recurrence status were compared using the Wilcoxon rank sum test to measure discriminant validity. All reported p-values were two-sided, with statistical significance set at p < 0.05. Statistical analyses were performed using R project freeware (ver. 3.3.2).

4. Ethical statement

The institutional review boards of each contributing institution (Seoul National University Hospital: IRB#1607-206-782, National Cancer Center: IRB#2016-0194, Seoul National University Bundang Hospital: IRB#1608-358-305, Asan Medical Center: IRB#2016-0831, Samsung Medical Center: IRB# 2016-07-087, and Gangnam Severance Hospital: IRB#3-2016-0163) approved the study protocol, and all participants provided written informed consent.

Results

During the 3-month recruitment period, 250 women were enrolled in the study. The main analyses were limited to 237 patients (94.8%), after excluding withdrawals and nonresponses. Re-testing without missing data was conducted in 190 patients (80.2%). Table 1 displays socio-demographic and clinical characteristics of the study subjects. The mean age was 54.27±10.75 years, with most patients registered as non-smokers, non-drinkers, and menopausal. More than half of the patients (55%) exercised regularly. The median duration of treatment was 5 months. Paclitaxel-carboplatin based chemotherapy was received by 97% of patients. Cancer stages were distributed as follows: stage 1, 81/237 (34.6%); stage 2, 30/237 (12.8%); stage 3, 90/237 (38.5%); and stage 4, 27/237 (11.5%). The overall rate of cancer relapse was 25.7%.

Table 1.

Demographic and disease characteristics of study population

Characteristic	No. (%) (n=237)
Age, mean±SD (yr)	54.27±10.75
Marital status
Single	26 (11.0)
Married	191 (80.6)
Divorced	9 (3.8)
Widowed	11 (4.6)
Educational level (missing=2)
Elementary school or less	17 (7.3)
Middle school	20 (8.5)
High school	72 (30.6)
College or above	126 (53.6)
Employment (missing=1)
Employed	67 (28.4)
Unemployed	169 (71.6)
Family income (1,000 won/mo) (missing=10)
< 200	47 (20.7)
200-300	43 (18.9)
300-400	41 (18.1)
400-500	39 (17.2)
> 500	57 (25.1)
Smoking
Current	3 (1.3)
Past	11 (4.6)
Nonsmoker	223 (94.1)
Drinking (missing=1)
> 4 times/wk	0
2-3 times/wk	3 (1.3)
Once/wk	17 (7.2)
Nondrinker	216 (91.5)
Regular exercise
Yes	130 (54.9)
No	107 (45.1)
Menopausal status
Yes	216 (91.1)
No	21 (8.9)
Height, mean±SD (cm)	157.88±6.17
Weight, mean±SD (kg)	57.86±8.12
Treatment duration (missing=6), median (min-max, mo)	5 (1-38)
FIGO stage
1	81 (34.6)
2	30 (12.8)
3	90 (38.5)
4	27 (11.5)
Unknown	6 (2.6)
Grade (missing=15)
1	20 (9.0)
2	54 (24.3)
3	137 (61.7)
4	11 (5.0)
Chemotherapy regimens
Paclitaxel+carboplatin	230 (97.0)
BEP	7 (3.0)
Recurrence or metastasis
Yes	61 (25.7)
No	176 (74.3)

SD, standard deviation; min, minimum; max, maximum; FIGO, International Federation of Gynecology and Obstetrics; BEP, bleomycin+etoposide+cisplatin.

Each item in K-NFOSI-18 and K-NTX-4 was determined according to the time of occurrence (i.e., within the previous 7 days). Median total scores for first-round survey administration were 49 (range, 20 to 72) and 9 (range, 0 to 16), respectively. The most frequent symptom was nausea (100%), followed by cramping (99.6%), vomiting (99.6%), swelling of the stomach area (97.9%), feeling ill (97.9%), lack of energy (97.9%), pain (97%), constipation (96.6%), and skin problems (95.7%). Numbness or tingling sensations affected the hands (91.9%) more commonly than the feet (77.6%).

Table 2 provides a summary of structural validity, reliability, and reproducibility of K-NFOSI-18 and K-NTX-4. K-NFOSI-18 exhibited high internal consistency and reliability (Cronbach's α=0.84), with good reproducibility in re-testing: disease-related symptoms, physical (ICC=0.75); diseaserelated symptoms, emotional (ICC=0.67); treatment side effects (ICC=0.59); function/well-being (ICC=0.53); and total score (ICC=0.77). K-NTX-4 also showed high reliability (Cronbach's α=0.89) and reproducibility (ICC=0.84).

Table 2.

Internal structural validity, reliability, and reproducibility of K-NFOSI-18 and K-NTX-4 for the first-round questionnaires

Dimension	No.	Items	IIC (min-max)	IDV (min-max)	IDV (%)	Median scale scores (min-max)	Floor (%)	Ceiling (%)	Cronbach's α	ICC[a)]
K-NFOSI-18
DRS-Physical (0-36)	237	9	0.06 to 0.63	–0.04 to 0.47	88.9	24 (7 to 36)	0	1.69	0.74	0.75
DRS-Emotional (0-4)	236	1	-	-	-	2 (0 to 4)	14.41	12.71	-	0.67
Treatment side effects (0-20)	237	5	0.42 to 0.63	0.07 to 0.5	100	16 (4 to 20)	0	22.78	0.73	0.59
Function/Well-being (0-12)	237	3	0.71 to 0.86	0.09 to 0.37	100	9 (0 to 12)	5.06	18.99	0.88	0.53
Total score (0-72)	237	18	-	-	-	49 (20 to 72)	0	1.69	0.84	0.77
K-NTX-4
Neurotoxicity (0-16)	237	4	-	-	-	9 (0 to 16)	3.80	12.24	0.89	0.84

K-NFOSI-18, Korean versions of the National Comprehensive Cancer Network/Functional Assessment of Cancer Therapy (NCCN-FACT) Ovarian Symptom Index-18; K-NTX-4, FACT/Gynecologic Oncology Group (FACT-GOG) Neurotoxicity 4-item; IIC, item internal consistency; IDV, item discriminant validity; ICC, intraclass correlation coefficient; DRS, disease-related symptoms.

Calculated from second-round questionnaires (n=190).

Table 3 lists the Spearman correlation coefficients for comparisons among K-NFOSI-18, K-NTX-4, and K-EQ5D scores for the first-round survey administration. There was weak positive correlation between total K-NFOSI-18 and K-NTX-4 scores (Spearman correlation coefficient=0.348) (Table 3). K-EQ5D items showed weak-to-moderate correlation with K-NFOSI-18 and K-NTX-4 (Tables 4 and 5). K-EQ5D incorporates five dimensions with three levels, generating one continuous score (Visual Analogue Scale). Correlation coefficients were negative because lower scores imply better status in K-EQ5D, whereas lower scores imply poorer status in K-NFOSI-18 and K-NTX-4 (Tables 4 and 5).

Table 3.

Correlation between K-NFOSI-18 and K-NTX-4 scores

Dimension	NTX 1	NTX 2	NTX 3	NTX 4	K-NTX-4 total
DRS-Physical (0-36)	0.291	0.217	0.275	0.281	0.308
DRS-Emotional (0-4)	0.143	0.120	0.066	0.091	0.121
Treatment side effects (0-20)	0.340	0.266	0.314	0.222	0.330
Function/Well-being (0-12)	0.250	0.231	0.266	0.219	0.272
Total NFOSI-18 score (0-72)	0.331	0.274	0.317	0.281	0.348

Data are Spearman’s correlation coefficients for all dimension values. K-NFOSI-18, Korean versions of the National Comprehensive Cancer Network/Functional Assessment of Cancer Therapy (NCCN-FACT) Ovarian Symptom Index-18; K-NTX-4, FACT/Gynecologic Oncology Group (FACT-GOG) Neurotoxicity 4-item; DRS, disease-related symptoms.

Table 4.

Correlation between K-EQ5D items and VAS with K-NFOSI-18 scores

Dimension	DRS-Physical	DRS-Emotional	Treatment side effects	Function/Well-being	K-NFOSI-18 total
Mobility	–0.292	–0.102	–0.209	–0.320	–0.324
Self-care	–0.252	–0.091	–0.190	–0.294	–0.295
Usual activities	–0.381	–0.245	–0.354	–0.444	–0.480
Pain/Discomfort	–0.421	–0.121	–0.232	–0.250	–0.374
Anxiety/Depression	–0.351	–0.487	–0.376	–0.255	–0.434
VAS	0.510	0.357	0.301	0.313	0.500

Data are Spearman’s correlation coefficients for all dimension values. K-EQ5D, Korean EuroQoL-5 Dimension survey; VAS, Visual Analogue Scale; K-NFOSI-18, Korean versions of the National Comprehensive Cancer Network/Functional Assessment of Cancer Therapy (NCCN-FACT) Ovarian Symptom Index-18; DRS, disease-related symptoms.

Table 5.

Correlation between K-EQ5D items and VAS with K-NTX-4 scores

Dimension	NTX 1	NTX 2	NTX 3	NTX 4	K-NTX-4 total
Mobility	–0.399	–0.383	–0.352	–0.475	–0.471
Self-care	–0.237	–0.205	–0.277	–0.263	–0.287
Usual activities	–0.340	–0.370	–0.369	–0.416	–0.435
Pain/Discomfort	–0.423	–0.371	–0.409	–0.460	–0.482
Anxiety/Depression	–0.168	–0.134	–0.200	–0.128	–0.178
VAS	0.314	0.363	0.317	0.422	0.411

Data are Spearman’s correlation coefficients for all dimension values. K-EQ5D, Korean EuroQoL-5 Dimension survey; VAS, Visual Analogue Scale; K-NTX-4, FACT/Gynecologic Oncology Group (FACT-GOG) Neurotoxicity 4-item.

Discussion

In the present study, we developed reliable and reproducible Korean versions of standard NFOSI-18 and NTX-4 surveys (Cronbach's α=0.84 and 0.89, respectively; ICC=0.77 and 0.84, respectively). Furthermore, there was positive correlation between total K-NFOSI-18 and K-NTX-4 scores.

A variety of validation tools are used to measure QoL in gynaecologic cancers, but they do not always align with the specific problems of patients with ovarian cancer [10,11]. Recently, ovarian cancer trials have added QoL as a secondary end-point [12,13], so tailoring questionnaires to specific countries for international use is assuming greater importance. The clinical impact of QoL is addressed in only 30% of such queries because of methodologic issues [14], underscoring the need to adopt Korean versions of the NFOSI-18 and NTX-4.

In this study, > 95% of patients exhibited disease-related symptoms in the physical realm; these were more common than emotional symptoms. In the dimension of treatment side effects, patients consistently experienced nausea (100%) and vomiting (99.6%). However, by adding neurologic symptoms to the body of questions, we found that most patients (87%) experienced neurotoxicity as well, more often involving the hands than the feet. Although patients were excluded if they received chemotherapy within 1 month of study onset, it seems to contain a large number of patients currently on the chemotherapy course.

A distinct advantage of the present study is its large-scale, multicenter design. Furthermore, the short time required to complete the questionnaires (< 10 minutes) greatly encouraged compliance. Hence, the combined instrument (K-NFOSI-18 and K-NTX-4) may not be burdensome for patients. Although the ICC for constipation scoring in the K-NFOSI-18 was < 0.80, ICCs for most subscales and single items displayed good reproducibility. With ICC values of 0.7 or higher, two repeated test results may be expected to lie with a probability of 95%. These findings indicate that the Korean version of NFOSI-18 is a stable adaptation. In discriminant validity analysis, a result greater than 0.85, indicates that the two constructs overlap greatly and are likely measuring the same thing. In our study, IDV values were 88.9 to 100, indicating that K-NFOSI-18 and K-NTX-4 have good discriminant validity.

Previous studies have reported worse QoL in patients with disease recurrence [4,15-17]. However, other investigators have shown opposite results, generating controversy [18,19]. The present study confirmed (as expected) that patients with (vs. without) recurrence differed in terms of function/wellbeing scores. However, other dimensions, including diseaserelated symptoms and treatment side effects, showed no significant relationship with disease status.

Some study limitations are acknowledged. We did not assess changes in QoL over time, during chemotherapeutic cycles, or during multiple treatment cycles and courses. In addition, no patients hospitalized after surgery or chemotherapy were similarly surveyed to generate comparative results. Another limitation was the possibility that patients with symptoms may have been more likely to agree to participate in the study, leading to over-estimation of symptom rates. However, we chose the same inclusion criteria as those used in the study for which NFOSI-18 was first applied.

In conclusion, Korean versions of the NFOSI-18 and NTX-4 questionnaires were successfully developed and demonstrated high internal consistency and reproducibility in the test population. Few modifications were required. Further efforts are required to address their reliability and validity in clinical research.