Validation of the Comprehensive Needs Assessment Tool in Patients with Advanced Cancer.

AIMS: The 59-item Comprehensive Needs Assessment Tool (CNAT) for cancer patients is an English language survey developed in South Korea. The objective of this study was to validate the English version of CNAT in advanced cancer patients in Singapore.
METHODS: This was a cross-sectional survey where advanced cancer patients completed the CNAT in English. Confirmatory factor analysis was used to assess construct validity. For known groups validity, independent samples t-test was used to compare CNAT scores based on the Karnofsky performance status and outpatient versus inpatient setting. Cronbach's alpha was used to measure internal consistency.
RESULTS: A total of 328 advanced cancer patients were recruited. The mean age was 59.6 years and 49.1% were male. Majority (68.0%) were Chinese, 20.4% were Malay, 7.9% were Indian, and 3.7% were of other ethnicities. The 7-factor model previously established in Korea showed sufficient construct validity with root mean square error of approximation 0.037 and comparative fit index 0.944. All 59 items had a factor loading ≥0.5. Group invariance test showed no difference in the pattern of factor loadings between ethnic Chinese and other ethnic groups (P = 0.155). For known groups validity, there were significant differences in CNAT scores by performance status and outpatient versus inpatient setting. The CNAT total and factor scores showed good internal consistency with Cronbach's alpha of between 0.80 and 0.937.
CONCLUSIONS: The CNAT showed construct and known-group validity and internal consistency in this study sample and can be used to assess the unmet needs of advanced cancer patients in the Singapore context.

INTRODUCTION

Cancer is a leading cause of death and a highly burdensome disease for patients, their families, and the health-care system.[1] Previous studies indicate that patients with advanced cancer have high levels of unmet need that significantly contribute to the health-care burden at the end of life.[2] Therefore, needs assessment in these patients is important so as to identify patients requiring greater input from as well as aid service planning by health-care organizations.

The assessment of patients’ need for care is a vital step in achieving patient-centered care. However, these needs can be significantly affected by factors related to the socioeconomic-cultural setting and health-care system.[34567] Therefore, tools related to patient-reported outcomes developed in one setting may not be entirely or readily usable in another, giving rise to the need to validate assessment tools for the appropriate setting.[8]

The 59-item Comprehensive Needs Assessment Tool in Cancer (CNAT) is an English language survey for needs assessment developed and validated in South Korean cancer patients.[9] It has been used as the tool to describe the prevalence of needs in cancer patients but has not be specifically validated elsewhere.[1011] There is also a corresponding version of CNAT for caregivers and the potential for both the patient and caregiver versions to be used concurrently make the CNAT attractive.[1213] The aim of this study was to validate the CNAT in advanced cancer patients in Singapore.

METHODS

Study setting and participants

Singapore is a multiethnic society in Southeast Asia, where English is the working language and medium of instruction in schools. Most Singaporeans are bilingual in English and a mother tongue. According to the census in 2010, 74.1% of the residents are of Chinese descent.

This was a cross-sectional study. English-speaking participants were recruited from specialist outpatient clinics in the National Cancer Center of Singapore or inpatient oncology wards in Singapore General Hospital. The patient inclusion criteria were those aged 21 years or older, intact cognition as deemed by the attending physician at study entry, able to read or understand English, diagnosed with metastatic or locally advanced solid cancer that was not receiving curative treatment, Karnofsky Performance Status (KPS) >20, and able to give informed consent. This study was approved by the SingHealth Centralized Institutional Review Board (reference number 2013-082-A). Written informed consent was obtained before research participation.

Study instruments

The CNAT is a 59-item questionnaire with seven factors as follows: (1) information and education, (2) psychological problems, (3) healthcare staff, (4) physical symptoms, (5) hospital facilities and services, (6) social and religious/spiritual support, and (7) practical support. These seven factors were established by exploratory factor analysis (principal component analysis with varimax rotation).[9] Items are scored on a 4-point scale of severity according to the level of need. The original questionnaire was found to be reliable, valid, and acceptable in South Korean cancer patients.[9]

In this study, the CNAT was modified (with written permission from the CNAT developers) to replace the word “cancer” with the word “illness.” This modification was important, as it is not uncommon for patients to be unaware of their diagnosis in the Singapore context.[1415] Similar modifications have been previously reported for the adaptation and validation of the Functional Living Index-Cancer questionnaire in Singapore.[16]

Demographic and clinical data were obtained from eligible consented patients as well as medical records at the study entry. The CNAT was administered once to eligible participants by trained interviewers.

Statistical analysis

Participants rated each CNAT item according to their level of need for help – 0, (no need for help), 1 (a little), 2 (moderately), and 3 (a lot). Higher scores indicated higher levels of unmet need.[9] The factor scores were obtained by computing the mean score of items in each factor. The total CNAT score was obtained similarly by computing the mean score of all 59 items. The factor and total scores then rescaled to a 0–100 scale.

The percentage of respondents who obtained the minimum (floor) and maximum score (ceiling) were tabulated. Less than 30% of participants on the floor or ceiling were considered acceptable levels of floor or ceiling effects.[17]

To assess construct validity, we conducted confirmatory factor analysis (CFA) of the 7-factor model for the 59 items previously proposed by Shim et al.[9] We used the robust-weighted least squares method for the CFA.[18] The root mean square error of approximation (RMSEA) and Comparative Fit Index (CFI) were used for model selection and assessment of goodness-of-fit.[1920] While there is no golden rule to determine what cutoff values are optimal, we consider RMSEA 0.05 and CFI 0.95 as approximate reference values, which are often discussed in the literature.[1819] We tested group invariance in factor loadings between ethnic Chinese (n = 223) and non-Chinese (n = 105) participants.[18]

Known-groups validity was assessed based on the KPS and study setting. Independent samples t-test was used to compare CNAT factor and total scores for those with KPS 0–60 (requires assistance) versus KPS 70–100 (able to care for self), and those in the outpatient setting versus in the inpatient setting.

Internal consistency was measured using Cronbach's α. Test–retest reliability was not planned, as there may be rapid changes in functional status and psychosocial well-being due to complications from cancer progression resulting in different responses unrelated to the measurement properties of the CNAT questionnaire. Cronbach's α values of ≥0.7 were considered acceptable.[21]

Sample size considerations

For validity assessment by known-group comparison using the independent samples t-test, with group size that may vary by up to a 2:3 ratio, a total sample size of at least 275 gives 90% power, at 5% two-sided Type 1 error rate, to detect an effect size of 0.4 standard deviation (SD) in the mean difference between the two groups. Furthermore, the recommendation of Comrey and Lee[22] is that, for factor analysis, a sample size of 300 is “good.” The study sample size satisfied both criteria.

RESULTS

A total of 328 patients were recruited. The mean age was 59.6 years, 49.1% were male, and most (74.1%) were married with living with their spouse. Majority (68.0%) were Chinese, 20.4% were Malay, 7.9% were Indian, and 3.7% were of other ethnicities [Table 1].

Table 1

Demographic and clinical characteristics (n=328)

Characteristic	Frequency (%)
Age (years), mean (SD)	59.6 (10.7)
Ethnicity
Chinese	223 (68.0)
Malay	67 (20.4)
Indian	26 (7.9)
Others	12 (3.7)
Gender
Male	161 (49.1)
Female	167 (50.9)
Marital status
Single	50 (15.2)
Married and living with spouse	243 (74.1)
Married and living separately from spouse	5 (1.5)
Divorced	10 (3.1)
Widowed	20 (6.1)
Highest education level
Primary	29 (8.8)
Secondary	168 (51.2)
Postsecondary	130 (39.6)
Employed	114 (34.8)
Clinical setting
Inpatient	131 (39.9)
Outpatient	197 (60.1)
Cancer type
Lung	54 (16.5)
Breast	57 (17.4)
Colorectal	62 (18.9)
Others	155 (47.3)
Receiving chemotherapy	166 (50.6)
Receiving radiotherapy	26 (7.9)
Karnofsky performance status 70 and above (able to care for self)	198 (60.4)

SD: Standard deviation

On a 0–100 scale, the mean CNAT total score was 14.1 [Table 2]. Less than 2% of participants reached the ceiling for any of the domains. Substantial (>30%) floor effect was observed in 5 of 7 factors; the highest floor effect was seen in factor 6 “social and religious/spiritual support” (58.2%).

Table 2

Distribution of comprehensive needs assessment tool factor and total scores

	Mean (SD)	Floor score, n (%)	Ceiling score, n (%)
Factor 1: Information and education (10 items)	19.5 (22.6)	130 (21.7)	9 (1.5)
Factor 2: Psychological problems (10 items)	16.4 (21.5)	200 (33.3)	3 (0.5)
Factor 3: Healthcare staff (8 items)	11.0 (18.3)	302 (50.3)	2 (0.3)
Factor 4: Physical symptoms (12 items)	13.3 (16.6)	159 (26.5)	0 (0)
Factor 5: Hospital facilities and services (6 items)	12.9 (17.5)	238 (39.7)	2 (0.3)
Factor 6: Social and religious/spiritual support (5 items)	7.9 (14.1)	349 (58.2)	1 (0.2)
Factor 7: Practical support (8 items)	13.4 (17.4)	217 (36.2)	2 (0.3)
Total score (59 items)	14.1 (14.7)	23 (3.8)	0 (0)

SD: Standard deviation

CFA of the 7-factor model gave RMSEA 0.037 and CFI 0.944, indicating approximately sufficient fit. All 59 items had a factor loading ≥0.5. The 7-factor structure is, therefore, valid in this study. We tested for group invariance between the ethnic Chinese participants and participants of other ethnicities. The difference testing gave P = 0.155, demonstrating lack of difference in factor structure between the two groups.

When comparing participants with different KPS, there was a statistically significant difference in the CNAT total scores, 13.0 (16.2) for those with good performance status versus 20.0 (17.5) for those with poor performance status (P < 0.01) [Table 3]. The effect size was approximately 0.4 SD. There was also the good separation of scores for Factor 1 (information and education), Factor 2 (psychological problems), Factor 4 (physical symptoms), Factor 5 (hospital facilities and services), and Factor 7 (practical support). There was no significant difference in scores for Factor 3 (health-care staff) and Factor 6 (social and religious/spiritual support).

Table 3

Comprehensive needs assessment tool factor and total scores by Karnofsky performance status

	Mean (SD)		Pa
	KPS 70 and above (able to care for self), (n=198)	KPS 60 or less (needs assistance), (n=130)
Factor 1: Information and education (10 items)	20.6 (25.3)	26.8 (25.6)	0.03
Factor 2: Psychological problems (10 items)	14.7 (21.0)	22.5 (25.1)	<0.01
Factor 3: Health-care staff (8 items)	11.3 (20.5)	15.4 (21.9)	0.08
Factor 4: Physical symptoms (12 items)	11.6 (18.3)	18.4 (18.5)	<0.01
Factor 5: Hospital facilities and services (6 items)	10.5 (17.8)	19.6 (20.7)	<0.01
Factor 6: Social and religious/spiritual support (5 items)	7.8 (15.8)	10.6 (16.7)	0.13
Factor 7: Practical support (8 items)	10.1 (16.3)	20.8 (22.0)	<0.01
Total score (59 items)	13.0 (16.2)	20.0 (17.5)	<0.01

aP value from independent samples t-test. KPS: Karnofsky performance status, SD: Standard deviation

When comparing participants in the outpatient versus inpatient setting, there was a statistically significant difference in the CNAT total scores – 11.4 (13.9) for those in the outpatient setting and 22.2 (19.3) in the inpatient setting (P < 0.01) [Table 4]. The effect size was approximately 0.6 SD. There was also good separation of factor scores.

Table 4

Comprehensive needs assessment tool factor and total scores by setting

	Mean (SD)		Pa
	Outpatient setting (n=197)	Inpatient setting (n=131)
Factor 1: Information and education (10 items)	19.4 (24.3)	28.6 (26.5)	<0.01
Factor 2: Psychological problems (10 items)	13.1 (19.8)	25.0 (25.5)	<0.01
Factor 3: Health-care staff (8 items)	8.6 (17.1)	19.4 (24.7)	<0.01
Factor 4: Physical symptoms (12 items)	9.9 (15.1)	21.0 (21.3)	<0.01
Factor 5: Hospital facilities and services (6 items)	8.9 (15.3)	21.9 (22.4)	<0.01
Factor 6: Social and religious/spiritual support (5 items)	7.1 (13.5)	11.6 (19.4)	0.01
Factor 7: Practical support (8 items)	9.2 (13.9)	22.2 (23.6)	<0.01
Total score (59 items)	11.4 (13.9)	22.2 (19.3)	<0.01

aP value from independent samples t-test. SD: Standard deviation

Overall, the internal consistency was high, with a Cronbach's alpha of 0.97 for the total score. All factors had Cronbach's alpha of between 0.80 and 0.93 [Table 5].

Table 5

Cronbach’s alpha for comprehensive needs assessment tool factor and total scores (n=328)

Factor	Cronbach’s alpha
Factor 1: Information and education (10 items)	0.92
Factor 2: Psychological problems (10 items)	0.93
Factor 3: Health-care staff (8 items)	0.91
Factor 4: Physical symptoms (12 items)	0.90
Factor 5: Hospital facilities and services (6 items)	0.80
Factor 6: Social and religious/spiritual support (5 items)	0.81
Factor 7: Practical support (8 items)	0.84
Total score (59 items)	0.97

DISCUSSION

This is a validation study of the English version of the CNAT for advanced cancer patients. The results show little ceiling effect but considerable floor effect in this study population. Despite the floor effect, we found good levels of construct validity in terms of CFA results, known-groups validity in relation to performance status and setting of care, and internal consistency.

Significant floor effect was observed for a few factors and may be due to the usage of a narrow 4-point response scale. In the context of a tool that measures unmet needs, the floor effect is of lesser concern clinically as needs-based interventions aim to target those with higher CNAT scores indicating more unmet needs. However, the significant floor effect could have suppressed the difference in CNAT scores by KPS – the two factors with the highest floor effect were the same two factors that did not have significant separation of scores by KPS. The minimal ceiling effect observed is a desirable property as a change from a high baseline level of unmet need can then be monitored.

Factor 6 (social and religious/spiritual support) had a particularly high floor effect with 58.2% endorsing the floor score. This may be due to the lower prevalence of problems and unmet needs with regard to spiritual issues in Asian countries compared to Western countries.[23] Dimensions of spirituality may also have varying salience in different cultures. In Asian culture, spirituality may be experienced more in terms of making sense of one's situation and role rather than as a matter of one's relationship with God.[2425] For example, a study on spiritual needs of cancer patients in China found that “to be accepted” and “to take responsibility” were more important than “to talk to someone,” “to pray,” and “to feel God with me.”[26]

Factor 3 (health-care staff) also had a high floor effect with 50.3% endorsing the floor score. The items in this factor related to the interactions with doctors and nurses, for example, to be “respected and treated as a person by my doctor” and to have “nurses to promptly attend to my discomfort and pain.” Having a low floor score may indicate that these aspects of professional care are routinely provided as part of standard care. Nonetheless, it is still a domain that is important to include in a comprehensive assessment of unmet needs, to identify and promptly address any gaps, where present.

A limitation of this study is that it was a cross-sectional study, and thus, we were unable to assess sensitivity to change or test–retest reliability. A longitudinal study was not done as our target sample of patients with advanced cancer were likely to have rapidly deteriorating functional status, and consequently, a high drop-out rate for a follow-up assessment. The low levels of unmet needs expressed by participants also make validity assessment difficult. Future studies may explore a shorter version of the CNAT, which will increase the feasibility of using this tool in the clinical setting.

CONCLUSION

The CNAT showed construct and known-group validity and internal consistency in this study sample and can be used to assess the unmet needs of advanced cancer patients in the Singapore context.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.