The Palliative Care Outcome Scale: Turkish Validity and Reliability Study.

OBJECTIVE: To examine the validity and reliability of the Turkish version of the Palliative Care Outcome Scale (POS).
METHODS: This methodological study consisted of 69 patients hospitalized in the palliative care (PC) service of three hospitals between June 2016 and August 2016, 69 carers who undertook continuously primary care of these patients, and 28 staff members working in the PC service and providing care to these patients. The data of the study were collected using the Personal Diagnosis Form, the POS. The scope, structure and criterion validity and internal consistency reliability of the scale were tested. Item analysis, Cronbach's alpha analysis, content validity ratio, confirmatory factor analysis for construct validity, criterion validity, patient validity, and intraclass correlation coefficient for the adaptation of outcomes of patient, carer, and staff groups were conducted.
RESULTS: The content validity ratio of the scale was found to be higher than 0.80 for patient, carer, and staff questionnaires. Item-total score correlation coefficients were determined between 0.27 and 0.72 for the items in the patient questionnaire, 0.33-0.67 for the carer questionnaire, and 0.34-0.72 for the staff questionnaire. The Cronbach's alpha reliability coefficients were determined as 0.64 for the patient questionnaire, 0.73 for the carer questionnaire, and 0.68 for the staff questionnaire.
CONCLUSIONS: The Turkish version of the POS was determined to be a valid and reliable tool to be used for assessing the needs of PC patients in three dimensions in terms of the perspectives of patient, carer, and staff. Copyright:

Introduction

Palliative care (PC) is a relatively new specialty that has evolved during the last five decades. Each year, an estimated 40 million people need PC. Seventy-eight percent of these people reside in low- and middle-income countries.[1] There are many advantages of PC. Patients and their families have reported an enhanced quality of life. Moreover, distressing symptoms such as pain, depression, and spiritual distress have been better controlled.[234]

As a result of the increase of PC centers, the necessity of evaluating care outcomes has arisen. Outcome measurement is a method used for measuring the changes in a patient's current and future health status that can be associated with the preceding healthcare. Outcome measurement involves the use of a measure to establish a patient's baseline health status and then assesses the changes over time against that baseline status. Outcome measures allow us to record these changes and enable us to measure the structure, process, and output of care.[5]

Having been translated into more than 12 languages, the Palliative Care Outcome Scale (POS) is frequently used in PC units.[678910] The POS was developed in 1999 for advanced stage cancer patients.[1112] It is used not only for cancer patients but also for patients with HIV/AIDS and chronic diseases and not only with patients but also with carers and staff members.[1213] It has been used as a research tool and for informing clinical practice, monitoring service interventions, and assessing and enhancing the quality of care.[1214] A symptom module (the POS-S) was developed as an addition to the POS to describe the effect of PC-specific symptoms.

The number of PC centers is increasing in Turkey. However, these centers do not have a valid and reliable scale to evaluate patient outcomes. The POS has three versions. These three versions are all guided by clinical experiences and patients' general condition. The POS enables two types of scores to be generated. First, scores can be related to individual question items. Individual item scores can enable staff members to monitor change over time and focus on particular patients. Second, scores can be summarized. The summary score is generated by the total score possible, which can only be generated when interventions have not made any significant impact.

There is no published study using a Turkish POS as an assessment tool in Turkey. The aim of this study is to investigate the validity and reliability of the Turkish version of the POS.

Methods

In this study, the POS and the POS-S were adapted into Turkish, and their validity and reliability were methodologically assessed.

The population of this study was composed of the PC patients receiving treatment in three hospitals and the carers and staff members providing care for these patients. The study included 69 patients who were conscious and receiving treatment in a nonintubated PC unit. In addition, 69 carers who carried out the continuous primary care of the patients and 28 staff members working in PC service and providing care for these patients were included in the study. In previous validity and reliability studies, it was recommended for the sample size to be 5–10 times greater than the total number of scale items.[15] There are 10 items in the POS, and the aim was to reach five times the number of patients, carers, and staff members.

The personal identification form, developed by the researcher, the POS, and the – Palliative care Outcome Scale- Symptom Scale for Palliative Care (POS-S) were used to collect the data.

The personal identification form was created separately by the researcher for the patients, the carers, and the staff members based on the literature. The personal identification form for staff members is an information form that includes questions about personal characteristics and knowledge about PC. The personal identification form for patients is composed of 11 questions about the personal characteristics of the patients, characteristics related to the disease process, and knowledge about PC. The personal identification form for the carer composed of 10 questions, including information about personal characteristics and knowledge about care.

The POS is used to measure outcomes by evaluating many important and necessary requirements in the PC of advanced stage patients. The POS was developed by Hearn and Higginson in 1999. Each item is scored using a 5-point Likert scale. The 10 items consist of questions such as physical symptoms of the patient, psychological needs (emotional and spiritual) and determining information and practical support. The assessment tool also has two questions, one of which is open ended, evaluating the performance status of the patient. The POS scale has 3 versions: patient, carer, and healthcare professional. A high score on the scale indicates high PC needs.[11]

POS-S is an assessment tool composed of 11 items that evaluate the patient's severity of symptoms. Symptom severity was measured using a five-point Likert scale, with each item rated from “0” meaning “none” to “4” meaning “very severe.”[7914]

Data collection

The data for this study were collected between June and August 2016 in the PC units of three hospitals.

Statistical analysis

In the study, the number, percentage, mean, and standard deviation were used for the descriptive characteristics of the participants and the descriptive statistics of scale scores. P < 0.05 was considered as statistically significant.

Pearson correlation analysis (reliability analysis) for item analysis (item-total score analysis), Cronbach's alpha analysis (reliability analysis) for scale internal consistency, confirmatory factor analysis (CFA) (validity analysis) for construct validity, Pearson correlation analysis (validity analysis) for criterion validity, intraclass correlation coefficient (ICC) for the adaptation of the outcomes of patient, carer, and staff groups were used. Repeated measures analysis of variance was used (advance analysis: Bonferroni test) to compare the POS mean scores of the three groups.

Ethical approval

To conduct this study, written permission was obtained from the Research Project and Coordination Assistant on behalf of the website “A resource of Palliative Care” (https://pos-pal.org/) from which the scale was taken. Approval from the Ethics Committee was obtained to conduct this study (Approval No. 2016-4/19).

Results

Of the 69 patients, 34 (49.3%) were female, and 35 (50.7%) were male. The mean age of the patients was 69.28 ± 14.34 years. Of the patients, 34 (49.3%) were primary school graduates, and 55 (79.7%) were married. Thirty-seven (53.6%) of the patients had cancer, 15 (21.7%) had neurological diseases, 9 (13%) had respiratory diseases, and 8 (7.2%) were hospitalized due to other disease diagnoses. The mean duration of hospitalization of the patients in the PC unit was 12.38 ± 14.07 years.

Of the 69 carers participating in the study, 51 (73.9%) were female, and 18 (26.1%) were male. The mean age of the carers was 47.45 ± 13.00 years.

Fifty-nine (85.5%) of the staff members were female, and 25 (36.2%) were married. The mean age of the staff members was 27.43 ± 5.57 years. Of the staff members, 49 (71.0%) had a bachelor's degree, and 5 (7.2%) had a graduate degree. The average working duration of the staff members in the PC profession was 4.57 ± 5.22 years. A total of 64 (92.8%) of the staff members were working in PC for the last year.

Language and content validity

The scale was translated into Turkish by three linguists who had a good command of English. The final version of the scale items, which was formed by selecting the most appropriate expressions from the Turkish translations, was backtranslated into English by a linguist whose mother tongue was Turkish and who was informed about this subject in detail. The researchers made the necessary revisions by selecting the most appropriate expressions from the Turkish translation of the scale, and the finalized scale items were presented to the three experts for their opinions. The experts were given both the original version and the translated version of the scales and asked to score the items between 1 and 4 (1 = many changes are needed, 4 = very appropriate) to determine their appropriateness. The items were then revised based on the feedback from the experts. The Turkish version was then backtranslated into English by a linguist who was fluent in both Turkish and English. The content validity examines how much the scale covers the basic elements of the structure requested to be measured. The content validity ratio was developed by Lawshe (1975).[16] According to the results, content validity ratio of the scale items was found to be between 0.80 and 1.00.

Pilot testing

Pilot testing of the scale, language, and scope validity was conducted with five patients, 5 carers, and 5 staff members who were not included in the study. Following the pilot study, any required changes were made in line with the feedback provided by the participants, and the items took their final form.

Descriptive statistics of the Palliative Care Outcome Scale items

Table 1 shows the lowest and highest scores obtained from the patient, carer, and staff POS items and the mean score and standard deviation of each item.

Table 1

Descriptive statistics of the palliative care outcome scale items

Items	Patient score (n=69)			Carer score (n=69)			Staff score (n=28)
	Minimum-maximum	x̄±SD	If the item is deleted α	Minimum-maximum	x̄±SD	If the item is deleted α	Minimum-maximum	x̄±SD	If the item is deleted α
1. Pain	0-4	1.68±1.40	0.61	0-4	1.86±1.43	0.69	0-4	1.16±0.93	0.63
2. Other symptoms	0-4	1.39±1.41	0.60	0-4	1.51±1.37	0.69	0-4	1.06±1.03	0.65
3. Patient anxiety	0-4	1.26±1.31	0.55	0-4	1.32±1.43	0.68	0-4	1.19±1.09	0.61
4. Family anxiety	0-4	1.39±1.48	0.59	0-4	1.70±1.55	0.69	0-3	1.22±1.03	0.67
5. Information	0-4	1.55±1.38	0.62	04	1.42±1.39	0.72	0-4	0.67±0.93	0.68
6. Support	0-4	1.03±1.37	0.67	0-4	1.45±1.62	0.74	0-4	1.17±1.10	0.66
7. Life worthwhile	0-4	1.03±1.14	0.62	0-4	1.09±1.18	0.71	0-3	0.90±0.88	0.64
8. Self-worth	0-4	2.09±1.23	0.61	0-4	1.93±1.38	0.68	0-3	1.38±0.93	0.66
9. Wasted time	0-2	0.12±0.47	0.64	0-2	0.26±0.68	0.72	0-4	0.26±0.74	0.68
10. Personal affairs	0-4	0.16±0.68	0.64	0-4	0.35±1.03	0.73	0-4	0.83±1.44	0.71
-	0-27	11.70±5.97		0-27	12.87±7.18		0-22	9.83±5.22
Total score	α=0.64			α=0.73			α=0.68

SD: Standard deviation

Reliability analysis of the Palliative Care Outcome Scale

Item scores and total score correlations of the POS translated into Turkish and total score correlations were evaluated via Pearson correlation analysis [Table 2].

Table 2

Item-total score correlations of the palliative care outcome scale

Items	Item-total score correlations coefficients
	Patient (n=69)		Carer score (n=69)		Staff score (n=28)
	r	P	r	P	r	P
1. Pain	0.55	0.000	0.63	0.000	0.62	0.000
2. Other symptoms	0.58	0.000	0.62	0.000	0.58	0.000
3. Patient anxiety	0.72	0.000	0.66	0.000	0.72	0.000
4. Family anxiety	0.60	0.000	0.63	0.000	0.48	0.000
5. Information	0.49	0.000	0.47	0.000	0.40	0.001
6. Support	0.29	0.015	0.41	0.000	0.51	0.000
7. Life worthwhile	0.46	0.000	0.52	0.000	0.58	0.000
8. Self-worth	0.53	0.000	0.67	0.000	0.48	0.000
9. Wasted time	0.29	0.018	0.45	0.000	0.34	0.004
10. Personal affairs	0.27	0.025	0.33	0.006	0.45	0.000

r: Pearson correlation analysis

When examining item-total score correlations of the items for the reliability of the POS (Patient Questionnaire) results, it was determined that the correlation reliability coefficients of 10 items (Pearson correlation/Pearson product-moment correlation) were between r = 0.27 and 0.72, positive, and statistically significant [P < 0.05 for items 6, 9, and 10; P < 0.001 for the others, Table 2].

When examining item-total score correlations of the items for the reliability of the POS (Carer Questionnaire) results, it was determined that the correlation reliability coefficients of 10 items were between r = 0.33 and 0.67, positive, and statistically significant [P < 0.01 for the item 10; P < 0.001 for the others, Table 2].

When examining item-total score correlations of the items for the reliability of the POS (Staff Questionnaire) results, it was determined that the correlation reliability coefficients of 10 items were between r = 0.34 and 0.72, positive, and statistically significant [P < 0.01 for items 5 and 9; P < 0.001 for the others, Table 2].

In the internal consistency analysis, which is one of the reliability indicators of the Turkish version of the POS, the Cronbach's alpha reliability coefficients were α = 0.64 for the patient questionnaire, α = 0.73 for the carer questionnaire, and α = 0.68 for the staff questionnaire [Table 1]. Since omitting the items with low factor loadings (path coefficients) in the CFA did not cause any significant change, there was no need to omit any item.

The alpha reliability coefficients of the overall scale when an item was omitted from the structure of the measurement tool were α: 0.55–0.67 for the patient group, α: 0.68–0.74 for the carer group, and α: 0.61–0.71 for the staff group [Table 1]. Since omitting the items with low factor loadings (path coefficients) in the CFA did not cause any significant change, there was no need to omit any item.

Comparison of the Palliative Care Outcome Scale results for patients, carers, and staff members

One-way variance analysis (post hoc analysis: Bonferroni test) was used in the dependent groups for the mean scores' comparison of the patients, the carers, and the staff members for the POS [Table 3].

Table 3

Comparison of palliative care outcome scale mean scores of the patient, carer, and staff

Group	x̄±SD	F	P	ICC	P
Patient	11.70±5.97
Carer score	12.87±7.18	5.404	0.007	0.70	0.000
Staff score	9.83±5.22	Staff score < carer score

F: Repeated measures analysis of variance: Bonferroni test. Standard Deviation : 2. ICC: Intraclass correlation coefficient, SD: Standard deviation

When the mean scores of the patients, carers, and staff members for the POS were examined, it was determined that there was a very significant difference among the groups [P < 0.001, Table 3]. In the advanced analysis, the mean scores of the patient needs obtained from the diagnosis of staff members were significantly lower than the mean scores of the patient needs diagnosed by carers (P < 0.05), and the difference between the other pairwise groups was not significant (P > 0.05).

When the concordance among the POS mean scores from the three groups was examined with an ICC (two-way random effect, consistency model), the concordance coefficient among the scores of the patients, carers, and staff members was found to be 0.70 and very significant [P < 0.001, Table 4].

Table 4

Confirmatory factor analysis compatibility values of the palliative care outcome scale (s=69)

CFA fit values	Patient	Carer score	Staff score
Chi-square test value/P	40.69/0.17 (P>0.05)	44.21/0.14 (P>0.05)	40.24/0.25 (P>0.05)
Degrees of freedom	35	35	35
Chi-square test value: Degrees of freedom	40.69/35=1.16	44.21/35=1.26	40.24/35=1.15
RMSEA/P	0.059 (P<0.05)	0.062 (P<0.05)	0.047 (P<0.05)
SRMR	0.087	0.085	0.083
CFI	0.93	0.95	0.93
NNFI	0.90	0.94	0.91
GFI	0.89	0.88	0.89
AGFI	0.82	0.82	0.83

CFA: Confirmatory factor analysis, RMSEA: Root Mean Square Error of Approximation, SRMR: Standardized Root-mean-Square Residual, CFI: Comparative Fit Index, NNFI: Non-Normed Fit Index, GFI: Goodness of Fit Index, AGFI: Adjusted Goodness of Fit Index

According to the performance states of the patients who were not scored in the study, 9 (13.0%) of the patients stated they completed the scale by themselves, 19 (27.5%) with the help of a friend or relative, and 41 (59.4%) with the help of a member of the care team. Fourteen (20.3%) of the carers expressed the activity status of their patients as limited, 8 (11.6%) as outpatient, 25 (36.2%) as limited in care, and 22 (31.9%) as entirely dependent. The staff members stated that 5 (7.2%) of their patients were completely active, 28 (40.6%) had limited activity, 7 (10.1%) were outpatient, 20 (29.0%) were limited in care, and 9 (13.0%) were entirely dependent.

Validity analysis of the Palliative Care Outcome Scale

To confirm the compatibility of the Turkish version of the POS, CFA was performed, and CFA fit values are presented in Table 4. The path coefficients (factor loadings) of all items for their own factor were found to be between 0.09 and 0.68 in the patient questionnaire, between 0.16 and 0.67 in the carer questionnaire, and between 0.15 and 0.85 in the staff questionnaire.

For the criterion validity of the POS, the correlation between the scale scores and the scores of POS-S measured simultaneously was examined by Pearson correlation analysis.

A strong, positive, and statistically very significant correlation was determined between the total score from the POS and scores of the POS-S (P < 0.001). As the POS scores increased, the POS-S scores increased.

The compliance of patient, carer, and staff POS results is given in Table 5.

Table 5

Compliance of patient, carer, and staff palliative care outcome scale results

POS	S (%)*			ICC	P	Deal rate in 1 score (%)
	Patient (n=69)	Carer (n=69)	Staff (n=28)
1. Pain	19 (27.5)	21 (30.4)	6 (8.6)	0.58	0.001	7 (10.1)
2. Other symptoms	15 (21.7)	16 (23.1)	7 (10.1)	0.65	0.001
3. Patient anxiety	16 (23.1)	19 (27.5)	12 (17.4)	0.57	0.001
4. Family anxiety	20 (29.0)	24 (34.8)	12 (17.4)	0.49	0.001
5. Information	23 (32.3)	19 (27.5)	1 (1.4)	0.65	0.001	23 (32.3)
6. Support	11 (15.9)	21 (30.4)	9 (13.0)	0.43	0.003
7. Life worthwhile	9 (13.0)	8 (11.6)	5 (7.2)	0.56	0.001
8. Self-worth	30 (43.5)	25 (36.2)	10 (14.5)	0.43	0.001
9. Wasted time	4 (5.8)		1 (1.4)	0.40	0.006
10. Personal affairs	2 (2.8)	4 (5.8)	10 (14.5)	0.30	0.029

*Percentage (high/very high) of 3 and 4 points for items. POS: Palliative care outcome scale, ICC: Intraclass correlation coefficient

Discussion

The POS includes questions that have been used effectively in other outcome measurement scales. The items measure physical and psychological symptoms; spiritual, practical and emotional concerns; and psychosocial needs.

Generally, individual POS item scores of zero or one require less clinical attention than items that score three or four. For example, if a patient scores a four for question one when rating their pain, the patient is reporting pain that is overwhelming to them and hinders their ability to think of anything else.

In Turkey, there is no valid and reliable scale that evaluates the results of PC in three dimensions by patients, carers, and staff members. In this study, the POS and the POS-S, which were adapted into Turkish, were found to be valid and reliable measurement tools.

Discussion of reliability results

Reliability shows how accurately a scale measures the property it wants to measure as well as the productivity and continuity of the scale. There are many methods used to assess reliability.[17] In this study, the determination of Cronbach's alpha coefficient and correlation analysis were used.

Based on the literature, a measurement tool is not reliable if the alpha coefficient from the internal consistency reliability analysis is lower than 0.40, has a low reliability if the alpha is between 0.40 and 0.59, is quite reliable if the alpha is between 0.60 and 0.79, and is highly reliable if the alpha is between 0.80 and 1.00.[18] In this study, the scale was found to be quite reliable according to the literature.[7911]

The level of item-total score correlations in item selection is an important criterion.[1819] The reliability of the item-total score of these scale items is at an adequate level. Item correlations are similar to the original version of the scale and its versions applied in other countries.[72021]

Since omitting the items with low factor loadings (path coefficients) in the CFA did not cause any significant change, there was no need for omitting any item.

When the concordance between the POS mean scores from the three groups was examined with an ICC (two-way random effect, consistency model), the concordance coefficient between the scores of the patients, carers, and staff members was found to be 0.70 and very significant.

According to the identification of performance states of the patients who were not scored in the study, 9 (13.0%) of the patients filled out the scale by themselves, 19 (27.5%) with the help of a friend or relative, and 41 (59.4%) with the help of a member of the team. Fourteen (20.3%) of the carers expressed the activity status of their patients as limited, 8 (11.6%) as outpatient, 25 (36.2%) as limited in care, and 22 (31.9%) as entirely dependent. The staff members stated that 5 (7.2%) of their patients were completely active, 28 (40.6%) had limited activity, 7 (10.1%) were outpatient, 20 (29.0%) were limited in care, and 9 (13.0%) were entirely dependent.

Discussion of validity results

Different methods are used in the validity analysis of the scales. Some of these methods included construct validity, simultaneous criterion validity, content validity, predictive validity, and face validity.[17] The construct validity indicates the ability of the scale to measure the related concept or the whole conceptual structure. One of the most commonly used methods for assessing construct validity is factor analysis. Factor analysis is a process conducted to evaluate whether the items in the scale are collected under different dimensions.[19]

Construct validity of the Palliative Care Outcome Scale: Confirmatory factor analysis

For an item to measure a structure or factor well, the factor loading should be 0.30 or above.[22] In the CFA, the goodness of fit statistics should be at the desired level. In this study, the fact that the Chi-square results of the patient, carer, and staff questionnaires of the POS adapted into Turkish was not significant according to the result of the CFA, and the value obtained by dividing the Chi-square value into the degree of freedom was <2 which showed that the scale structure was a good model.

Simultaneous criterion validity of the Palliative Care Outcome Scale: Concurrent validity

There was a strong, positive, and statistically significant correlation between the total score from the POS and the scores from the POS-S. The PC symptom scores of the patients increased as the POS score increased. The presence of a significant correlation between the POS scores and the POS-S scores of the patients indicated that the results were compatible and that the Turkish scale was valid in measuring the desired characteristic.

The r value of the correlation coefficient should be ≥0.60, and the ICC should be at least 0.70. ICC values between 0.70 and 0.84 show moderate fit, values between 0.85 and 0.94 show high fit, and values between 0.95 and 1.00 show perfect fit.[23] According to the average difference, there is no exact similarity between the results of the three groups in this study. While the patient and carer results were close to each other, the results of the staff members were lower. According to the ICC results, there was a moderate fit among the three group results, and the fit between the groups was 70%.

Conclusion

The Turkish POS is a valid and reliable tool that can be used with patients, carers, and staff members in three dimensions for the evaluation of patients' needs. The scale needs to be tested in larger sample groups, and further studies are required. For evaluating PC results of the scale and enhancing the quality of care, its use is recommended in all PC centers and in oncology, neurology, and cardiology services where PC patients are frequently followed.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.