Palliative Care Quiz for Nurses-Indonesian Version (PCQN-I): A Cross-cultural Adaptation, Validity, and Reliability Study.

AIM: The study is aimed to perform a cross-cultural adaptation of the palliative care quiz for nurses in an Indonesian context to evaluate its psychometric properties.
METHODS: A two-phase design was used in this study, including cross-cultural adaptation and psychometric testing. The PCQN-I was tested on a convenience sample of health-care providers recruited from 20 primary health-care (PHC) centers in Yogyakarta, Indonesia, from July to October 2017.
RESULTS: The cross-cultural adaptation ultimately resulted in a total of 20 items from the PCQN-I. An expert committee revised the words in Item 4 from "adjuvant therapies" to "adjuvant therapies/additional therapies," in Item 5 from "bowel regimen" to "gastrointestinal therapies," and Item 16 from "Demerol" to "Pethidine" based on the Indonesian context. The experts retained Item 5 "It is crucial for family members to remain at the bedside until death occurs," although none of the participants (n = 40) involved in the pilot testing could answer it correctly, and Item 19 "The loss of a distant or contentious relationship is easier to resolve than the loss of one that is close or intimate" although few participants responded correctly. A total of 150 health-care providers consisting of 100 nurses and 50 physicians were involved in the psychometric testing. The scale-level content validity index/Ave of the PCQN-I was 97 and 93 in terms of its relevancy and clarity, respectively. The reliability was calculated as the Kuder-Richardson formula 20 score was 0.71.
CONCLUSIONS: The PCQN-I is a valid and reliable instrument for assessing palliative care knowledge among PHC providers. Further research is warranted to examine its reliability in different clinical settings. Copyright:

INTRODUCTION

Palliative care is related to approaches used to relieve the physical, psychological, and spiritual aspects of suffering and improve the quality of life of patients and their families.[1] To provide palliative care, health-care providers require fundamental knowledge of care.[2] Palliative care knowledge has been identified as an essential component when dealing with palliative patients and families.[34] A lack of knowledge will lead to failure in terms of the identification of needs, insufficient pain management, and inadequate psychological and spiritual support.[5678] Therefore, identification of what knowledge is critical to palliative care is important among health-care providers.[910]

The palliative care quiz for nurses (PCQN) has been extensively used to measure palliative care knowledge and has been adapted into several languages.[111213] The PCQN measures the cognitive understanding of palliative care, including the philosophy and principles of palliative care, management of pain and other symptoms, and provision of psychosocial and spiritual care to individuals and families.[14] The PCQN is an easy tool that has been used in various settings, such as hospitals,[1516] cancer centers,[15] communities,[17] pediatric medical services,[18] long-term care institutions,[1920] and nursing homes.[4]

Although palliative care was established in 1992 in Indonesia,[21] it is still in its infancy.[22] The development of palliative care has been very slow, which has been attributed to factors such as government policies, a culture of strong familial support, a lack of palliative care education, and a lack of support from regional and local governments.[2324] Furthermore, the concept of palliative care has not actually been clearly understood by some health-care providers.[23] Evaluating palliative care knowledge among health-care providers could help identify the misconceptions associated with palliative care and educational needs. However, there is a lack of instruments that have been developed and validated in an Indonesian context to examine palliative care knowledge among health-care providers. Therefore, this study was conducted to perform a cross-cultural adaptation of the PCQN into the Indonesian context and to evaluate its psychometric properties.

METHODS

A two-phase design was used, including (1) a cross-cultural adaptation of the PCQN using a six-step process and (2) psychometric testing, including an evaluation of the validity and reliability of the PCQN-I.

Settings and samples

This study was conducted from July to October 2017 at 20 primary health-care (PHC) centers in Yogyakarta, Indonesia. A convenience sampling method was used to recruit participants. Regarding the inclusion criteria, we included all registered nurses and physicians working in the above mentioned 20 PHC centers. In terms of the exclusion criteria, registered nurses and physicians (PHC providers) who were unable to provide their consent or had taken a leave of absence (e.g., annual leave, maternal leave) during the data collection period were excluded from the study.

Instruments

The PCQN is a 20-item instrument with true, false, or “don't know” options used to evaluate palliative care knowledge. It was originally developed by Ross et al.[14] and comprises the following three domains: philosophy and principles of palliative care (items 1, 9, 12, 17), pain and symptom management (items 2–4, 6–8, 10, 13–16, 18, 20), and psychosocial and spiritual care (items 5, 11, 19). A correct answer receives one point. The total score ranges from 0 to 20, with higher scores indicating better palliative care knowledge. The content validity has been found to be acceptable, and the internal consistency is also high (Kuder–Richardson 20 = 0.78).[14]

Permission to conduct a cross-cultural adaptation of the PCQN into the Indonesian context and use of the instrument for assessing palliative care knowledge among physicians and nurses was obtained from the University of Ottawa. In addition, details regarding the participants' background information, including age, gender, level of education, working experience, palliative care experience, and professional education in palliative care, were collected. Palliative care experience consisted of participants' experience related to providing palliative care and experience with caring for terminally ill patients. Professional education in palliative care included formal palliative care education and continuing education in palliative care.

Procedures and data collection

Phase I: Cross-cultural adaptation

A cross-cultural adaptation was applied to the PCQN based on a well-established guideline.[25] This process integrated translation and cultural adaptation issues for the purpose of preparing an acceptable instrument to use in Indonesian settings. The following six steps were involved in the assessment of cross-cultural adaptation: forward translation, synthesis, backward translation, expert committee review, pretesting, and instrument appraisal.[25]

In the forward translation step, a nurse and a physician whose first language was Bahasa Indonesia and who were also fluent in English were invited to be translators for the purpose of translating the instrument from English into Bahasa Indonesia. The outcome of this step was two versions of translation in Bahasa Indonesia (T1 and T2). Moreover, the forwarding translation outcome (T1 and T2) was synthesized by the researcher (Nuzul Sri Hertanti [NSH]) and the above-referenced translators to produce a draft of the Indonesian version of the instrument (T-12).[25]

Next, two translators with English as their first language and who were fluent in Bahasa Indonesia were invited to conduct a backward translation for the purpose of translating the instrument from Bahasa Indonesia into English. The two translators were unaware of the original English version. This step was a process to validate whether the Indonesian version (T-12) consistently reflects the meaning of the original instrument.[25] The outcome of this step was two back-translations (BT1 and BT2).

The fourth step was a review by an expert committee consisting of seven experts, including two nurses with experience in palliative care, a palliative care physician, two nursing faculty members with specialties in palliative care, a PHC physician, and an anesthesiologist. The committee members reviewed the original instrument (PCQN) and all the translations (T1, T2, T-12, BT1, and BT2) in terms of semantic, idiomatic, experiential, and conceptual equivalence. At the end of the review, an expert committee provided suggestions, reached a consensus on any discrepancies, and produced the prefinal version of the instrument.[25]

Step five was pilot testing the prefinal version of the instrument. A total of 40 health-care providers working in 10 PHC centers in Yogyakarta completed the prefinal version of the PCQN-I and provided their comments on whether they understood the items.

In the final step, the researcher and the expert committee rediscussed the prefinal version of the instrument after the completion of the pilot testing. The experts suggested to change some words in Items 4, 8, 16, and 17 to overcome the difficulty among participants in understanding the words. In addition, the expert committee suggested to retain Items 5 and 19, although the majority of participants had misunderstanding these items due to cultural considerations. At the end of this step, the discussion appraised and finalized 20 items of the PCQN-I.[25]

Content validity

In this study, the content validity index (CVI) was calculated to assess content validity. A cover letter and the PCQN-I finalized in phase I were included with the content validity survey explaining why experts were invited to participate, along with clear and concise instructions on how to rate each item. Six experts were invited to participate, including two palliative care nurses, a physician with expertise in palliative care, two nursing lecturers with expertise in palliative care, and a primary care physician. The experts were asked to assess the relevancy and clarity of each item using a four-point scale (1 = not relevant/not clear, 2 = somewhat relevant/item requires some revision, 3 = quite relevant/clear but requires minor revision, and 4 = highly relevant/very clear).[2627] The item-level CVI (I-CVI) and scale-level CVI/Ave (S-CVI/Ave) were calculated.[26] To obtain the I-CVI, the number of experts who gave rating scores of either 3 or 4 was divided by the total number of experts, which ranged from 0.00 to 1.00.[2627] The S-CVI/Ave was calculated by averaging the I-CVIs.[26] At least an I-CVI of 0.78 was considered to be an acceptable value, whereas an S-CVI/Ave with an average value approaching 0.90 or higher was considered to indicate excellent content validity.[2628]

Reliability

The validated PCQN-I was also evaluated for reliability using a convenience sample of health-care professionals recruited from 20 PHC centers in Yogyakarta Province, Indonesia. The research team recruited two research assistants (RAs) to assist with data collection. The researcher (NSH) contacted the head of each PHC center and provided information related to the research after obtained the research permission from the local Government and ethical approval from Universitas Gadjah Mada. Then, the head of each PHC center provided the list of registered nurses and physicians. NSH and RAs selected the eligible PHC providers and visited the PHC centers to approach them for data collection. The research aims, study requirements, and participant rights were provided when approaching potential participants. After obtaining signed consent forms from the eligible participants, they were provided with an anonymous study instrument and asked to complete it. The completed instruments were returned to a box for collection by the research team.

Statistical analysis

The data comprising the participants' background information were analyzed using the Statistical Package for the Social Sciences (SPSS) for Windows, version 17.0 (SPSS Inc., Chicago, Ill., USA). Categorical data were represented as frequency and percentage, whereas continuous data were represented as mean and standard deviation (SD).

To assess reliability, the Kuder–Richardson formula 20 (KR-20) was used to determine the internal consistency of the PCQN-I with a dichotomous variable (i.e., responses that were correct or incorrect). Reliability coefficients >0.50 were considered to be acceptable.[29]

The difficulty index was calculated to determine the percentage of participants answering an item correctly.[1426] This was calculated by dividing the total number of participants who answered an item correctly by the total number of participants giving the answer on that item.[1226] A difficulty index score ranging from 0.20 to 0.80 indicated that the item could be correctly answered by a reasonable number of individuals.[1426] Difficulty indexes <0.20 and >0.80 indicated that the item was very difficult and very easy, respectively. Furthermore, the discrimination index was calculated to determine whether the items were good quality (discrimination index >0.30) or poor quality (discrimination index <0.10 or negative value).[14] This was determined by subtracting the proportion of the lower quartile of the participants who obtained the lowest scores from the proportion of the upper quartile of the participants who obtained the highest scores of the PCQN-I.[1214]

Ethical consideration

This study was approved by the Medical and Health Research Ethics Committee of Universitas Gadjah Mada, Yogyakarta, Indonesia (Ref. KE/FK/0926/EC/2017). The research permit was obtained from the Government Office of Yogyakarta, Indonesia (Ref. 074/7091/Kesbangpol/2017). Before data collection, the participants received a letter outlining the study, along with an informed consent form. The participants provided written consent indicating their agreement participate in the study, but they still were permitted to withdraw from the study at any time without penalty. Their names and addresses were kept confidential and did not appear on the computerized data. Only study ID numbers were recorded.

RESULTS

Cross-cultural adaptation

The cross-cultural adaptation process was performed systematically. The first, second, and third steps were conducted without any difficulties. In the fourth step, the semantic, idiomatic, experiential, and conceptual equivalences of the PCQN-I were 89.8%, 94.6%, 94.2%, and 90.6%, respectively. In the fifth step, a total of 40 health-care providers, including 14 physicians (35%) and 26 nurses (65%), with a mean age of 40 years, completed the prefinal version of the PCQN-I. The time necessary to complete the instruments ranged from 7 to 10 min. The mean score of the prefinal version of the PCQN-I was 6.8 of a possible score of 20. All participants (n = 40) in the pilot study answered Item 5 “It is crucial for family members to remain at the bedside until death occurs” incorrectly. Furthermore, only one participant could answer Item 19 “The loss of a distant or contentious relationship is easier to resolve than the loss of one that is close or intimate” correctly. Approximately 70% of the participants (n = 27) provided comments regarding their inability to understand some of the words in Items 4, 8, 16, and 17. In the sixth step, the expert committee identified the difficult words these items and reached a consensus [Table 1]. The expert committee also discussed items 5 and 19 and retained these items due to cultural considerations. Thus, the cross-cultural adaptation process was finalized, resulting in retaining a total of 20 items in the PCQN-I [Table 2].

Table 1

Summary of item modifications

Item number	Original version	Adapted version
		English	Indonesian
4	Adjuvant therapies	Adjuvant therapies (additional therapies)	Terapi adjuvan (terapi tambahan)
8	Bowel regime	Gastrointestinal therapies	Terapi saluran pencernaan
16	Demerol	Pethidine	Pethidine
17	The accumulation of losses renders burnout inevitable for those who seek work in palliative care	The accumulation of losses leads to burnout in healthcare providers working in palliative care unit	Kejenuhan tenaga kesehatan yang bekerja di unit paliatif disebabkan oleh akumulasi rasa kehilangan karena kematian pasien

Table 2

The palliative care quiz for nurses-Indonesian and original version

Item number	Questions
1	Perawatan paliatif hanya sesuai diberikan pada pasien yang mengalami penurunan atau kemunduran kondisi (S)
	Palliative care is appropriate only in situations where there is evidence of a downhill trajectory or deterioration (F)
2	Morfin adalah standar yang digunakan untuk membandingkan efek analgesik dari golongan opioid lainnya (B)
	Morphine is the standard used to compare the analgesic effect of other opioids (T)
3	Keparahan penyakit menentukan metode penatalaksanaan nyeri (S)
	The extent of the disease determines the method of pain treatment (F)
4	Terapi ajuvan (tambahan) penting dalam manajemen nyeri (B)
	Adjuvant therapies are important in managing pain (T)
5	Anggota keluarga harus berada di samping pasien hingga pasien meninggal (S)
	It is crucial for family members to remain at the bedside until death occurs (F)
6	Selama hari-hari terakhir menjelang ajal, rasa kantuk yang dialami pasien akibat ketidakseimbangan elektrolit dapat menurunkan kebutuhan obat sedatif (B)
	During the last days of life, the drowsiness associated with electrolyte imbalance may decrease the need for sedation (T)
7	Ketergantungan obat merupakan salah satu masalah utama yang terjadi jika morfin digunakan dalam jangka panjang untuk manajemen nyeri (S)
	Drug addiction is a major problem when morphine is used on a long-term basis for the management of pain (F)
8	Pasien yang mendapatkan terapi opioid seharusnya juga diberikan terapi saluran pencernaan (B)
	Individuals who are taking opioids should also follow a bowel regime (T)
9	Pemberian perawatan paliatif tidak memerlukan rasa empati (S)
	The provision of palliative care requires emotional detachment (F)
10	Selama tahap akhir penyakit, obat yang menyebabkan depresi pernafasan tepat digunakan untuk penatalaksanaan dyspnea berat (B)
	During the terminal stages of an illness, drugs that can cause respiratory depression are appropriate for the treatment for severe dyspnea (T)
11	Pada umumnya, laki-laki lebih cepat menghilangkan rasa duka daripada wanita (S)
	Men generally reconcile their grief more quickly than women (F)
12	Filosofi perawatan paliatif sesuai dengan prinsip terapi agresif (B)
	The philosophy of palliative care is compatible with that of aggressive treatment (T)
13	Plasebo (obat kosong) dapat digunakan dalam penatalaksanaan beberapa jenis nyeri (S)
	The use of placebos is appropriate in the treatment of some types of pain (F)
14	Kodein dosis tinggi lebih sering menyebabkan mual dan muntah daripada morfin (B)
	In high doses, codeine causes more nausea and vomiting than morphine (T)
15	Penderitaan dan nyeri fisik adalah hal yang sama (S)
	Suffering and physical pain are synonymous (F)
16	Petidine (analgesik golongan opioid) bukan analgesik yang efektif untuk mengontrol nyeri kronis (B)
	Demerol is not an effective analgesic in the control of chronic pain (T)
17	Kejenuhan tenaga kesehatan yang bekerja di unit paliatif disebabkan oleh akumulasi rasa kehilangan karena kematian pasien (S)
	The accumulation of losses renders burnout inevitable for those who seek work in palliative care (F)
18	Manifestasi nyeri kronis berbeda dengan nyeri akut (B)
	Manifestations of chronic pain are different from those of acute pain (T)
19	Rasa duka akibat kehilangan kerabat jauh lebih mudah diatasi daripada kehilangan kerabat dekat (S)
	The loss of a distant or contentious relationship is easier to resolve than the loss of one that is close or intimate (F)
20	Ambang nyeri dapat diturunkan melalui rasa cemas atau lelah (B)
	The pain threshold is lowered by anxiety or fatigue (T)

PCQN: Palliative care quiz for nurses, B: Benar, T: True, S: Salah, F: False

Validity testing

In terms of the relevancy, all six experts considered 16 of the 20 items of the PCQN-I to be highly relevant, so the I-CVI for each of these 16 items was 1.00, which was considered “excellent.”[26] Four items of the PCQN-I were given a score of 1 (not relevant) or 2 (somewhat relevant) by an expert. Thus, the I-CVIs were 1.00 for 16 items and 83 for four items.

Regarding the clarity, three of the 20 items of the PCQN-I were given a score of 1 (not clear) by an expert; three items were given a score of 2 (item requires some revision) by an expert, and one item was given a score of 2 (item requires some revision) by two experts. Thus, the I-CVIs were 1.00 for 13 of the 20 items, 0.83 for six items, and 0.67 for one item. Both these S-CVIs/Ave were 0.97 and 0.93, respectively, reflecting excellent content validity.[2628]

Reliability testing

Overall responses

A total of 150 health-care providers, including 100 nurses and 50 physicians recruited from 20 PHC centers in Yogyakarta, completed the PCQN-I in the reliability testing. The mean age of the participants was 40.75 years (SD = 9.88, range = 23–59 years). More than 64% (n = 97) of the participants were female. All the physicians had completed at least a bachelor's degree, whereas only 4% of the nurses (n = 4) had a bachelor's degree. More than 54% of the participants (n = 82) had hospital experience, but only 26.7% (n = 40) had an experience related to providing palliative care in hospitals. The mean years of working experience in PHC centers were 14.69 (SD = 10.01, range = 0.1–35 years). Only 30% (n = 45) of the participants had experience with caring for terminally ill patients. Half of the participants (n = 75) had received a formal palliative care education in their previous school education, whereas only a minority of them had participated in palliative care continuing education [Table 3].

Table 3

Participant background information (n=150)

Variables	Physician (n=50) n (%)	Nurse (n=100) n (%)	Total (n=150) n (%)
Gender
Male	17 (34.0)	36 (36.0)	53 (35.3)
Female	33 (66.0)	64 (64.0)	97 (64.7)
Education levels
Diploma and below	0 (0.0)	96 (96.0)	96 (64.0)
Bachelor’s degree	50 (100.0)	4 (4.0)	54 (36.0)
Hospital experience
Yes	30 (60.0)	52 (52.0)	82 (54.7)
No	20 (40.0)	48 (48.0)	68 (45.3)
Hospital PC experience
Yes	12 (24.0)	28 (28.0)	40 (26.7)
No	38 (76.0)	72 (72.0)	110 (73.3)
Caring for TP in hospitals
Yes	12 (24.0)	35 (35.0)	47 (31.3)
No	38 (76.0)	65 (65.0)	103 (68.7)
Caring for TP in PHC units
Yes	17 (34.0)	28 (28.0)	45 (30.0)
No	33 (66.0)	72 (72.0)	105 (70.0)
Previous PC education
Yes	33 (66.0)	42 (42.0)	75 (50.0)
No	17 (34.0)	58 (58.0)	75 (50.0)
PC seminar attendance
Yes	14 (28.0)	9 (9.0)	23 (15.3)
No	36 (72.0)	91 (91.0)	127 (84.7)
PC training
Yes	4 (8.0)	1 (1.0)	5 (3.3)
No	46 (92.0)	99 (99.0)	145 (96.7)
Variables	Physician (n=50) Mean ± SD	Nurse (n=100) Mean ± SD	Total (n=150) Mean ± SD
Age	37.5 (9.7)	42.3 (9.6)	40.7 (9.9)
Years of PHC unit experience	11.0 (9.2)	16.5 (9.9)	14.7 (10.0)

Here, a seminar is more like a lecture or classroom style of learning, an expert sharing information with the audience on a specific topic. Training is the act of teaching and or developing skills to improve the skills required to do one’s job. PC: Palliative care, PHC: Primary health care, TP: Terminally ill patient, SD: Standard deviation

The mean PCQN-I score among the 150 participants was 7.78 (SD = 3.56). As shown in Table 4, Item 5 “It is crucial for family members to remain at the bedside until death occurs (False)” and Item 19 “The loss of a distant or contentious relationship is easier to resolve than the loss of one that is close or intimate (False)” were answered correctly by the lowest percentage of participants (5.3%, n = 8). Table 4 also shows that Item 18 was very easy (difficulty index > 0.80), whereas Items 5, 19, 7, 3, and 11 were very difficult (difficulty index 0.04–0.15). Items 5 and 19 were poor quality (discrimination index <0.10).

Table 4

The results of the palliative care quiz for nurses-Indonesian version (n=150)

Item number	Physician (n=50)			Nurse (n=100)			Total (n=150)
	Correct answers, n (%)	Item response theory		Correct answers, n (%)	Item response theory		Correct answers, n (%)	Item response theory
		Difficulty	Discrimination		Difficulty	Discrimination		Difficulty	Discrimination
18	46 (92.0)	0.92	0.23	91 (91.0)	0.91	0.24	137 (91.3)	0.91	0.24
4	47 (94.0)	0.90	0.23	52 (52.0)	0.52	0.68	99 (66.0)	0.65	0.60
15	40 (80.0)	0.74	0.46	58 (58.0)	0.58	0.76	98 (65.3)	0.63	0.66
8	37 (74.0)	0.82	0.54	52 (52.0)	0.52	0.80	89 (59.3)	0.62	0.76
1	35 (70.0)	0.66	0.23	49 (49.0)	0.49	0.40	84 (56.0)	0.55	0.37
6	24 (48.0)	0.48	0.69	57 (57.0)	0.57	0.48	81 (54.0)	0.54	0.47
14	27 (54.0)	0.70	0.46	41 (41.0)	0.41	0.60	68 (45.3)	0.51	0.66
2	31 (62.0)	0.70	0.54	36 (36.0)	0.36	0.52	67 (44.7)	0.47	0.55
17	27 (54.0)	0.50	0.46	39 (39.0)	0.39	0.36	66 (44.0)	0.43	0.42
9	30 (60.0)	0.66	0.54	33 (33.0)	0.33	0.16	63 (42.0)	0.44	0.42
20	18 (36.0)	0.40	0.61	35 (35.0)	0.35	0.56	53 (35.3)	0.37	0.66
16	22 (44.0)	0.42	0.69	29 (29.0)	0.29	0.56	51 (34.0)	0.33	0.68
10	9 (18.0)	0.24	0.54	32 (32.0)	0.32	0.60	41 (27.3)	0.29	0.47
12	13 (26.0)	0.34	0.46	27 (27.0)	0.27	0.60	40 (26.7)	0.29	0.50
13	20 (40.0)	0.38	0.38	16 (16.0)	0.16	0.32	36 (24.0)	0.23	0.42
11	17 (34.0)	0.24	0.15	10 (10.0)	0.10	0.20	27 (18.0)	0.15	0.24
3	10 (20.0)	0.24	0.61	10 (10.0)	0.10	0.16	20 (13.3)	0.15	0.37
7	16 (32.0)	0.34	0.31	2 (2.0)	0.02	0.08	18 (12.0)	0.13	0.26
19	3 (6.0)	0.04	0.08	6 (6.0)	0.06	0.04	9 (6.0)	0.05	0.05
5	3 (6.0)	0.02	0.08	5 (5.0)	0.05	0.12	8 (5.3)	0.04	0.08

Internal consistency

The KR-20 was used to examine the internal consistency of the PCQN-I. The reliability coefficient was. 71, indicating moderate reliability.[29]

DISCUSSION

The instrument was originally developed by Ross et al.[14] to examine palliative care knowledge among nurses and nursing students. Because this was a general concept, in the current study, the instrument was used to assess the palliative care knowledge of both nurses and physicians working in the PHC centers. In this study, the PCQN-I comprised 20 items with true/false/don't know options.

Some items were revised during the cross-cultural adaptation process to improve the semantic, idiomatic, experiential, and conceptual equivalence. Item 4 “Adjuvant therapies are important in managing pain” was revised because 54% of the nurses (n = 14) in the pilot testing incorrectly answered this item and provided comments indicating that phrase “adjuvant therapies” was unfamiliar to them. The expert committee suggested the addition of the words “additional therapies” following the term “adjuvant therapies” as “adjuvant therapies/additional therapies” (terapi adjuvan/terapi tambahan-in Indonesian).

Item 8 “Individuals who are taking opioids should also follow a bowel regimen” was also reviewed. Approximately 60% of the nurses (n = 16) involved in the pilot study incorrectly answered this item. The expert committee viewed that “bowel regimen” was an unfamiliar term for nurses in PHC centers. Therefore, the expert committee changed “bowel regimen” to “gastrointestinal therapies” (terapi saluran pencernaan-in Indonesian).

The expert committee also revised Item 16 “Demerol is not an effective analgesic in the control of chronic pain.” Considering that Demerol is not distributed in Indonesia, the anesthesiologist changed “Demerol” to “Pethidine.” Thus, health-care providers could easily recognize the name of the drug.

The expert committee found that Item 17 “The accumulation of losses renders burnout inevitable for those who seek work in palliative care” was ambiguous. In the French and Spanish versions, this item referred to health-care providers working in the palliative care unit.[1112] This understanding was also assumed into the PCQN-I.

In addition, the expert committee retained Item 5 “It is crucial for family members to remain at the bedside until death occurs,” (False) although none of the participants (n = 40) involved in the pilot testing could answer it correctly. The experts suggested that the misconceptions related to Item 5 in the PCQN might be influenced by culture. In Indonesia, it is considered an obligation of the family to care for terminally ill patients.[30] This is because the family plays an important role in activities pertaining to the daily living, physical, spiritual, social, psychological, autonomous, and financial needs of patients during hospitalization.[31] In addition, the family has an important role in making treatment-related decisions for patients.[32] Therefore, almost all hospitals in Indonesia allow the family to be with patients 24 h a day.[3233] Due to this cultural norm, participants might have felt that the attendance of the family at the bedside until the death of the patient is important. The original version of the PCQN indicated that remaining at the bedside until death occurs could become exhausting for family members.[14] This item was retained because the experts considered Item 5 to be one of the critical concepts in palliative care.

Few participants answered Item 19 correctly, “The loss of a distant or contentious relationship is easier to resolve than the loss of one that is close or intimate” (False). This finding is similar to that in an Iranian study, with only 9.1% of the participants correctly answering this item.[16] Like Iran, Indonesian kinship is also strong, with all family members participating in care.[32]

The S-CVIs/Ave of the PCQN-I were 97 and 93 in terms of its relevancy and clarity, respectively. These values were higher than the Korean (S-CVI = 0.85)[13] and Spanish versions (S-CVI = 0.83).[12] The high S-CVIs/Ave score suggested that due to some revisions made during the cross-cultural adaptation process, a consensus was reached on any discrepancies found in the PCQN-I.

Regarding the difficulty index, Items 5, 19, 7, 3, and 11 were indicated to be very difficult (difficulty index <0.20), and Item 18 was indicated to be very easy (difficulty index >0.80) among all participants. However, these items were retained because they fell within the desired range in the physician's group, with the exception of items 5 and 19.

Regarding the discrimination index, items 5 and 19 were poor quality (discrimination index <0.10). We retained these items because earlier studies have explained that a positive discrimination index value indicates that an item is desirable.[1214] Furthermore, deleting these items did not significantly increase the reliability coefficient.

In terms of reliability, the KR-20 score of the PCQN-I was 71, indicating acceptable internal consistency. This result was similar to that of the Spanish version (KR-20 = 0.72).[12] Although the KR-20 score of the PCQN-I was slightly lower than that of the Korean (KR-20 = 0.78)[13] and the original version (KR-20 = 0.78),[14] it was higher than that of the French version (KR-20 = 0.60).[11] The internal consistency of the PCQN-I suggested that revising some items did not change the conceptual equivalence from that of the original instrument.

The findings of this study indicated that PCQN-I is a valid, reliable instrument for assessing palliative care knowledge among PHC providers. The PCQN-I is the first version that demonstrates the applicability of the instrument to examine palliative care knowledge not only among nurses but also among physicians in PHC settings. The PCQN-I could be used to identify health-care providers' knowledge as the basic data by which to assess their misconceptions and educational needs related to palliative care. In future, it may be used as an assessment tool to evaluate the knowledge of health-care providers who participate in palliative care education programs.

Limitations

This study had some limitations. First, the reliability testing was conducted only in PHC centers in Yogyakarta, which may reduce the generalizability of the results. An additional survey at the national level would be worthwhile. Second, the moderate scores for the internal consistency of the PCQN-I (KR-20 = 0.71) should be a concern. Further investigation is encouraged to assess the validity of the instrument among health-care providers working in hospitals.

CONCLUSIONS

The PCQN-I was found to be a valid, reliable instrument for assessing palliative care knowledge among health-care providers working in PHC centers. The instrument can be applied to evaluate palliative care programs in Indonesia. Further investigation is encouraged to examine its reliability in different clinical settings.

Financial support and sponsorship

This work was supported by the Indonesia Endowment Fund for Education (Lembaga Pengelola Dana Pendidikan- LPDP), Ministry of Finance of Indonesia (Ref: S-6209/LPDP. 3/2015) and in part by Higher Education Sprout Project, Ministry of Education to the Headquarters of University Advancement at National Cheng Kung University (NCKU).

Conflicts of interest

There are no conflicts of interest.