Psychometric testing of the Norwegian version of the questionnaire Family Satisfaction in the Intensive Care Unit (FS-ICU-24).

INTRODUCTION: The questionnaire, Family Satisfaction in the Intensive Care Unit (FS-ICU-24), was developed to assess relatives' satisfaction with care and involvement in decision-making processes when a close family member stays in the ICU. AIM: This study was aimed at describing the translation and exploring the psychometric properties of the Norwegian version of the questionnaire.
METHODS: The study design was a cross-sectional survey. After translating the questionnaire according to recommended procedures, 123 close relatives of patients, recently treated in ICU, responded to a mailed questionnaire including the FS-ICU-24-No. Item-to-total correlations and Cronbach's alpha coefficient were assessed for estimating reliability and construct validity was assessed by the "known groups" technique and explorative factor analysis.
RESULTS: The Cronbach's alpha coefficient of 0.96 and significant item-to-total correlations supported the homogeneity of the instrument. The construct validity was reflected in significant differences in median scores on the total scale and subscales between the group reporting lower degrees of satisfaction and the group reporting higher degrees of satisfaction. Two fixed factors with an eigenvalue >1, and an explained variance of 62.5%, emerged from the factor analysis.
CONCLUSION: The FS-ICU-24-No showed promising psychometric properties regarding reliability in this study group, which may indicate that the instrument is suitable for assessing family members' satisfaction with care and decision making in Norwegian ICU.

Introduction

Close family members of patients in intensive care units (ICU) are exposed to traumatic and stressful events which are reported to cause crises and negative experiences.1 In addition to the patient-related anxiety and uncertainty that close relatives experience, the environment in an ICU represents several stress elements due to highly technological equipment, busyness, and life-threatening situations and events.2,3 The patient is in a critical, often life threatening, situation, and close family members also need to be involved in difficult decision making on behalf of the patient.3–5 Hence, a family-centered care approach, focusing on both the patient and the family members, is of utmost importance.6 However, the concept of family satisfaction in ICU is complex, and a gold standard for accurately measuring the concept is lacking.7

The satisfaction of family members of patients in ICU, which is defined as the amount of fulfillment of perceived or real, implicit or explicit needs and expectations8 is reported to involve several aspects. One aspect is related to how good they perceive the patient care to be.8 Other aspects are how much they feel included and emotionally supported, and to what extent and in what way their informational needs are attended to.8,9 The study by Schwarzkopf et al10 showed that the consistency, clarity, and completeness of information, and emotional support and respect, could be improved for this group. The atmosphere and accommodation in the unit and in the waiting room are emphasized, as the surroundings are rarely well prepared for the relatives to be present and affect the feeling of comfort for individuals who experience a tough time.10,11

The perception of being involved or not in the decision-making processes about matters relating to the patient is one of the aspects most frequently found dissatisfactory by family members.10,12 Although most close relatives prefer being involved in the decision-making processes,3–5 this may also cause additional stress and burden because the decisions may severely influence the patient’s situation.

How a family member’s needs are met by the professionals in the ICU regarding care, respect, and involvement are reported to be crucial for their ability to cope with the situation after the ICU stay.2,11 Therefore, it is important to assess the close family members’ satisfaction with the care they and their critically ill relative receive during the stay. For that purpose, several tools, mostly questionnaires, have been developed and made available for use. Some of the tools have been presented in a newly published review by van den Broek et al.7 One of the assessment tools included in the overview is the Family Satisfaction in the Intensive Care Unit (FS-ICU) questionnaire, which was found appropriate for the intended purpose.

The FS-ICU questionnaire was developed by Heyland and Tranmer13 for use in a multicentre study in Canada. Among the questionnaires reviewed by van den Broek et al,7 the FS-ICU was found to be one of the best due to its psychometric properties and is one of the most extensively researched questionnaires. The questionnaire includes items related to family members’ satisfaction with the level of care they and their ill relative receive in the ICU, and the satisfaction with the level of involvement in the decision making. The included items reflect aspects that have been identified to be important for relatives’ satisfaction and the quality of care in the ICU. The original 34-item version of the instrument was later refined and modified to a 24-item questionnaire (FS-ICU-24).14 Information presented on the Canadian multicenter website15 and in the review by van den Broek et al,7 indicates that the questionnaire has been translated into several languages, for example German, Greek, Spanish, Portuguese, Swedish, Turkish, Korean, and Hebrew. According to Wall et al,14 the instrument is comprehensively developed, well tested, and widely available. For example, the German version was translated and validated by Stricker et al,16 and has been used to measure family satisfaction in the ICU by Schwarzkopf et al.10 Gerasimou-Angelidi et al12 have reported results from a study using the Greek version of the questionnaire. The Turkish version has been tested by Tastan et al,17 the Korean version was psychometrically tested by Kim et al,18 and Harrison et al19 published results of the psychometric testing of the questionnaire in UK. A general impression from the available testing studies is that the psychometric properties of the questionnaire seem to be solid and relevant.

The aim of the current study was to test the psychometric properties, by means of reliability and validity, of the Norwegian version of the questionnaire FS-ICU-24.

Materials and methods

Translation and adaptation of the instrument into a Norwegian context

The authors obtained permission to translate the FS-ICU-24 questionnaire for use in a Norwegian context. The original version of the English language questionnaire13 was translated into Norwegian by a bilingual professional translator, in accordance with the recommended translating and back-translating procedure.20 In addition, the back-translated version of the questionnaire was reviewed by the constructor of the original instrument to ensure that the original meaning of the included items was maintained.

Subsequently, the translated version of the questionnaire was reviewed by five intensive care professionals in a Norwegian ICU and a pilot study was conducted among six close family members of patients who had recently had a stay in an ICU. Both professionals and family members were asked to respond to and evaluate the translated version of the questionnaire. They found that the questions are easy to understand and meaningful for measuring family satisfaction in ICUs. The size and structure of the instrument were also considered to be appropriate.

Study sample

The study had a cross-sectional survey design. The study group consisted of 123 close relatives of patients who had stayed for a minimum of 24 hours in the ICU, 2 months to approximately 1 year before they received the mailed questionnaire. They had to be 18 years old or more, and either a family member, friend or another person who was registered as the closest relative in the patients’ hospital record. ICUs from nine different hospitals in Southern Norway were involved. Potential participants were recruited and the questionnaire, along with information about the study, was mailed by a contact person at the hospitals. From a list of patients who had been admitted to the ICU, one relative of every fifth patient who had survived and one relative of every tenth patient who had died in the ICU were chosen, reflecting the average distribution of these groups in Norwegian ICUs. Further inclusion criteria were: the patient should have been registered as an “intensive care patient” which implies more than 24 hours in the ICU or need of ventilation assistance such as mechanical ventilation, BIPAP, CPAP, intubated or mask ventilated, regardless of the length of the ICU stay. The nine ICUs involved represented two university hospitals, five regional hospitals, and two local hospitals. Initially, a sample of 261 individuals received the questionnaire, and 98 responded positively in the first round. One reminder was sent after 4 weeks, which resulted in 25 more responses. Thus, the final study group constituted a response rate of 47.1%.

Data collection

The following background information was assessed: age, sex, educational level (with the options of primary school, secondary school, or university/university college), the family member’s relationship with the patient (with the options of spouse/partner, child, parent, sibling, or other relatives), and whether the patient had died during the stay or not. One general question related to the relative’s overall satisfaction level with the care from nurses and physicians in the ICU was asked with response categories on a five-point scale ranging from “not at all” to “to a very high degree”.

The FS-ICU-24 questionnaire was developed to assess family satisfaction with care provided in an ICU.13,19 The questionnaire contains 24 items: 14 items family satisfaction with care subscale (FSCare) addressing overall satisfaction with the care and ten items family satisfaction with decision-making subscale (FSDM) addressing satisfaction with decision making. The subscale FSCare reflects aspects regarding the quality of care provided to the patient, by means of being careful, respectful, and competent. Furthermore, aspects such as communication, information access and quality, coordination, continuity, accessibility of care, and satisfaction with the ICU surroundings are assessed. The subscale FSDM includes aspects related to the relative’s satisfaction with, or with not, being involved in decisions related to the care and treatment of the patient. This subscale particularly reflects satisfaction with the amount and quality of information, involvement, and support during decision-making processes.

A five-point Likert scale is used as the scoring system in the questionnaire. It is converted to a score of 0–100 as recommended,14 with 0 representing the lowest satisfaction and 100 representing the highest satisfaction.

Previous studies using the FS-ICU-24 among relatives of adult patients have shown a Cronbach’s alpha coefficient of 0.94 for the total scale.14 Furthermore, Cronbach’s alpha coefficients of 0.92 and 0.95 for the subscale FSCare, and 0.88 and 0.87 for the subscale FSDM, have been reported by Wall et al14 and Stricker et al,16 respectively.

Statistical analyses

Descriptive statistics were used to display the background variables of the participants. The reliability of the Norwegian version of the questionnaire (FS-ICU-24-No) was assessed by estimating the internal consistency (homogeneity) with item-to-total correlations, reflecting to what degree all items in the instrument measure the same attribute. Internal consistency was estimated by calculating Spearman’s rank correlations (r) between each item and the total scale, after omitting the individual item from the total scale. Internal consistency was also estimated with Cronbach’s alpha reliability coefficients for the total scale and the FSCare and FSDM subscales.

The construct validity of the instrument was assessed by the “known groups” technique’, by comparing differences in scores of the total scale and subscales for the group which reported being overall satisfied or very satisfied with the care received in the ICU with the group which reported being slightly or not at all satisfied with the overall care. Thus, in order to perform this analysis, the ordinal level question concerning overall satisfaction was dichotomized and differences in median (Md) scores between the two groups were calculated using the Mann–Whitney U-test for independent samples.

The construct validity was also explored using the principal component method of factor analysis with varimax rotation and Kaiser normalization. In general, exploratory factor analysis is used as a statistical procedure for reducing a large set of variables into clusters of variables with common underlying dimensions and thereby, revealing some structures of the phenomenon or the instrument under investigation.22,23 The distribution of items about two fixed components or factors in our study was compared to the clusters of items on the two subscales of the FS-ICU-24 previously reported.13,14 IBM SPSS statistical software version 24.0 (Armonk, NY, USA) was used for performing statistical analyses, and a P-value of <0.05 was considered significant.

Ethics

The study was implemented according to the ethical principles for clinical research.24 It was approved by the Regional Committee for Medical Research Ethics in Southern Norway (reference number: REK sør-øst A 2013/458) and the Department for Privacy Protection at each hospital approved the study. Answering and returning the questionnaire were considered as consent to participate in the study.

Results

Description of the study group

The study group consisted of 123 close relatives (51.3% women) of patients admitted to an ICU. The participants’ mean age was 60.3 years (SD=12.9), ranging from 27 to 85 years. Regarding educational level, 16.3% had completed primary school, 48% had completed secondary school, and 41% were educated at a university or university college. A total of 52% of the relatives were a spouse or partner of the patient, 26.8% were their children, 11.4% were parents, and 7.3% were siblings or other family members. The number of patients who died during the ICU stay was 40 (32.5%).

The reliability and validity of the FS-ICU-24-No

The obtained Cronbach’s alpha reliability coefficient of 0.96 for the total scale of the FS-ICU-24-No indicated a high level of homogeneity. Cronbach’s alpha for the subscales of FSCare and FSDM was 0.93 and 0.94, respectively.

The homogeneity of the scale was also confirmed by the item-to-total correlations, which showed that all items were significantly correlated with the total scale (Table 1). The item with the lowest correlation coefficient (r=0.35) was that of “satisfaction of the level or amount of care the patient received”. The item with the highest correlation coefficient (r=0.90) was that of “completeness of information about what was happening”. It is recommended that, the lowest value for item-to-total correlations should be r=0.20.20

Table 1

Item-to-total correlations scores (Spearman’s rs) of the FS-ICU-24-No

Item no.	Item content	rs	P-value
1.	Courtesy, respect, and compassion by staff toward patient	0.65	<0.01
2.	Management of pain	0.67	<0.01
3.	Management of breathlessness	0.74	<0.01
4.	Management of agitation	0.78	<0.01
5.	How well staff considered family needs	0.83	<0.01
6.	How well the staff provided emotional support toward family	0.84	<0.01
7.	Coordination and teamwork by staff	0.83	<0.01
8.	Courtesy, respect, and compassion by staff toward family	0.86	<0.01
9.	Skill and competence of nurses	0.56	<0.01
10.	Communication by nurses	0.85	<0.01
11.	Skill and competence of doctors	0.80	<0.01
12.	Atmosphere of the ICU	0.77	<0.01
13.	Atmosphere of the ICU waiting room	0.45	<0.01
14.	Satisfaction of the level or amount of care the patient received	0.35	<0.01
15.	Frequency of communication by doctors	0.80	<0.01
16.	Willingness of staff to answer questions	0.86	<0.01
17.	Staff provided understandable explanations	0.85	<0.01
18.	Honesty of information provided about patient’s condition	0.85	<0.01
19.	Completeness of information about what was happening	0.90	<0.01
20.	Consistency of information about patient’s condition	0.84	<0.01
21.	Feel included in the decision-making process	0.64	<0.01
22.	Feel supported during the decision-making process	0.75	<0.01
23.	Feel control over the care of the patient	0.63	<0.01
24.	Adequate time to address concerns and answer questions	0.60	<0.01

Abbreviation: FS-ICU-24-No, the Norwegian version of the 24-item Family Satisfaction in the Intensive Care Unit questionnaire.

The construct validity of the FS-ICU-24-No was reflected in significant differences in Md scores of the total scale and the subscales between the group reporting lower degrees of satisfaction and the group reporting higher degrees of satisfaction (Table 2).

Table 2

Score differences on the FS-ICU-24-No between the groups who reported high and low satisfaction (n=109)

	Being slightly or not at all satisfied with the ICU care (n=30)Md (iqr)	Being satisfied or very satisfied with the ICU care (n=79)Md (iqr)	P-value
Total FS-ICU-24-No	47.9 (21.8)	80.2 (23.2)	<0.001
FSCare	54.5 (22.2)	81.3 (23.2)	<0.001
FSDM	36.3 (25.6)	80.0 (24.2)	<0.001

Abbreviations: FSCare, family satisfaction with care subscale; FSDM, family satisfaction with decision-making subscale; FS-ICU-24-No, the Norwegian version of the 24-item Family Satisfaction in the Intensive Care Unit questionnaire; Md, Median; iqr, interquartile range; ICU, intensive care unit.

Construct validity was also, to a certain degree, reflected in the factor analysis with two fixed factors explaining 62.5% of the variance in the instrument, and each factor had an eigenvalue > 1. The factor loadings, the distribution of items about the extracted factors, and their explained variances are summarized in Table 3. Notably, items 1–14 constitute the subscale of FSCare and items 15–24 constitute the subscale of FSDM in the original instrument.21

Table 3

Standardized factor loadings for the FS-ICU-24-No items and the explained variance of the factors

Item no. and contenta	Factor 1	Factor 2
1. Courtesy, respect, and compassion by staff toward patient	0.269	0.840
2. Management of pain	0.205	0.742
3. Management of breathlessness	0.238	0.629
4. Management of agitation	0.076	0.639
5. How well staff considered family needs	0.718	0.447
6. How well the staff provided emotional support toward family	0.629	0.503
7. Coordination and teamwork by staff	0.469	0.726
8. Courtesy, respect, and compassion by staff toward family	0.622	0.576
9. Skills and competence of nurses	0.271	0.810
10. Communication by nurses	0.679	0.585
11. Skill and competence of doctors	0.457	0.620
12. Atmosphere of the ICU	0.551	0.558
13. Atmosphere of the ICU waiting room	0.432	0.342
14. Satisfaction of the level or amount of care patient received	0.304	0.091
15. Frequency of communication by doctors	0.781	0.243
16. Willingness of staff to answer questions	0.656	0.578
17. Staff provided understandable explanations	0.720	0.467
18. Honesty of information provided about patient’s condition	0.756	0.466
19. Completeness of information about what was happening	0.793	0.458
20. Consistency of information about patient’s condition	0.774	0.348
21. Feel included in the decision-making process	0.687	0.068
22. Feel supported during the decision-making process	0.749	0.260
23. Feel control over the care of the patient	0.793	0.172
24. Adequate time to address concerns and answer questions	0.669	0.230
Percentage of the variance explained	55.42%	7.06%

Notes: The highest loading on each item is in bold.

Items 1–14 are included in subscale FSCare and items 15–24 are included in the subscale FSDM in the original FS-ICU-24 instrument.

Abbreviations: FSCare, family satisfaction with care subscale; FSDM, family satisfaction with decision-making subscale; FS-ICU-24-No, the Norwegian version of the 24-item Family Satisfaction in the Intensive Care Unit questionnaire.

Discussion

The present study aimed to test the reliability and validity of the Norwegian version of the questionnaire FS-ICU-24.

The obtained Cronbach’s alpha coefficients of 0.96 for the total scale of the FS-ICU-24-No, and 0.93 and 0.94 for the subscales of FSCare and FSDM, respectively, indicated a high level of homogeneity. Similar results have been reported in previous studies.14,16–19 However, although a very high Cronbach’s alpha coefficient may support reliability, it may also indicate a possible overlap among items which should be considered when the questionnaire is evaluated.20 The correlations of r ≥0.35 for each item with the total scale also reflect the homogeneity of the questionnaire and the adequacy of the included items.20 The study group might be considered as homogenous as they all had experienced being a close relative of a critically ill patient in the ICU. However, the participants’ characteristics regarding age, educational level, and family relationship to the patients differed, and the sample group included relatives of patients who had survived and who had died during the stay. Consequently, the results might indicate that the items in the questionnaire are relevant regardless of such circumstances.

The results showed a distinct relationship between the study groups’ overall satisfaction with care provided in the ICU, assessed in a separate question, and the total scale and the two subscales (Table 2). In this respect, the questionnaire’s construct validity and its ability to measure the family members’ satisfaction are supported.

The distribution of items about the two fixed factors in the factor analysis shows a different pattern compared to the distribution of items about the two subscales in the original instrument.21 In our study, the first factor explained 55.42% of the variance and included all ten items from the original FSDM subscale21 and five of the items from the original FSCare subscale. Hence, this might indicate that the logical and structural pattern of the questionnaire differs across countries and study groups. A relatively clear pattern that appeared in the factor analysis in our study was that several items concerning care for the family, which were included in the subscale FSCare in the original questionnaire,21 loaded higher on the factor reflecting satisfaction with decision making. However, the internal structure of the questionnaire in this Norwegian study might also be regarded as logical as all items that clustered about the first factor reflected care and support toward the family member, while items that clustered about the other factor reflected care for the patient. Hence, this could be another way to structure the questionnaire and create alternative subscales.

In the ICUs, family members’ experiences of the quality of care are particularly important, as patient-centered care also includes family-centered care.6,14 The FS-ICU-24 questionnaire appears to be useful for measuring family members’ satisfaction with care in ICUs. The questionnaire was developed to include aspects previously found to be important for experiencing satisfaction, such as expectations of care, feeling included and involved, receiving adequate information,10 the quality of communication, and the unit’s infrastructure.7 Receiving enough and clear information is also critical for being actively involved in decision making, either together with the patients or when acting as surrogate decision makers.11,13,14

Limitations and strengths

The small sample size of the study could be considered a limitation. Nevertheless, the study provides relevant and valuable knowledge about the psychometric properties of the FS-ICU-24 in a Norwegian context. The questionnaire was originally developed in Canada, and the cultural impact must be considered when adapted to another country, such as Norway. Another limitation might be that the data were collected only once, and evaluation of the questionnaire’s stability is, therefore, unavailable. The strength of the study is that nine ICUs of varying size and type were included and the participants constituted a diverse group with regard to socio-demographic characteristics. However, more studies that include larger samples are needed to arrive at a more general conclusion when it comes to the psychometric properties of the questionnaire.

Conclusion

The translated Norwegian version of the FS-ICU-24 questionnaire was shown to be a reliable instrument, in terms of consistency and homogeneity, for measuring family satisfaction with the care in ICUs. The psychometric testing results indicate that the included items are suitable and appropriate for measuring what they intend to measure, i.e., relatives’ satisfaction with care and decision making in the ICU. The internal structure of the questionnaire revealed a pattern that differed from the testing results reported by Heyland et al.21 In their study, two subscales, concerning satisfaction with care and satisfaction with decision making, emerged. In the current study, however, the factor analysis might indicate that satisfaction with care of the patient and satisfaction with care directed toward the family member constitute two different subscales.