Psychometric analysis of Persian version of patient safety competency self-evaluation in psychiatric wards.

BACKGROUND: Assessment of the patient safety competency is necessary for the growth of nursing and safe care profession as well as evaluation of the nurses' educational needs. The present study was conducted to determine psychometric properties of the Persian version of the patient safety competency self-evaluation (PSCSE) tool in Iranian psychiatric wards.
MATERIALS AND METHODS: All nurses (n = 209) working in two psychiatric hospitals of Kerman, Iran, were included in the present cross-sectional study using census method. This tool contains 41 items: six items are related to knowledge, 14 items deal with attitude, and 21 items are about skill dimension. Psychometric properties of the questionnaire including its content and face validity were also examined. The factor structure of the questionnaire was evaluated using exploratory factor analysis. The internal consistency and reliability were assessed by test-retest method with an interval of 14 days. Cronbach's alpha and corrected item-total correlation were used to measure reliability.
RESULTS: The content validity index was 0.65 and the content validity ratio was 0.89. Item 14 was removed from the skill domain because it was not related to the psychiatric ward. According to the results of factor analysis, there was a significant relationship between the questions and the relevant factors. The correlation coefficient for test-retest with 14 days interval was intraclass correlation coefficient = 0.92 for the whole instrument and 0.89, 0.89, and 0.92 for the domains of knowledge, attitude, and skill, respectively. The internal consistency coefficient (Cronbach's alpha) of the whole tool was 0.95 and 0.95, 0.79, and 0.95 for the domains of knowledge, attitude, and skill, respectively. Finally, PSCSE Questionnaire was obtained with 40 items in dimensions of knowledge (6 items), attitude (14 items), and skills (20 items).
CONCLUSION: Persian version of the nurses' competency tool in ensuring patient safety in psychiatric wards has acceptable psychometric characteristics. Copyright:

Introduction

Patient safety, defined as making concerted efforts to prevent preventable harms to the patients caused by the health-care process, is one of the most important public health challenges worldwide.[1] Nurses comprise of a significant number of the therapeutic team and the health-care system. Patients expect nurses to strive to render safe nursing care (SNC) services. This requires the competence of nurses in ensuring patient safety, which, in turn, doubles the need to assess the nurses' competence in ensuring patient safety.[2] Increase of the safety competence of nurses improves their teamwork and critical thinking.[3] However, nurses' access to professional competencies, including patient safety competencies, is not at an adequate level.[456] Patient safety in psychiatric wards depends on the type of illness that predisposes the individual to injury and clinical risks. These include susceptibility to escape, violence, self-harm, harm to others, suicide, misconduct and aggression, falling from bed, using isolation room, medication errors, and patient abuse that undermine patient safety and endanger medical personnel.[789] Therefore, the safety of patients in this ward is of special importance. As a result, nursing managers are required to improve nurses' competence in ensuring patient safety and to evaluate them regularly to ensure the quality of safe care.[10] Application of the nurses' competency assessment criteria in ensuring patient safety not only improves levels of knowledge and awareness in nurses and nursing managers about nurses' competency status but also identifies deficiencies and attitudes of nurses' attitudes, knowledge, and skills in the field of nurses' competence to ensure patient safety.[11] Assessment of the nurses' competency in ensuring patient safety and review of the nurses' competency-enhancing interventions in this field require access to valid tools. Several tools, such as Leung,[12] Madigosky,[13] and the Health Professional Education in Patient Safety Survey assess the safety qualifications of nursing students and nurses.[14]

In this regard, using valid and reliable tools that include attitude, knowledge, and skill dimensions of nurses in ensuring patient safety is crucial.[15] The Nurses' Competence Tool for patient safety competency self-evaluation (PSCSE), one of the tools that examines these three domains, was developed by Lee et al. (2014) in South Korea. It is a valid and reliable tool in assessing the nurses' competence in ensuring patient safety for both nurses and nursing students in various clinical and academic situations. One of the strengths of this tool is its desired validity and reliability as well as its design based on the nurses' knowledge, attitude, and skills in the field of patient safety.[16] Although this tool was previously used in general wards and its validity was confirmed,[315171819] its validity was not examined in psychiatry. Moreover, the validity and reliability of the Persian version of this tool were not examined in assessing the nurses' competence in ensuring patient safety. Therefore, the present study was conducted to determine the psychometric properties of the Persian version of the PSCSE tool in psychiatric wards.

Materials and Methods

Study design and population

This cross-sectional study was a part of a larger study conducted from March to June 2020 in wards of two psychiatric hospitals in Kerman, one of the cities in southeastern Iran.

The study protocol was approved by the Ethics Committee of University of the vice chancellery of Shiraz University of Medical Sciences (No: IR.SUMS.REC.1399.301) and conducted in concordance with the standards of Helsinki declaration.

All 209 nurses working in the wards of two psychiatric hospitals in Kerman were included in the study by census method. Inclusion criteria were having at least the bachelor's degree in nursing and at least 6 months of employment experience. The exclusion criteria were submitting incomplete questionnaires or lacking willingness to participate in the study. All nurses signed informed consent forms prior to participating in the study.

Patient safety competency self-evaluation

This tool contains 41 items on a 5-point Likert scale: six items are related to knowledge (from 1 = I am not aware to 5 = I am very aware), 14 items deal with attitude (from 1 = strongly disagree to 5 = strongly agree), and 21 items are about skill dimension (from 1 = I am not very comfortable to 5 = I am very comfortable). The total attainable score from PSCSE was computed by calculating the average score of the means for all three domains. Scores may range from 1 to 5 so that higher scores indicate greater patient safety competency.[16]

Translation process

To translate PSCSE, we followed the guidelines introduced by Beaton et al.[20] In the initial translation stage, the English version of PSCSE was translated into Persian by two native bilingual translators independently. In the next stage, synthesis of the translations, a team consisting of the translators and two researchers compared the original scale with the prepared translations and then provided the finalized forward translation of PSCSE. In the back translation stage, the prepared forward translation of PSCSE was translated into English by two bilingual translators separately. In the fourth stage, an expert committee was formed to consolidate all the versions and to develop prefinal version of the scale. This committee comprised translators, a psychiatrist, and two expert nurses. Later, the prefinal version of PSCSE was completed by 10 nurses working in the psychiatric wards. These nurses were interviewed about the clarity and comprehensiveness of the items. In the final stage, namely submission of documentation to the coordinating committee for appraisal of the adaptation process, all the reports were discussed in the expert committee and after reaching consensus, the final adapted version of the scale was provided.

Validity

Content validity

Content validity of the questionnaire was determined using the opinions of seven experts, including nurses, psychiatrists, and psychiatrists. The experts were asked to provide their opinions with regard to the relevance, clarity, and simplicity of each item based on a 4-point scale (1 = nonrelevant, 2 = requires revision, 3 = relevant but requires revision, and 4 = relevant). The content validity index (CVI) of the tool was calculated using the formula (the ratio of agreement scores for each item with ranks 3 and 4 on the total number of responses) for each item as well as the whole tool. If an item from the questionnaire receives a score equal to or >0.79, it is approved.[21]

Face validity

To examine face validity of the Persian version of the tool, 20 nurses working in psychiatric wards were asked to provide their opinions with regard to the clarity and comprehensibility of the questionnaire items. Later, the necessary revisions were made based on the nurses' opinions.

Construct validity

Construct validity of the scale was evaluated by exploratory factor analysis (EFA) using principal component analysis and varimax rotation.[22] The eigenvalues >1 were considered as having a significant contribution in explaining the overall model variation. Sampling adequacy was examined using Bartlett's test of sphericity and Kaiser–Meyer–Olkin (KMO). A KMO value >0.6 was defined as the acceptable level and 0.80 indicated a good level of compatibility of the variables within the test.[23]

Convergent and divergent validity

Convergent validity of PSCSE was evaluated using the assessment SNC (ASNC) questionnaire, which was designed and validated by Rashvand et al. (2017) in Iran. The ASNC evaluates the performance of nurses with regard to the provision of SNC. The ASNC includes 32 items in four dimensions: evaluation of the nursing skills (16 items), assessment of the patients' psychological needs (4 items), assessment of the patient's physical needs (7 items), and assessment of the nurses' teamwork (5 items).

The ASNC was scored using the 5-point Likert scale including options from never = 1 to always = 5. The actual score of each item was obtained by multiplying the score of each item in the Likert scale by the weight of each item. In this regard, higher scores indicate better SNC, scores within the range of 73–170 showed low or undesirable levels of safe care, scores from 171 to 267 were considered at the moderate level of safe care, and scores ranging from 268 to 365 were determined at the desirable level of SNC. A panel of experts confirmed the content validity of the ASNC and its construct validity was calculated by factor analysis. The internal consistency of the questionnaire was acceptable considering the Cronbach's alpha of 0.92 and 0.86–0.95 for the total questionnaire and its dimensions, respectively.[24]

To determine discriminative validity of the questionnaire, Mann–Whitney U-test was run to confirm no significant difference between the participants' scores at the baseline and re-test. Floor and ceiling effects, defined as if more than 15% of the participants achieve either the least (floor) or the greatest (ceiling) scores, were calculated for the Persian version of PSCSE scale.[25]

Reliability

To determine the instrument's reliability, the test–retest method and internal consistency were used. Thirty nurses working in two psychiatric hospitals in Kerman were selected and asked to complete the Persian version of the tool twice within 2 weeks. Furthermore, internal stability of the instrument was evaluated using Cronbach's alpha calculation.

The Cronbach's alpha of about 0.7 and >0.80 are considered sufficient and high internal consistency of the tool.[2627]

Results

Nurses (n = 209) working in psychiatric wards participated in this study. Most participants were married (79.9%), female (78%), and within the age range of 25–45 years (60.3%). Furthermore, most nurses had bachelor degrees (91.4%), employed in men's ward (57.4), had formal contracts (69.9%), worked in shifts (76.1%), and had <10 years of work experience (38.8%) [Table 1].

Table 1

Demographic and clinical characteristics of nurses (n=209)

Variables	Groups	n (%)
Marital status	Married	167 (79.9)
	Single	42 (20.1)
Gender	Males	46 (22)
	Females	163 (78)
Age groups	>25	26 (12.4)
	25-45	126 (60.3)
	46-65	57 (27.3)
Educational level	Bachelor	191 (91.40)
	Master	18 (8.60)
Ward	Men	120 (57.4)
	Women	47 (22.5)
	Nursing office	16 (7.7)
	Children	13 (8.3)
	Urgency	10 (4.8)
	ECT	3 (1.4)
Employment status	Permanent	146 (69.9)
	Contractual	10 (4.8)
	Temporary to permanent	22 (10.5)
	Corporate	5 (2.4)
	Conscription law’s conscripts	26 (12.4)
Position	Nurse	124 (59.3)
	Head nurse	14 (6.7)
	Staff	10 (4.8)
	Supervisor	16 (7.7)
	Merton	2 (1)
	Practical nurse	43 (20.6)
Shifts	Fixed	50 (23.9)
	Rotational	159 (76.1)
Working experience	>10	81 (38.8)
	10-20	80 (38.3)
	20-30	48 (23)

ECT=Electroconvulsive therapy

The results showed that the nurses' total mean of patient safety competency was 2.54 ± 0.52, which was low. The nurses' highest mean score of patient's safety competency was related to the attitude dimension (3.30 ± 0.42), while the lowest mean score was related to knowledge dimension (1.22 ± 0.58) [Table 2].

Table 2

Mean and standard deviation of patient safety competency scores in psychiatric wards

Scales	Mean±SD
Total score of nurses’ safety Competency	2.54±0.52
Attitude	3.30±0.42
Skills	2.41±0.69
Knowledge	1.22±0.58

SD=Standard deviation

Content validity of the translated instrument was assessed using relevance, clarity, and simplicity. Furthermore, content validity ratio (CVR) = 0.65 and CVI = 0.89 were obtained. Item 14 was removed from the skill domain because it was not related to the psychiatric ward.

Construct validity

According to the results, parameters and t-index of the relationship between questions and relevant subscales of the t-value were higher than two in all questions, indicating a significant relationship between the questions and the relevant factors. In other words, all the observed variables could predict their factors. A closer examination of the parameter estimation values of each question indicates that question 38 in the knowledge factor (factor load = 0.68 and t-value = 9.84), question 7 in the attitude factor (factor load = 0.76 and t-value = 12.19), and question 20 in the skill factor (factor load = 0.81 and t-value = 15.77) are the most important predictor variables in their respective factors. As a result, 40 items and three dimensions of knowledge (6 items), attitude (14 items), and skills (20 items) were confirmed with acceptable values.

Bartlett's test of sphericity was significant and Kaiser-Mayer-Olkin, KMO (0.76). The scree plot of EFA is shown in Figure 1.

Figure 1

Scree plot of exploratory factor analysis of the Persian version of patient safety competency self-evaluation scale

The Pearson correlation coefficient between Persian version of PSCSE and ASNC was moderate (r = 0.57, P < 0.001). Discriminative validity revealed no significant difference between the scores achieved from the PSCSE dimensions at the baseline (n = 209) and re-test (n = 30). The comparison of the dimensions between test and re-test is summarized in Table 3.

Table 3

Comparison of the scores achieved from dimensions of the Persian version of patient safety competency self-evaluation

Domains	Test (n=209)		Re-test (n=30)		Mann-Whitney U P
	Median	IQR (25%-75%)	Median	IQR (25%-75%)
Knowledge	3	1-4	3	1-4	>0.99
Attitude	3	2-4	3	1-4	>0.99
Skill	4	1-4	4	1-4	>0.99
Total competency score	3	2-4	3	1-4	>0.99

IQR=Interquartile range

Considering the lowest (2.5%) and the highest (1.5%) achieved scores of the Persian version of PSCSE, neither floor nor ceiling effects were found in the scale.

The correlation coefficient for test–retest with 14 days' interval was intraclass correlation coefficient (ICC) = 0.92 for the whole instrument and 0.89, 0.92, and 0.89 for the knowledge, attitude, and skill domains, respectively. The Cronbach's alpha coefficient of the questionnaire was 0.95 generally and 0.95, 0.79, and 0.95 for the subscales of knowledge, attitude, and skill, respectively [Table 4].

Table 4

Reliability measures of the Persian version of patient safety competency self-evaluation

	Number of questions	Test median	Retest median	Cronbach’s alpha if item deleted	Alpha coefficient	ICC	95% CI
Attitude	14	3	3	0.65	0.79	0.89	0.0-92.85
Skill	20	3	4	0.72	0.95	0.92	0.0-95.90
Knowledge	6	4	4	0.89	0.95	0.89	0.0-95.85
Safety competence of nurses	40	3	3	0.67	0.95	0.92	0.0-97.92

ICC=Intraclass correlation coefficient, CI=Confidence interval

The SEM and MDC of the Persian version of the scale were calculated as 0.30 and 0.95, respectively.

Discussion

Despite the importance of patient safety and safe care in psychiatric wards, valid and reliable tools are not available in Persian to assess the competence of nurses working in psychiatric wards. Therefore, this study was carried out to investigate PSCSE in psychiatric wards.

In this study, the PSCSE scale was translated into Persian and its psychometric properties and structural factors were evaluated. When a tool is translated or used in another culture or society, its psychometric properties should be evaluated and customized in the target community.[26] Furthermore, face validity, content validity, construct validity, and reliability of this tool were assessed.

Based on the findings, the Persian version of PSCSE had good reliability and validity and the construct validity of its dimensions was similar to that of the original version.[16]

Given the PSCSE's CVI of 0.89, the Persian version of PSCSE had a good content validity, suggesting that the CVI value of 0.79 or higher should be considered as a standard for content validity of the scales.[21]

Of 41 items of the main questionnaire, item 14 (related to the skill dimension) was removed because it was not related to psychiatric wards (CVI = 0.27).

Construct validity was performed among 209 nurses working in psychiatric wards and quality of the model was confirmed for 40 items and three dimensions of knowledge, attitude, and skill, which were equivalent to the original version indicating appropriate construct validity of the Persian version. Designers of this tool reported a factor load of 0.43–0.99 for 41 items. The components' loads were different from the components obtained from the Persian version of scale.[16]

Regarding the Saudi version of this scale, quality of the model was confirmed for 41 items and three domains of knowledge, attitude, and skills, which was equivalent to the original version.[9] The Chinese version of the Construct Validity Questionnaire performed this questionnaire based on the results of confirmatory factor analysis and confirmed the reliability and validity of its construct at a high level for 41 items and dimensions of knowledge, attitude, and skills.[19] However, none of the studies reported factor loading, KMO, and Bartlett's test.

In our study, the convergent validity of PSCSE was assessed using the ASNC questionnaire.

In the study by Rashvand et al., to determine the content validity, a panel of experts, consisting of 11 nurse managers and nursing faculty members, and nine specialists in the field of safe nursing care were asked to determine CVR and CVI, respectively. They assessed its grammar, wording, item allocation, and scaling indices and reported CVr = 0.63 and CVI = 0.78.

EFA resulted in a four-factor solution, which accounted for 63.54% of the observed variance.[24] The Pearson correlation coefficient between Persian version of PSCSE and ASNC was moderate (r = 0.57, P < 0.001).

Other versions did not examine the correlation of this tool with other similar tools. Among the safety questionnaire, only the ASNC questionnaire was validated in Persian. Therefore, we evaluated the convergent validity of the Persian version of PSCSE with the ASNC questionnaire. It is noteworthy that no study has ever evaluated the convergent version of PSCSE with ASNC.

Discriminative validity of the scale revealed stability of the scale as the participants interpreted items similarly at the baseline and at the re-test. The Persian version of the scale revealed no floor or ceiling effects; however, we could not compare our findings due to dearth of evidence.

Correlation coefficient of the Persian version using test–retest was ICC = 0.92 for the whole instrument, which indicated acceptability and appropriateness of time reliability or repeatability of the questionnaire. The internal correlation was Cronbach's alpha = 0.95, showing that all subscales were within the acceptable range (Cronbach's alpha coefficient above 0.7).

In psychometrics of the original version of this tool (2011), its Cronbach's alpha was 0.90 for the whole scale and 0.85, 0.79, and 0.91 for the dimensions of knowledge, attitude, and skills, respectively.[16] These values are close to the coefficients obtained in this study showing the acceptable reliability of the initial version of the tool. In the Saudi version of the questionnaire, the total Cronbach's alpha was 0.91 for all domains.[15] Kim noted that the total Cronbach's alpha was 0.91, but he did not report the Cronbach's alpha for the domains.[18] Cho and Choi reported the Cronbach's alpha of 0.95 for the whole test and the Cronbach's alpha ranging from 0.87 to 0.95 for its dimensions.[19]

The Persian version of PSCSE scale indicated excellent test–retest reliability. We also evaluated the absolute reliability of the Persian version by calculating SEM and MDC. To the best of our knowledge, no study has ever evaluated the absolute reliability of this scale; hence, our findings could not be compared to other versions of the scale.

Finally, the Persian version of the Nurses' Competency Questionnaire in ensuring patient safety in psychiatric wards was obtained with 40 items in dimensions of knowledge (6 items), attitude (14 items), and skills (20 items).

The present study was probably the first to evaluate absolute reliability and floor/ceiling effects of PSCSE scale.

Limitations

Similar to any scientific research, our study had some limitations. First, we examined the Persian version of PSCSE psychometrics in only two psychiatric hospitals in southeastern Iran. Hence, the findings may be influenced by the participants' demographic and cultural characteristics. Second, due to the lack of a sufficient number of Persian safety and safe scales, only convergent validity of PSCSE with ASNC scale was evaluated.

Conclusion

The aim was to psychometric analysis of the Persian version of PSCSE in psychiatric hospitals in Iran. Based on the findings, Persian version of the PSCSE questionnaire has acceptable content validity, construct validity, and reliability to be administered in psychiatric wards. Due to the lack of valid and reliable tools in the field of nurses' competence in ensuring patient safety in Iran, the findings of this study can help researchers, managers, and nurses to provide comprehensive safe care. Moreover, this study can be a building block for future studies in evaluating the competence of nurses working in psychiatric wards. Due to the multidimensionality of patient safety and nurses' competence in rendering patient safety services, more studies should be designed using qualitative approach in this field.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.