Psychometric properties of the Persian version of the Patient Measure of Safety (PMOS).

Background: Patient safety practice reduces the adverse events that may occur in the health care system during procedures, diseases, and diagnoses. Failure and negligence in identifying and resolving health care system errors may result in financial and physical harm. Thus, this study aimed to investigate the psychometric properties of the Patient Measure of Safety in Hospitals (PMOS).
Methods: This study was conducted on 264 patients in 4 hospitals. The patient measure of safety questionnaire has 44 items and 9 domains. To translate the PMOS questionnaire, standardized forward-backward procedure was used, and a panel of experts assessed the face and content validity of the Persian version. Internal consistency, confirmatory factor analysis (CFA), and test-retest method were used to test the validity and reliability of the instrument. Also, AMOS (version 23) and SPSS (version 16) software were used for data analysis and modeling.
Results: The average CVI score was 0.85, indicating well results in the Persian context. CVR score was 0.65. The indices of goodness of fit were acceptable for Iranian sample (CFI=0.91, TLI=0.89, RMSEA=0.063, relative/normal Chi-Square Statistic (X2/df)=2.85). All items were significantly loaded on the domains, except the 33rd and 38th items that were related to the eighth domain. Thus, the final Persian version was developed with 8 domains and 42 items. Internal consistency was acceptable for these domains, and test-retest method showed a good reliability (r=0.984). Conclusions: The Persian version of PMOS is an appropriate instrument to assess the safety of patients in Persian language communities. Also, PMOS is an optimal tool to identify and avoid preventable errors.

Introduction

Patient safety practices are defined as those processes and structures that decrease the adverse events associated with medical care over diagnoses or conditions (1). Patient safety is a major problem associated with health care worldwide. According to a WHO report, medical errors are the third leading cause of death in the United States after heart disease and cancer. In the United Kingdom, recent estimations show that on average, 1 incident of patient harm is reported every 35 seconds (2). Health care safety is a main global concern. Unsafe and low-quality services lead to weakened health results and may lead to harm (2). Every year, unsafe health care leads to a major number of patients dying or suffering injuries (3).

The number of hospitalizations that take place every year is estimated at about 421 million. The number of adverse events that occur in these patients during their hospitalizations is estimated 42.7 million. Of these numbers, two thirds take place in low- and middle-income countries (2). In developing countries, such as Jordan, Kenya, Morocco, and Tunisia, 8.2% of the 15 548 reported medical errors led to adverse events. The range of these adverse events was about 2.5% to 18.4% per country and 30% of them led to the death of patients (4). Thus, millions of patients suffer from injuries because of system failures, improper care, or bad working conditions (3).

Moreover, individual domains, such as lack of knowledge, tiredness, and personal characteristics of staff can affect patients’ safety (5). Therefore, identifying the drawbacks and weaknesses in wards and hospitals’ systems is vital, and thus there is an urgent need to design a simple and inexpensive applicable instrument. The Hospital Survey on Patient Safety Culture (HSOPSC) questionnaire was one of these instruments. This instrument identifies the determinants of safety and can improve patient safety. It has 12 dimensions and 42 items and has been used in many countries (5-7). The validation study of this questionnaire in an Iranian sample was done by Moghri et al in 2012. They found that Patient Safety Culture questionnaire is a reliable and valid tool in Iran's hospitals (8). Mohebi Far et al applied this questionnaire in educational hospitals in Tehran and found that using this instrument can improve conditions in hospitals (9). In 2012, Momeni et al conducted a study to evaluate patient safety culture in Rajaie Cardiovascular, Medical and Research Center using this tool. They reported that using this instrument can help identify the weak and strong points of patient safety culture (10).

HSOPSC questionnaire measures patient safety from the perspective of the staff of hospitals. Patients can report precise information about medical errors and adverse events (11). Many of the events encountered by patients are not recorded by hospital systems, and thus patients can play a vital role in reporting and identifying these events in hospital settings (12, 13). To provide patients’ feedback and direct report on their safety in hospitals, it is essential to provide a tool. One of the tools provided by Giles et al is the Patient Measure of Safety (PMOS) and is completed only by patients. PMOS has 9 domains and 44 items and describes the key domains of patient safety in terms of patients perspective (14). The reliability and validity of the PMOS was tested in a hospital setting by Lawton R et al (15) and they found this tool to be both valid and reliable for use in hospital settings.

The aim of this study was to evaluate the psychometric properties of the Persian version of PMOS and provide a reliable instrument to measure patient safety in Iranian hospitals.

Methods

This cross sectional study was conducted in selected hospitals in Hamadan County from March to May 2017. This study was approved by the Ethics Committee of Hamadan University of Medical Sciences (IR.UMSHA.REC.1394.410). In this study, participants signed an informed consent form and their contribution was completely voluntary. There are 6 training hospitals in the city of Hamadan (the capital of Hamadan province) and they are all affiliated to Hamadan University of Medical Sciences. Stratified cluster sampling method was used to select the sample. Each training hospital was considered as a cluster. Then, for simplicity, 4 of these 6 hospitals were selected randomly (Besat, Farshchian, Sina, Beheshti). The wards of each hospital were regarded as a stratum and the patients of each ward were selected randomly and asked to complete the questionnaires. According to Heckler’s recommendation (The minimal number of observations for reliable results is more than 100 observations and 5 times the number of the items of the questionnaire.) and because of potential loss, 270 hospitalized patients aged 18 or older were selected from 12 wards of these hospitals. Parents completed the questionnaire for children under 18 years (16).

Six patients were discharged before an interview, so their information was not included in the study, and 264 patients remained in the study. For this study, exclusion criteria were being too ill to answer the questions and refusing to participate. A member of the PMOS research team was stationed at the patient's bedside to guide the patient and deliver the questionnaire. The patients were assured that their information will remain confidential. Response rate was 97.8%.

Patient measure of safety questionnaire

Patient measure of safety questionnaire has 44 items and 9 domains (14). All items are in the 5-point Likert scale, with 1 implying strongly disagree and 5 strongly agree. It also has the ‘not applicable’ option that patients can select. The questionnaire has 9 domains: (1) communication and teamwork (9 questions, with scores ranging from 9 to 81), (2) organization and care planning (5 questions, with scores ranging from 5 to 25), (3) access to resources (4 questions, with scores ranging from 4 to 20), (4) ward type and layout (12 questions, with scores ranging from 12 to 70) , (5) information flow (3 questions, with scores ranging from 3 to 15), (6) staff roles and responsibilities (4 questions, with scores ranging from 4 to 20), (7) staff training (2 questions, with scores ranging from 2 to 10), (8) equipment design and functioning (with 2 questions, with scores ranging from 2 to 10), (9) and delays (2 questions, with scores ranging from 2 to 10). It also has items that imply dignity and respect that are not linked to any domain at the beginning of the questionnaire.

Translation

For the systematic translation of PMOS questionnaire, the standardized forward-backward procedure was used. In the first step; 2 professionals separately translated the questionnaires from English to Persian and reached an agreement. They discussed the accuracy of statements in terms of correctness and conceptualization, and the final version of the translation closest to the English version was adopted.

In the second step, an expert who had no access to the original English version performed the back translation and translated the Persian version to English. Then, this version was compared carefully with the original English version and the final version was developed.

Face and content validity

After translating the questionnaire into Persian, face and content validity of PMOS was calculated. To obtain face validity, 10 patients from the mentioned hospitals studied the translated version and expressed their opinion to improve it. To obtain content validity, 10 experts were recruited from 4 selected hospitals. Content validity measures how essential a particular item is and to obtain it. Each of the experts responded to the following question for each item: “Is the skill or knowledge measured by this item 'essential,' 'useful, but not essential,' or 'not necessary' to the performance of the construct?" According to Lawshe, if more than half of the panelists indicate that an item is essential, that item has at least some content validity (17). To determine the relevance of the items (content validity index), each of the experts responded to the 4-point Likert question for each item,” (not relevant = 1 to highly relevant = 4), and the average of CVI for 44 items was regarded as the scale CVI. After this step, the questionnaires were distributed to be filled by patients. Data were collected by 2 MSc. students of Biostatistics and 1 MSc. student of nursing.

Statistical analysis

Confirmatory factor analysis (CFA) was performed to determine if the structure domain of the original 9 dimensions of the questionnaire could be used for the data set. With CFA, we could evaluate how well our Iranian data can be modeled using extracted domains.

The fit indices applied for CFA were as follow: (1) Comparative Fit Index (CFI), a CFI value above 0.90 indicates a good fit model and CFI value above 0.95 indicates an excellent fit model (18); (2) Tucker-Lewis Index (TLI >0.90 acceptable and >0.95 good fit); and (3) the next fit statistics focuses on the root mean square error of approximation (RMSEA). RMSEA value under 0.08 indicates a good fit model and RMSEA value under 0.05 indicates an excellent model fit (19).

In this study, relative/normal Chi-Square Statistic (X2/df) proposed by McIntosh (20) was reported instead of Chi-Square Statistic (21). First, Because Chi-square test assumes multivariate normality, and if deviations from normality severed, the model may be rejected (20). Second, because Chi-Square Statistic is sensitive to sample size (22). The acceptable range of relative/normal Chi-Square Statistic is from 2 to 5 (23, 24).

Cronbach’s alpha (α) was used to assess the internal reliability of the retained domains where ≥ 0.6 indicates that the items can measure the same concept. According to the recommendations of Briggs and Cheek to calculate internal reliability for domains with 6 or fewer than 6 in this study, average inter item correlations were used (25). To assess test-retest reliability, 20 patients were selected and a code was assigned to each patient. After 2 weeks, these patients were asked to complete the questionnaires again (26). Then, Pearson correlation was calculated. According to Cohen’s recommendation, correlations ≥ 0.3 indicate a medium and ≥0.5 indicate a large effect (27). For data analysis and modeling, AMOS (Analysis of Moment Structures) (version 23) and SPSS (Software Package for Social Sciences) (version 16) software were used. Significance level was set at 0.05 in all analysis.

Results

A total of 270 patients (115 male and 149 female) in 4 hospitals and 14 wards were identified, of whom 6 were discharged before an interview. Thus, the analysis was based on the data from 264 patients.

The mean age of parents who completed the PMOS questionnaire was 44.33 years (SD 11.36 years), and the mean age of all patients who participated in the study and completed the questionnaire was 49.42 years (SD 17.17 years). All participants were residents of city of Hamadan and other cities of Hamadan province. The majority of the participants (n=100, 37.9%) resided in the city of Hamadan, 195 were married (84%), and 145 did not have a college degree (54.9%). Summary of patients’ demographic characteristics by ward is demonstrated in Table 1.

Table 1

Response rates and sample description

Hospital ward	Consented and complete PMOS (percentage)	Mean age in year (SD)	Min-Max of age
Hematology	15 (5.7)	48.15 (18.7)	19-75
Digestion	9 (3.4)	52.33 (16.7)	27-75
Surgery	43 (16.3)	49.25 (17.6)	18-84
General	3 (1.1)	62.33 (23.46)	36-81
Respiratory	12 (4.5)	52.15 (20.17)	24-76
Vascular	79 (29.9)	58.62 (14.27)	23-82
Neurology and psychiatry	13 (4.9)	47.23 (18.25)	25-85
Trauma	13 (4.9)	45.72 (14.25)	20-74
Obstetrics and Gynecology	30 (11.4)	38.8 (15.98)	23-82
Orthopedic	12 (4.5)	28.09 (9.02)	18-44
Oncology	7 (2.7)	44.85 (11.7)	23-57
Poisoning	12 (4.5)	41.18 (12.20)	25-61
Pediatric	10 (3.8)	44.33 (11.36)	27-58
Ear, nose, throat and eyes	6 (2.3)	49.8 (11.38)	31-82
Total	264 (100)	49.42(17.17)	18-85

PMOS: Patient measure of safety

To ensure the validity of the Persian version of PMOS, the difficulty and ambiguity of the items were assessed by 10 patients. They suggested a number of changes in wording and structure that resulted in an improved version.

The CVI score, which was calculated by 10 experts, was found to be 0.85, representing an acceptable cultural relevance. CVR score for the Persian version of PMOS was 0.65, indicating that all items of the questionnaire were essential. The final Persian version of PMOS was distributed in hospitals to evaluate its construct validity.

Testing the original model (9-domains): CFA

For CFA, the original PMOS dimensions were used and showed satisfactory fit (CFI=0.91, TLI=0.89, RMSEA= 0.063, relative/normal Chi-Square Statistic (X2/df)=2.85). All items were significantly loaded on the domains, except the 33rd and 38th items which belonged to the eighth dimension. Nobody responded to the item 25 (Other (free-text only)), so this item was deleted from the analysis (Table 2).

Table 2

Domain loadings of the Persian version of PMOS questionnaire

Items	Domain loadings
Dignity and respect (not linked to any domains – comes at beginning of questionnaire on its own)
1. I was always treated with dignity and respect	0.81
Domain 1. Communication and team working
4. I got answers to all the questions I had about my care	0.642
5. Staff were always able to get advice from other teams about my care if needed	0.635
28. Staff always seemed to know what they were meant to be doing	0.758
40. Staff always agreed about my treatment/care	0.543
41. I always felt staff that listened to me about my concerns	0.619
43. When staff talked about my care with others the information they shared was correct	0.546
18. Sometimes there was no-one available to deal with aspects of my care R	-0.653
26. I felt that the attitude of staff towards me was poor R	-0.542
32. Staff did not work together as a team here R	-0.467
Domain 2. Organization and care planning
2. I knew who to go to if I needed to ask a question	0.563
3. The drugs I have been prescribed were always available in the hospital	0.575
6. A doctor changed my plan of care and other staff didn’t know about it R	-0.498
30. Staff gave me different information about my care R	-0.536
34. I have needed treatment and there was no-one available who was trained to do it R	-0.561
Domain 3. Access to resources
36. Nurses were always able to get help from other staff when they asked for it	-0.540
31. Staff/patients waited a long time for porters to arrive R	0.730
42. Staff seemed to struggle to get help when they needed it R	-0.520
39. Equipment and supplies were not always available when needed e.g. hoists, bed pans, drugs R	-0.502
Domain 4. Ward type and layout
16. Staff were prompt in answering my buzzer	0.51
15. I was on a ward that was not able to deal with my treatment needs R	-0.455
11. The following aspects of the ward made it difficult for staff to do their jobs: Position of nurses’ station R	-0.611
12. The following aspects of the ward made it difficult for staff to do their jobs: Lighting levels R	-0.573
13. The following aspects of the ward made it difficult for staff to do their jobs: Clutter & untidiness R	-0.617
14. The following aspects of the ward made it difficult for staff to do their jobs: Lack of space R	-0.457
20. The following aspects of the ward made it uncomfortable for me: Noise levels R	-0.462
21. The following aspects of the ward made it uncomfortable for me: Lighting levels R	-0.706
22. The following aspects of the ward made it uncomfortable for me: Temperature R	-0.721
23. The following aspects of the ward made it uncomfortable for me: Poor cleanliness R	-0.785
24. The following aspects of the ward made it uncomfortable for me: Lack of space R	-0.920
25.Other (free-text only)	-
Domain 5. Information flow
44. Information about me that my health care team needed was always available	0.559
35. Staff were kept waiting for my test results R	-0.707
7. After a shift change staff did not appear to know important information about my care R	-0.513
Domain 6. Staff roles and responsibilities
16. Staff were prompt in answering my buzzer	0.71
17. It was clear who was in charge of staff	0.568
27. I knew which consultant was in charge of my care	0.630
38. I always knew which nurse was responsible for my care	0.567
Domain 7. Staff training
9. On at least one occasion a member of staff was not able to use the necessary equipment R	-0.650
19. On at least one occasion a member of staff was not able to carry out a task that they should have been able to do R	-0.681
Domain 8. Equipment (design and functioning)
38. Equipment needed for my care was always working properly	0.043
34. There was equipment that staff found difficult to use (e.g. monitoring equipment, beds, hoists) R	-0.23
Domain 9. Delays
29. There were enough staff on the ward to get things done on time	0.535
10. My treatment/ procedure/ operation did not always happen on time R	-0.637

Reliability

Internal consistency was estimated for the translated version. The results of the reliability analysis of the eighth dimensions are displayed in Table 3. For domains with >6 items, the Cronbach’s alpha criteria were applied. Also, the average inter item correlations with values between 0.2 and 0.4 were acceptable for domains with 6 or fewer than 6 items, based on Briggs and Cheek recommendation (25). According to Cronbach’s alpha, only 1 domain yielded α>0.8 (Ward type and layout; α=0.823) and other domains were acceptable (communication and team working, α=0.730). According to the average of inter item correlation, the reliability of other dimensions was also acceptable.

Table 3

Internal reliability of the Persian version of the PMOS questionnaire

Domains	No. of items	Cronbach’s alpha (α)*	Average of correlation**
Communication and team working	9	0.730
Organization and care planning	5	-	0.319
Access to resources	4	-	0.319
Ward type and layout	12	0.823
Information flow	3		0.355
Staff roles and responsibilities	4		0.360
Staff training	2		0.390
Delays	2		0.337

Test-retest reliability

Test-retest reliability was excellent for PMOS questionnaire (r=0.984, p<0.001).

Discussion

Patients are able to identify safety issues and provide feedback about their safety in hospital settings and this can be an important source of information (11, 13, 28). Patient measure of safety is an important scale that allows patients to determine potential risks to safety in hospital settings. This was the first study in Iran to evaluate the psychometric properties of PMOS questionnaire among Iranians. The aim was to clarify cultural differences and comparison of results in measuring the concept.

Calculation of CVI and CVR was an advantage of this study. With acceptable CVI, it can be ensured that the original and translated items can be matched well technically and semantically. Model fit indices of CFA (relative/normal Chi-Square Statistic (X2/df), CFI, TLI, RMSEA) showed a good fit of the model for an Iranian sample. Moreover, the results of this study were similar to original PMOS, however, only the items of the eighth domain (33rd and 37th items) had small loading domains and were deleted from the Persian version. Cronbach’s alpha and the average inter item correlations showed that the dimensions had good reliability, and test-retest reliability test exhibited good repeatability of the instrument.

This result is similar to the original study in which the Cronbach’s alpha ranged 0.66–0.89 and test-retest reliability was good (r=0.75) (15).

Acceptable values for CVI and CVR Showed that the translated items adapted well with the original items and the items were equivalent with original English items (15). This finding is similar to the work of Najjar et al who considered the psychometric properties of Hospital Survey on Patient Safety Culture (HSOPSC) in an Arabic sample in which CVI was yielded 0.96 (29).

According to the results of confirmatory factor analysis, the model had a good fit and can be used in Iranian sample (8-10).

In this study, an optimal version of PMOS questionnaire with 42 items and 8 domains was made for an Iranian sample. This is an important tool with fewer items that is comparable with its English version and can measure patient’s safety in future studies (30-32).

The first limitation of this study was the low internal consistency of some domains compared to the original survey. This was the first study to consider the validity and reliability of the Persian version of PMOS questionnaire in the hospital setting and no similar study was conducted to compare the results across languages and cultures.

PMOS questionnaire is an instrument designed for hospital settings, and future studies may be conducted to determine whether this tool can be applied to other group of patients in different settings.

This tool is an optimal instrument that can be useful in hospitals to identify unsafe domains involved in patient safety, and other countries can test its validation on their own patients.

Conclusion

The Persian version of PMOS is similar to the original PMOS. Nevertheless, the domain loadings of 2 items were low and exclusive, which led to improvement of the fit and reliability of the instrument. The findings of this study revealed that the Persian version of PMOS is an appropriate instrument to evaluate the safety of patients in Persian language communities. To provide high-quality health services, preventable errors should be avoided. PMOS is an optimal tool to identify such errors and can improve the quality of health services. Different communities can conduct the psychometric evaluation of the translated versions and can use PMOS in their health care settings.

Acknowledgments

We gratefully thank all staff members in the hospitals of Hamadan who contributed to recruitment, data collection, and interviewing. We thank the participants for their time and interest in the study. The study was funded by The Vice-chancellor for Research and Technology of Hamadan University of Medical Sciences (Grant number: 950131291).

Conflict of Interests

The authors declare that they have no competing interests.