Development and Psychometric Validation of a Questionnaire to Evaluate Knowledge and Attitude Towards Medication Error Reporting Among Pharmacists.

PURPOSE: Assessment of medication errors (ME) is crucial to improving the quality of health care. A questionnaire that can be used to explore pharmacists' perspectives regarding ME would be very useful as part of an ongoing process of quality improvement in patient care. The aim of this study was to develop and validate a questionnaire to measure perceived causes of ME and attitude towards ME reporting among pharmacists.
METHODS: The questionnaire was developed from the literature together with outcomes from focus group discussions. It was divided into two domains which are knowledge on ME and attitude towards ME reporting. Content validity index (I-CVI), exploratory factor analysis (EFA), Cronbach alpha and intraclass correlation coefficient (ICC) to assess test-retest reliability were obtained during the validation process.
RESULTS: Overall Cronbach alpha for internal consistency was good (0.742), where subscale of the questionnaire demonstrated adequate internal consistency, with Cronbach alpha value 0.83 for knowledge and 0.70 for reporting behaviour attitude. The I-CVI showed good scores (knowledge=0.88) and (attitude=0.81), while ICC was moderately accepted with a value of 0.77. Two factors were extracted from the 16 items in EFA.
CONCLUSION: The questionnaire to assess knowledge on ME and attitude towards ME reporting among pharmacists is valid and reliable. It demonstrates good psychometric properties.

Introduction

Medication error (ME) is defined as any preventable event that may cause or lead to inappropriate medication use or patient harm while the medication is in the control of the health care professional, patient or consumer.1 It can occur at any of the five stages of a medication use process. In some studies, the majority of medication error have been found to occur in the prescribing stage,2,3 while others have shown that ME predominantly occurred in the administration stage.4,5 In each stage of a medication process, different types of MEs can occur. For example, in settings without electronic prescriptions, illegible handwriting may contribute to prescribing error.6,7 In ambulatory care setting, the types of ME at the dispensing stage include labelling and filling errors.8 In the inpatient setting, nurses are usually involved in medication administration activities and the most common type of administration error reported was wrong time error.9

Pharmacists are essential health-care providers who can help ensure patients receive their medications safely. Although they are not usually directly involved with prescribing and administering medications, they play important roles to ensure that each patient receives the right medication in the right dose via the right route and at the right time. To perform these duties, they should be able to rapidly detect and prevent ME before they reach and possibly harm patients. At the same time, they should also see ME reporting as a learning opportunity to improve health care delivery process and patient safety.

ME documentation and reporting in Southeast Asian countries are inadequate.10 In Malaysia, MEs are reported online to the national medication error reporting system. Compared to other health professionals, pharmacists have contributed significantly to the overall ME reporting.2 Nevertheless, there exist strong barriers to it including poor knowledge and wrong attitude.11,12 For a successful implementation of patient safety program, an appropriate reporting culture and feedback mechanisms should be in place.13 Therefore, an assessment tool to explore pharmacists’ perspectives regarding ME would be very useful as part of an ongoing process of quality improvement. The aim of this study was to develop a valid and reliable questionnaire to measure perceived causes of ME and attitude towards ME reporting among pharmacists working in hospitals and healthcare clinics. Psychometric data are also presented.

Materials and Methods

Development of the Questionnaire

The questionnaire was developed in two phases. Phase I was for questionnaire development, while Phase II was for psychometric validation.

A focus group discussion (FGD) with five senior pharmacists, each with more than 10 years working experience, was conducted to obtain in-depth information related to ME. All the discussions were recorded and transcribed for further analysis. Two themes were identified, which were knowledge on ME and attitude towards ME reporting. Using feedback from the FGD and questionnaires used in other similar studies,11,12,14,15 we developed 10 questions on perceived causes of ME and six questions on attitude towards barriers to ME reporting. The questionnaire used a 5-point Likert scale, ranging from “strongly agree” to “strongly disagree”.

Validation of the Questionnaire

Psychometric validation was conducted in Phase II. This process was divided into three stages, (i) Content validity by utilizing items-content validity index (I-CVI) technique, (ii) Test retest in 40 respondents within 14 days apart and (iii) Exploratory factor analysis (EFA) to explore pattern of domains and also to act as a pilot study. Pharmacists were selected by convenient sampling from two public hospitals and 10 primary health-care clinics in the state of Pahang, Malaysia. Data were collected using a web-based survey at , which was emailed to 200 pharmacists. Within the survey, information explaining the purpose and methodology of the study together with consent statements were provided. Pharmacists were required to provide informed consent by clicking on the radio button/check box before they could proceed with the survey.

Reliability: Internal Consistency and Temporal Stability

To measure the reliability of the questionnaire, Intraclass Correlation Coefficient (ICC) was used to assess temporal stability while Cronbach alpha was used to analyze internal consistency. Individual item analysis was utilized by calculating the corrected item-total for ICC and Cronbach alpha value. Items that were below references value were removed. For Cronbach alpha, items within the range of 0.60 (moderately good) till 0.95 (excellent) were retained in the questionnaire.16

Content Validity: Content Validity Index for Items (I-CVI) and Construct Validity

Content validity index is an index of interrater agreement that expresses the proportion of agreement among a team of expert.17 I-CVI was defined as the number of experts giving a rating either 3 or 4 divided by the total number of experts. I-CVI was established by gaining expert agreement from seven experts. Overall CVIs were calculated based on the average of I-CVIs.17 Construct validity was determined by utilizing the factorial structure with exploratory factor analysis (EFA). EFA was performed in 186 respondents. The principal axis factoring method was used and the rotated matrix was extracted with varimax orthogonal extraction.

Data were imported into Microsoft excel to calculate I-CVI. All other statistical analyses were performed using SPSS software version 23.

Ethics Considerations

This study was approved by the Malaysia Research Ethics Committees (MREC), Ministry of Health, Malaysia [NMRR-19-476-45627 (IIR)].

Results

Characteristics of Respondents

Different phases of this study had different number of respondents. Table 1 summarizes the characteristic of the respondents in this study from the development until the validation of the questionnaire. EFA was based on data collected from 186 respondents (93% response rate). The majority were female (74.2%). The mean age (± SD) of these respondents was 28.5 ± 3.17 years old. There were more respondents working in hospital (52.7%) than those in primary health-care clinics (47.3%). The majority of respondents (67.2%) had less than 8 years working experience.

Table 1

Respondents Characteristics for Different Phases of the Study

Phase	N	Age, Mean (± SD)	Gender, Male %/Female %
Development (FGD)	5	39.7 (4.55)	20%/80%
Intraclass Correlation Coefficient (ICC)	40	26.5 (1.15)	30%/70%
Content Validity (I-CVI)	7	37.7 (3.15)	10%/90%
Construct Validity (EFA)	186	28.5 (3.17)	23%/67%

Abbreviation: FGD, focus group discussion.

Experts in FGD agreed that pharmacists who have good knowledge on ME will tend to have good attitude towards reporting. Findings obtained from FGD were used to construct the questionnaire. The first formulated questionnaire for this study was divided into two parts. The first part was for demographic information consisting of six items. The second part was for knowledge on ME (10 items), and for attitude towards ME reporting (six items).

Reliability: Internal Consistency

The internal consistency to assess reliability was conducted among 186 respondents. The Cronbach α as shown in Table 2 demonstrates that knowledge domain was excellent (α=0.85), while for attitude it was acceptably good (α=0.67). The overall Cronbach alpha was 0.74. Intraclass Correlation Coefficient conducted using test-retest methods among 40 respondents was found to be 0.77 (95% CI, 0.69–0.83).

Table 2

Cronbach Alpha and Corrected Item Total Correlation

Items in Questionnaires	Mean (± SD)	Corrected Item- Total Correlation	Cronbach α If Item Deleted	Cronbach’s α
Knowledge on causes of medication error (ME)
ME occurs when pharmacist fails to ensure the right patient receiving the right drugs with right dose and quantity	1.64 (0.76)	0.48	0.84
ME occurs when physician’s writing on the prescription is difficult to read or illegible	1.62 (0.69)	0.63	0.83
ME occurs when medication labels are of poor quality or damaged	1.92 (0.85)	0.53	0.84
ME occurs when there is confusion between drugs with similar names	1.53 (0.55)	0.58	0.84
ME occurs when physician prescribes the wrong dose	1.76 (0.82)	0.52	0.84	0.85
ME occurs when pharmacist miscalculates the dose	1.56 (0.59)	0.73	0.83
ME occurs when pharmacist fills the wrong drug	1.49 (0.60)	0.68	0.83
ME occurs when pharmacist tells wrong or confusing instruction during dispensing	1.80 (0.72)	0.62	0.83
ME occurs when pharmacist is distracted by other patients or coworkers	1.73 (0.74)	0.41	0.85
ME occurs when pharmacist is tired and exhausted	1.80 (0.74)	0.48	0.85
Attitude towards medication error reporting
I fail to report the ME because I am afraid of the reaction from superior	2.89 (1.02)	0.59	0.60
I fail to report the ME because I am afraid of the reaction received from my coworkers	3.06 (1.10)	0.58	0.60
I fail to report the ME because most of my colleagues ignore the reporting	3.38 (1.12)	0.51	0.60	0.67
I fail to report the ME because I think the error is not serious to warrant reporting	4.15 (1.05)	0.36	0.64
I fail to report the ME because I am afraid that I might lose the job	2.96 (1.37)	0.41	0.62
I fail to report the ME because the error does not reach or harm the patient	3.58 (1.05)	0.21	0.68

Note: Overall Cronbach α = 0.74.

Content Validity (I-CVI)

I-CVI obtained for this study was good. The average I-CVI in knowledge domain was 0.88 (Table 3). There were four items where all seven experts were in agreement for those items. For the domain on attitude towards ME reporting (Table 4), the I-CVI value was 0.81 and there were two items that had the same agreement between all experts. Both domains obtained values more than 0.78, hence, all items in both domains were retained for exploratory factor analysis (EFA).

Table 3

Rating on a 10-Item Scale for Knowledge on Medication Error by Seven Experts: Items Rated 3 or 4 on 4-Point Relevant Scales

Item	Expert 1	Expert 2	Expert 3	Expert 4	Expert 5	Expert 6	Expert 7	Expert in Agreement	Item CVI
1	✓	✓	✓	✓	✓	✓	✓	7	1.00
2	✓	✓	–	✓	✓	✓	✓	6	0.86
3	✓	✓	✓	✓	✓	–	–	5	0.71
4	✓	✓	✓	✓	✓	✓	✓	7	1.00
5	✓	✓	✓	✓	✓	✓	✓	7	1.00
6	–	✓	✓	✓	✓	–	✓	5	0.71
7	✓	✓	✓	–	✓	✓	✓	6	0.86
8	✓	–	✓	✓	✓	✓	✓	6	0.86
9	✓	✓	✓	✓	✓	✓	✓	7	1.00
10	✓	✓	✓	✓	✓	✓	–	6	0.86
								Average I-CVI=	0.88
Proportion Relevant	0.90	0.90	0.90	0.90	1.00	0.80	0.80

Abbreviation: CVI, content validity index.

Table 4

Rating on a 6-Item Scale for Attitude on Reporting the Medication Error by Seven Expert: Items Rated 3 or 4 on 4-Point Relevant Scales

Item	Expert 1	Expert 2	Expert 3	Expert 4	Expert 5	Expert 6	Expert 7	Expert in Agreement	Item CVI
1	✓	✓	✓	✓	–	✓	✓	6	0.86
2	✓	✓	–	✓	✓	✓	✓	6	0.86
3	–	✓	✓	✓	✓	✓	–	5	0.71
4	✓	✓	✓	✓	✓	✓	✓	7	1.00
5	✓	–	✓	–	✓	✓	✓	5	0.71
6	–	✓	✓	✓	✓	–	✓	5	0.71
								Average I-CVI	0.81
Proportion Relevant	0.67	0.83	0.83	0.83	0.83	0.83	0.83

Abbreviation: CVI, content validity index.

Table 5 shows the results of EFA. Two factors were extracted among 16 items in this validation questionnaire process. These factors were “Knowledge on ME” and “Attitude towards ME reporting”. For “Knowledge”, items were found to have communalities ranging from 0.310 to 0.653. Meanwhile for “Attitude towards ME reporting”, the communalities for each item varied from 0.266 to 0.528. Factor loading ranged from 0.409 to 0.808 for “Knowledge”, and 0.315 to 0.724 for “Attitude towards ME reporting”. All items were retained in the EFA model as no deletion was required.

Table 5

Exploratory Factor Analysis for Medication Error Questionnaire

Factor	Item	Factor Loading	Communality
Knowledge:	K1	0.526	0.416
	K2	0.681	0.520
	K3	0.556	0.310
	K4	0.626	0.395
	K5	0.582	0.653
	K6	0.808	0.569
	K7	0.748	0.569
	K8	0.698	0.508
	K9	0.409	0.480
	K10	0.474	0.471
Attitude:	A1	0.675	0.476
	A2	0.724	0.528
	A3	0.635	0.432
	A4	0.544	0.302
	A5	0.678	0.460
	A6	0.315	0.266

Notes: Kaiser-Meyer-Olkin (KMO) was 0.732, Bartlett’s Test of sphericity was significant (p<0.001), Principal axis factoring was applied. K=knowledge item, A=attitude item.

Discussion

This study provides sufficient face validity, content validity, construct validity, internal consistency and test-retest reliability of a self-administered questionnaire on knowledge and attitude towards ME reporting among pharmacists. To the best of our knowledge, this is the first psychometric validation of a self-administered tool to assess the level of knowledge and attitude towards ME reporting among pharmacists working in government hospitals and health centers in Malaysia.

In the development phase of the questionnaire, a focus group discussion (FGD) was conducted between five senior pharmacists who had at least 10 years of experience in pharmacy practice. FGD is commonly used as a method for face validity where crucial information related to the study objective are discussed thoroughly between all experts. FGD is a very crucial and important method during questionnaire development phase where a researcher can use FGD as a tool to identify and discuss main issues related to the study objectives. Findings from the FGD in combination with those from the literature can be utilized to develop a new questionnaire.18,19

Content validity was obtained for this study by measuring the value of I-CVI. Sufficient values were obtained for both domains which indicated the overall content in each domain were acceptably good and retained in the questionnaires. I-CVI is a method to measure the interrater argument calculated for each item in the questionnaire. Hence, this method enables researchers to amend or retain the items according to the scale given by the experts.20 Polit and Beck described that I-CVI is a good method to study the validity of the items in the questionnaires and have suggested the value for I-CVI must exceed 0.78 if more than 6 experts were involved, whilst if the number of experts was five or fewer, all experts must be in agreement and the value will become 1.00.17 Findings from this study have shown that the I-CVI values for both domains are knowledge = 0.88 and attitude = 0.81, hence all the items in both domains are acceptable.

Meanwhile, for construct validity, the outcome was analyzed to determine the number of factors in the questionnaires by using exploratory factor analysis (EFA). The value of this analysis represents the direct correlations between the items and the factors. From 16 items in the questionnaire construct, all items were extracted into two factors which confirmed the discriminant validity between items in both domains. According to Hair et al, the recommended value for factor loading are 0.5 but 0.3 is the minimum value for the item to be accepted.21 The EFA obtained from this study shows all items were above 0.5 factor loading except three items which were between 0.32 and 0.47. All items in the questionnaires were retained and no deletion were required.

In terms of construct validity for hypothesis testing, the relationship between the score on knowledge and attitude on ME reporting was assessed. Spearman’s Rank correlation coefficient was utilized to determine the relationship between these domains (r = 0.10; p<0.05). This correlation was in positive direction, unfortunately the relationship was very weak. This finding shows that, regardless of the knowledge on ME, the relationship is just minimal towards the attitude of reporting the ME. A similar study by George et al has shown that good knowledge on ME did not give significant effect on attitude on reporting the ME.22

Regarding the reliability, the values for Cronbach alpha for both domains; knowledge and attitude demonstrate an acceptable internal consistency which allows the use of this questionnaire in descriptive population studies. The analysis of each item in both domains affirmed that none of the items were needed to be removed in order to give significant increase of α as suggested by some researchers.22–25

Temporal stability, calculated using ICC from this study shows a high degree of reliability where the ICC value was 0.77 (95% CI, 0.69–0.83). A few authors have suggested that ICC values of less than 0.5 are indicative of poor reliability, values between 0.5 and 0.75 indicate moderate reliability, values between 0.75 and 0.9 indicate good reliability, and values greater than 0.90 indicate excellent reliability. Thus, ICC value for this study demonstrates good reliability, and all items in this study are acceptable.16,26,27 Polit et al have suggested that ICC can also be used to reliably determine the chance of agreement between respondents and measure the redundancy of items in the questionnaires.27

The study has several limitations. Firstly, the questionnaires were conveniently distributed among pharmacist in hospitals and primary health clinics, thus the convenience sampling employed in this study might have resulted in sampling bias. Since the aim of this study was to assess the psychometric properties of a self-administered measure, this type of sampling method will not significantly affect the findings. Secondly, due to the nature of data collection in this study which was self-administered, the results could be exposed to recall and social-desirability biasness. Thirdly, this study did not measure criterion validity which can be associated with the study objective. This limitation occurred due to limited “gold standard” questionnaire for pharmacists on ME which can be used in the validation process. We believe that further studies can use this validated questionnaire as a criterion to develop new questionnaires.

Conclusion

The questionnaire developed from this study is a valid and reliable instrument to assess the knowledge on ME and attitude on ME reporting among pharmacists as it has shown a good psychometric value. Policy maker, researchers and pharmacists can use this questionnaire as a tool to measure perceived causes of ME and attitude towards ME reporting among pharmacists working in hospitals and healthcare centers.