Validation of a survey tool to assess the patient safety attitudes of pharmacy students.

OBJECTIVE: Patient safety education is a key strategy to minimise harm, and is increasingly being introduced into junior pharmacy curricula. However, currently there is no valid and reliable survey tool to measure the patient safety attitudes of pharmacy students. This study aimed to validate a modified survey tool, originally developed by Madigosky et al, to evaluate patient safety attitudes of junior pharmacy students.
DESIGN: A 23-item cross-sectional patient safety survey tool was utilised to evaluate first and second year pharmacy students' attitudes during May 2013 with both exploratory and confirmatory factor analyses performed to understand the psychometric properties of the survey tool and to establish construct validity.
SETTING: Undergraduate university students in Sydney, Australia PARTICIPANTS: 245 first year and 201 second year students enrolled in the Bachelor of Pharmacy Programme at The University of Sydney, Australia in May 2013.
RESULTS: After exploratory factor analysis on first year student responses (55.76% variance explained) and confirmatory factor analysis on second year responses, a 5-factor model consisting of 14 items was obtained with satisfactory model fit (χ(2) (66)=112.83, p<0.001, RMSEA=0.06, CFI=0.91) and nesting between year groups (Δχ(2)(7)=3.079, p=0.878). The five factors measured students' attitudes towards: (1) being quality improvement focused, (2) internalising errors regardless of harm, (3) value of contextual learning, (4) acceptability of questioning more senior healthcare professionals' behaviour and (5) attitude towards open disclosure.
CONCLUSIONS: This study has established the reliability and validity of a modified survey tool to evaluate patient safety attitudes of pharmacy students, with the potential for use in course development and evaluation. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Although there are many studies evaluating the patient safety attitudes of health care professionals, there is no validated published survey tool to evaluate the patient safety attitudes of pharmacy students.

This study adapted one of the most highly utilised survey tools that measure patient safety attitudes of healthcare students in order to suit pharmacy students.

This survey tool was then validated using a rigorous analytical approach, employing both exploratory and confirmatory factor analyses.

This validated survey tool can be used to measure and evaluate the effectiveness of patient safety interventions across five key domains.

Introduction

Patient safety has become a key priority for health systems around the world since the publication of the seminal reports To Err is Human1 and An Organisation with a Memory2 15 years ago. In 2002, the World Health Organization (WHO) member states recognised the need to reduce the harm and suffering that patients and their families experience from healthcare errors, and agreed on a resolution to improve patient safety. Education has since been considered a crucial element in minimising patient harm.3 In 2011, the WHO published a multiprofessional Patient Safety Curriculum Guide to assist healthcare schools to implement patient safety education.4 However, the implementation of patient safety specific education can be challenging in already full university teaching curricula.5

Most pharmacy degree programmes currently include education on some elements from the WHO curriculum, including aspects related to medication safety, communication and patient-centred care.6 Many students now gain work experience in healthcare settings at an earlier stage of their degrees, either through experiential placements or through casual employment and therefore, there is a greater emphasis on the need to integrate patient safety education earlier on in professional degree programmes.7 8 In response to this, many pharmacy schools now incorporate patient safety education earlier in the curriculum.9 10 Although evaluating patient safety knowledge is a key consideration when undertaking curriculum evaluation, it is also crucial that patient safety attitudes are understood and evaluated. This is particularly important in light of evidence that attitudes can considerably influence behaviours.11

There are a number of survey tools that have been used to the measure patient safety attitudes and values of healthcare students, each to varying degrees.9 10 12–19 The most widely adapted and validated tool is the Patient Safety/Medical Fallibility Survey, originally developed by Madigosky et al12 for use in medical students. Previous studies among pharmacy students have focused on evaluating patient safety knowledge and practice beliefs using unvalidated survey tools,9 10 and attitudes to patient safety have been largely unstudied. Therefore, the aim of this study was to validate an adaptation of Madigosky et al's12 survey tool in order to evaluate patient safety attitudes and values of junior pharmacy students, and specifically understand the psychometric properties that underpin the survey.

Methods

A cross-sectional survey was conducted among first (n=281) and second (n=269) year undergraduate pharmacy students enrolled in the 4-year Bachelor of Pharmacy programme at the University of Sydney. As both year groups would have completed an introductory pharmacy practice unit of study and introductory clinical placements (4 hours) at the time of survey completion, it was hypothesised that these two groups of students would have the most comparable clinical experience and be suitable participants in the validation of the survey instrument. Data were collected between 27 and 31 May 2013.

Survey

Survey modification

The survey tool was adapted from the Patient Safety/Medical Fallibility Curriculum Survey developed by Madigosky et al.12 Specifically, the original survey items that were included, suited first year Bachelor of Pharmacy students’ level of knowledge and understanding of healthcare systems, which resulted in the exclusion of skill and knowledge-based items from the original survey. The survey consisted of two sections. The first section consisted of 23 attitudinal items, and included 17 of the original 18 attitudinal items and utilised the original five-point Likert-type scale to measure student attitudes, with possible responses ranging from strongly disagree to strongly agree. As the survey was being modified for pharmacy students, the use of the term ‘physician’ was changed to ‘pharmacist’ or ‘medical practitioner’ based on the item. Items that related to the reporting of errors were split into two items to evaluate whether students’ responses would change due to the presence or absence of patient harm. In addition, two items to evaluate attitudes towards questioning more senior healthcare professionals, one item on patients’ role in healthcare and one item on peer learning were added. The second section collected demographic details including gender, age, stage of education, prior healthcare experience and involvement with an incident that resulted in harm or potential harm as a result of receiving healthcare.

Face validation

The face validity of the survey instrument was assessed through focus groups among three populations: initially among five pharmacy academics, five practising pharmacists and seven pharmacy student representatives. Based on feedback from the three groups, one of the original questions relating to uncertainty in healthcare was considered ambiguous and was removed from the final survey tool. Pharmacy academics also perceived that due to junior pharmacy students’ limited clinical experience, definitions of ‘Patient Safety’, ‘Error’ and ‘Incident’ should be included in the pretext to the survey. The student group were provided with terms defined by a range of healthcare organisations. As a result, the definitions used by the Australian Commission for Safety and Quality in Healthcare were selected due to both the perceived ease of understanding and perceived contextual relevance to junior pharmacy students. The final survey was approved by each group in a subsequent focus group.

Analysis

All data analyses were completed using IBM SPSS Statistics V.21 (SPSS Inc, Chicago, Illinois, USA) and AMOS V.21 (Amos Development Corporation, Crawfordville, Florida, USA). Surveys with missing data were excluded from the analysis. The survey response rate was calculated by dividing the total number of surveys completed by the number of students enrolled in each year group. Participant characteristics were compared across year groups using χ2 tests for categorical variables and Mann-Whitney and Kruskal-Wallis tests for continuous variables. In addition, the potential relationship between each of the participant demographic characteristics and their effects on survey responses were evaluated. A Bonferroni correction was applied to account for multiple comparisons, reducing the p value for significance to 0.002.

An exploratory factor analysis (EFA) was performed on survey responses from the first year students to understand the latent structure underpinning student responses to the survey using maximum likelihood estimation and varimax rotation. As adequate sample sizes across both year groups were obtained, Kaisers criterion for factor retention was adopted with individual factors loading greater than 0.25 considered significant for retention.20 The factor structure was assessed for a theoretical basis, with an examination of the Scree plot used to verify the number of factors retained.

The construct validity of the survey was evaluated using a confirmatory factor analysis (CFA) on the survey responses from the second year students. Each item was considered to have a latent construct and a measurement error, with both causal effects depicted by unidirectional arrows. Correlations between variables within the model were depicted using bidirectional arrows.21 Maximum likelihood estimation was performed to calculate item loading. Items were removed from the model where there were: poor factor loading scores (being less than 0.25), insufficient number of items loading on the construct, or an insufficient theoretical basis to the construct after item removal.20

Boomsma's method of estimating a minimum sample size to conduct a CFA was performed based on the number of items to number of factors ratio of the model; it was estimated that 200 student responses would be adequate.22 To evaluate the goodness of fit of the model, a number of fit statistics were examined. First, the χ2 statistic was used to evaluate model parsimony (ie, that the model accomplishes a desired level of explanation with as few variables and relationships between variables as possible). In addition, root mean-square error of approximation (RMSEA) was used to evaluate absolute fit (a measure of how well the data fits the proposed model) and the Comparative Fit Index (CFI) was used to evaluate the comparative fit (a measure of how well the data fits a model where relationships exist between the survey items compared to a model where no relationships exist).23 24

Results

Participant characteristics

A total of 245 first-year and 201 second-year pharmacy students completed the survey, resulting in survey response rates of 87.5% and 74.7%, respectively. The characteristics of the first and second year students are compared in table 1. There were very few differences in the characteristics between the two groups of students, with the only significant difference being the number of students engaged in current employment in a pharmacy (15.6% vs 44.4%, p<0.001) and mean months worked (2.4 vs 6.9, p<0.001). However, as most students that are engaged in employment in pharmacy are undertaking non-clinical roles (19.7% vs 8.6%), it is unlikely that current employment will influence junior students’ responses to the survey questions.

Table 1

Participant characteristics

Characteristic	First year students (n=245)	Second year students (n=201)	p Value
Gender
Males, n (%)*	90 (36.9)	65 (32.8)
Females, n (%)*	154 (63.1)	132 (66.7)	0.37
Age, in years, mean (SD)	19.4 (3.1)	20.0 (2.0)	<0.001
Students currently working in a pharmacy, n (%)*	38 (15.6)	88 (44.4)	<0.001
Months worked in pharmacy (mean, SD)	2.4 (9.5)	6.7 (11.9)	<0.001
Students who have been involved in or witnessed harm while working, n (%)*	21 (9.7)	29 (11.9)	0.06
Students who have witnessed harm to a loved one, n (%)*	35 (15.9)	35 (19.1)	0.14

*percentages based on denominator of number of valid responses only.

Comparisons of year group and other demographic characteristics with each of the survey items showed that demographic characteristics did not influence student responses after accounting for multiple comparisons with the Bonferroni correction (see online supplementary appendix 1). However, 2 of the 23 attitudinal items which related to the inevitability of errors in healthcare and involving the patient in healthcare, showed statistical significance between year groups (p=0.001). Prior to the EFA, these items were removed as their exclusion was deemed not to have a significant impact on the model due to the inclusion of other survey items which measured similar concepts.

Exploratory factor analysis

Following the removal of seven items, either due to low communalities (less than 0.2) or low factor loadings (less than 0.25) and examination of the Scree plot, a five-factor solution was determined (table 2). This solution explained 55.71% of the variance. Only one item cross-loaded and was assigned to a single factor based on theoretical reasoning. The five factors were labelled as (1) being quality improvement focused; (2) value of contextual learning; (3) internalising errors regardless of harm; (4) acceptability of questioning more senior healthcare professionals’ behaviour and (5) attitude towards open disclosure of errors.

Table 2

EFA rotated factor structure

		EFA constructs
Question number	Item	1α=0.422	2α=0.673	3α=0.591	4α=0.533	5α=598	Cronbachs α if item deleted
Q7	Learning how to improve patient safety is an appropriate use of time in pharmacy programmes at university	0.62					0.22
Q3	Pharmacists should routinely spend part of their professional time working to improve patient care	0.48					0.32
Q23	Peer-led education, such as from pharmacist colleagues or fellow students can help my understanding of patient safety concepts	0.47					0.28
Q19	The care that we provide on a day-to-day basis could be improved	0.47					0.37
Q5	Patients have a role to play in their own safety	0.38					0.34
Q18	After an error occurs, an effective strategy is to work hard to be more careful	−0.26					0.63
Q22	Patient safety education requires university lecturers to teach patient safety concepts		−0.78				0.40
Q8	Healthcare professionals, including pharmacy staff, routinely share information about errors and what caused them		0.76				0.54
Q6	The culture of the pharmacy workplace makes it easy for pharmacy staff to deal constructively with errors		0.40				0.74
Q16	If I saw an error that DID NOT cause harm, I would keep it to myself			0.86			0.37
Q15	If I saw an error that DID cause harm, I would keep it to myself			0.48			0.49
Q14	If there is no harm to a patient, there is no need to address an error			0.42			0.58
Q10	Pharmacists should report errors to an affected patient and their family if harm to the patient has occurred				0.97		–
Q11	Pharmacists should discuss and report errors to an affected patient and their family even if the patient is NOT harmed				0.38		–
Q21	It is acceptable for a registered pharmacist to question the decisions of a prescriber (such as a doctor or nurse practitioner)					0.97	–
Q20	It is acceptable for an intern pharmacist to question the actions of a registered pharmacist	0.34				0.36	–

Note that values less than 0.25 suppressed. EFA, exploratory factor analysis.

Confirmatory factor analysis

In the second phase of the analysis, the construct validity of the instrument was established using CFA. After mapping the responses from the second year students to the suggested model determined by the EFA of first year students’ responses, two items (Q5 and Q18) were removed due to low factor loading (less than 0.25), resulting in the final factor structure (table 3). The χ2 values for overall model fit was significant, χ2 (69)=134.23, p<0.001, which suggested a significant misfit between the data and the model. However, it is known that in larger samples, the χ2 value can be over-sensitive and other fit indices were assessed (RMSEA=0.07 CFI=0.88), which suggested potential fit.25 Modification indices suggested that freeing the covariance between two error terms in factor 1, and one error term in factor 3, as well as between one error term in factor 2 and one error term in factor 3, would improve model fit. A model including these specified correlations resulted in a subsequent model having better fit to the constrained model, χ2 (66)=112.83, p<0.001, RMSEA=0.06, CFI=0.91. Utilising data from both first year students and second year students as part of a multigroup analysis, unconstrained nested model comparisons showed no significant difference in the unconstrained model between year groups (Δχ2(7)=3.079,p=0.878). This indicates that both year groups satisfactorily fit the model. The combined data set of first and second year student responses (N=446) was used to calculate the final factor loadings as seen in figure 1.

Table 3

Final CFA factor structure

Explanation of factor structure
Item number	Item description	Standardised regression weights	Unstandardised regression weights (URW)	Standard error of URW	Squared multiple correlations
Factor 1: Being quality improvement focused (α=0.654)
Q19	The care that we provide on a day-to-day basis could be improved	0.40	1.00	0.39	0.16
Q3	Pharmacists should routinely spend part of their professional time working to improve patient care	0.60	1.49	0.28	0.36
Q7	Learning how to improve patient safety is an appropriate use of time in pharmacy programmes at university	0.60	1.53	0.30	0.36
Q23	Peer-led education, such as from pharmacist colleagues or fellow students can help my understanding of patient safety concepts	0.57	1.44	0.31	0.33
Factor 2: Internalising errors regardless of harm (α=0.705)
Q16	If I saw an error that DID NOT cause harm, I would keep it to myself	0.72	1.00	0.45	0.52
Q15	If I saw an error that DID cause harm, I would keep it to myself	0.65	0.63	0.27	0.42
Q14	If there is no harm to a patient, there is no need to address an error	0.53	0.63	0.49	0.28
Factor 3: Value of contextual learning (α=0.570)
Q22	Patient safety education requires university lecturers to teach patient safety concepts	0.95	1.00	0.06	0.90
Q8	Healthcare professionals, including pharmacy staff, routinely share information about errors and what caused them	−0.59	−0.68	0.48	0.34
Q6	The culture of the pharmacy workplace makes it easy for pharmacy staff to deal constructively with errors	−0.34	−0.35	0.51	0.12
Factor 4: Acceptability of questioning more senior healthcare professionals’ behaviour (α=0.718)
Q20	It is acceptable for an intern pharmacist to question the actions of a registered pharmacist	0.64	1.00	0.31	0.40
Q21	It is acceptable for a registered pharmacist to question the decisions of a prescriber (such as a doctor or nurse practitioner)	0.77	1.00	0.14	0.60
Factor 5: Attitude towards open disclosure of errors (α=0.534)
Q10	Pharmacists should report errors to an affected patient and their family if harm to the patient has occurred	0.74	1.00	0.22	0.55
Q11	Pharmacists should discuss and report errors to an affected patient and their family even if the patient is NOT harmed	0.53	1.00	0.71	0.28

CFA, confirmatory factor analysis.

Figure 1

Final confirmatory factor analysis (CFA) model with factor loading calculated on combined first and second year data.

Discussion

This study has validated a modified version of an existing patient safety attitudinal survey tool, the Patient Safety/Medical Fallibility Curriculum survey,12 in pharmacy students. A robust two-staged analytical method, involving EFA followed by CFA, was used to assess the reliability and validity of the survey tool. The results of these analyses demonstrated that the attitudes which underpin students’ responses could be explained by five underlying dimensions: (1) being quality improvement focused, (2) internalising errors regardless of harm, (3) value of contextual learning, (4) acceptability of questioning more senior healthcare professionals’ behaviour and (5) attitude towards open disclosure of errors. Four of these dimensions related to patient safety attitudes (factors 1, 2, 4 and 5) and one pertained to the delivery of patient safety interventions (factor 3). This survey tool can therefore be used to help assess the educational needs of students and evaluate patient safety educational interventions.26

The first factor pertained to willingness to undertake quality improvement activities. The EFA on first year students’ responses revealed a relatively low internal consistency reliability for this factor (Cronbach α=0.422). Two items (Q5—Patients have a role to play in their own safety and Q18—After an error occurs, an effective strategy is to work hard to be more careful) had a relatively low loading on the factor and made little contribution to the meaning of the factor. After the removal of these items during the CFA process, there was a significant improvement in the Cronbach α in second year responses (0.654), thereby demonstrating improved internal consistency. This factor examined a positive attitude towards patient safety. Specifically this factor, focused on quality improvement as an indicator of positive safety culture, with higher scores indicating a greater emphasis towards taking a systems approach to dealing with errors, a desired outcome of many patient safety programmes.27 The second factor, however, measured a negative attitude toward patient safety. This factor related to managing and reporting risk, whereby students internalise the error rather than take action, regardless of whether the patient suffered harm. Thus higher scores indicate that students may be less likely to appropriately manage an error. Scores on these factors are important given that there is a push towards teaching incident reporting from junior years to foster good behaviours and to develop a culture of understanding and preventing errors.28 Consequently, as students become more quality improvement focused (as measured by factor 1), it would be expected that they would be more likely to appropriately manage an error rather than internalising the issue (resulting in a corresponding decrease in scores on factor 2).29

The fourth factor measured how acceptable it is to students to question the decisions of more senior healthcare professionals, an important part of managing risk in healthcare. While the two items in this scale are clearly related, these two items do differ significantly. The first item in the factor relates to questioning the decision of a prescriber, while the second item relates to the questioning of an action of a more senior pharmacist. Previous studies have identified that a major obstacle to good patient safety practices among students is the hierarchical structure of healthcare organisations, including community pharmacies where most pharmacy students obtain their first clinical experience.30 31 Being able to work well within teams has been associated with reduced medical errors and improved outcomes in primary healthcare.32–34 In addition to effective communication, being able to deal with conflict, particularly with more senior healthcare practitioners, is also considered an important skill.35 Many patient safety education programmes now include training in managing situations resulting in conflict, and while this factor may not be able to directly examine this skill, by measuring students’ attitudes, it indirectly evaluates whether there is a need for further training in this area.

A core element of all patient safety programmes is the concept of patient-centred care, which includes involving the patient in decisions about their own care and openly disclosing incidents when they occur. Factor five related to open disclosure of errors and hence may be used as a measure of students’ willingness to openly disclose errors to patients, regardless of whether or not harm occurs. Despite being uncommon in practice, open disclosure of errors by healthcare practitioners is desired by patients and required by healthcare authorities.36 Furthermore, it has been shown that it is important for educators to start open disclosure training as early as possible in order to have the greatest impact on changing this behaviour.37

The final factor (factor 3) related to the educational delivery of patient safety interventions. It focused on the pedagogical method that would be most effective in delivering patient safety education to junior pharmacy students26 with items relating to the didactic method of teaching patient safety through university lectures, and learning from experience in the workplace. It is known that the learning preferences of students change throughout their degree, with more meaning-directed approaches preferred as they progress through their degree.38 This factor may therefore be useful to guide the development of teaching materials, tailored to better suit students’ learning style preferences.

Despite the survey being used previously in evaluating patient safety attitudes of both medical and nursing students, only one study has investigated the psychometric properties of the original survey.15 Schnall et al15 utilised 17 of the skill and attitudinal items from the original survey to identify a nine-item, three-factor solution: ‘Error detection, time investment and creating a culture of safety’. Five of the nine items included in Schnall's factor analysis were also included in our final CFA model, however, were placed under different factors in our analysis. Like Schnall et al, the present study observed low reliability scores in our factors during the EFA with first year students. However, when applying the EFA factor structure to our second year students, reliability scores increased, which indicates that students may understand and relate to survey items better the further they have progressed in their degrees as a result of receiving more practice-specific education.

Implications for educators

The use of this survey tool provides a number of benefits for educators. Given that the literature identifies a significant need to provide more training to pharmacy and other healthcare students on all aspects of patient safety, it is crucial that pharmacy schools have a mechanism for evaluating the impact of these programmes.26 39–41 It is noteworthy that current patient safety programmes for pharmacy students often include elements of identifying, understanding, reporting, managing and communicating risk. The underlying attitudes leading to the practice of these positive safety behaviours can all be evaluated using the survey tool. In addition, there are a number of potential benefits which may arise through the repeated use of this tool throughout a student's degree programme. First, it will provide a means to evaluate the longitudinal effect of patient safety education interventions and changes in students’ attitudes. It can also be used to measure the effect of the informal and hidden curricula on students’ patient safety attitudes, which is particularly important as students start experiential learning placements and as more students engage in casual employment in assistance roles. Thus the evaluation of these changes can provide useful information about the educational needs of students through their degrees and when additional and more targeted interventions will need to be provided.

Strengths and limitations of this study

This study has a number of strengths. First, in the absence of a published survey tool to evaluate the patient safety attitudes of pharmacy students, this study modified one of the most highly utilised survey tools to measure the patient safety attitudes of healthcare students12 13 15 in order to suit the requirements of junior pharmacy students. Furthermore, the relatively large sample size obtained (N=446) allowed for a rigorous analytical approach to be undertaken, enabling both exploratory factor analysis and confirmatory factor analysis (CFA) to be performed with sufficient sample sizes for validation of the survey tool. In addition, the high response rates of students completing the survey (87.5% of first year students and 74.7% of second year students) means that the findings are likely to be representative of the attitudes of junior pharmacy students undertaking the Bachelor of Pharmacy programme at the University of Sydney. However, as the sample was drawn exclusively from a single institution, the findings may not be representative of students enrolled in other pharmacy programmes. In addition, despite test–retest reliability not being performed, conducting a CFA on data collected at the same time ensured a form of reliability in the study. Finally, two of the factors (factor 4-questioning behaviours and factor 5-open disclosure) consisted of only two items. While this is considered acceptable,42 it is also a potential limitation that is likely a consequence of the relatively short survey tool utilised. However, the two items that loaded on these two factors adequately described the latent concepts being measured, and are appropriate to the level of understanding and knowledge that junior pharmacy students have of the healthcare system.

Conclusion

This study has demonstrated the validity of a tool to evaluate the attitudes of pharmacy students across a number of patient safety areas. Given that there is growing recognition of the need to educate pharmacy students in patient safety concepts, this survey can be used by pharmacy schools to evaluate the underlying dimensions of students’ patient safety attitudes, which have direct effects on the manner in which students practice. Through the use of this tool, pharmacy schools will be able to further develop and tailor their patient safety training to better suit students’ educational needs.