Self-reported patient safety competence among new graduates in medicine, nursing and pharmacy.

BACKGROUND: As efforts to address patient safety (PS) in health professional (HP) education increase, it is important to understand new HPs' perspectives on their own PS competence at entry to practice. This study examines the self-reported PS competence of newly registered nurses, pharmacists and physicians.
METHODS: A cross-sectional survey of 4496 new graduates in medicine (1779), nursing (2196) and pharmacy (521) using the HP Education in PS Survey (H-PEPSS). The H-PEPSS measures HPs' self-reported PS competence on six socio-cultural dimensions of PS, including culture, teamwork, communication, managing risk, responding to risk and understanding human factors. The H-PEPSS asks about confidence in PS learning in classroom and clinical settings.
RESULTS: All HP groups reported feeling more confident in the dimension of PS learning related to effective communication with patients and other providers. Greater confidence in PS learning was reported for learning experiences in the clinical setting compared with the class setting with one exception-nurses' confidence in learning about working in teams with other HPs deteriorated as they moved from thinking about learning in the classroom setting to thinking about learning in the clinical setting.
CONCLUSIONS: Large-scale efforts are required to more deeply and consistently embed PS learning into HP education. However, efforts to embed PS learning in HP education seem to be hampered by deficiencies that persist in the culture of the clinical training environments in which we educate and acculturate new HPs.

Several key international bodies have made recommendations to restructure how we educate health professionals (HPs) to ensure they are properly equipped with the knowledge, skills and attitudes required to practice safely.1–6 This requires introduction and integration of patient safety (PS) content into HP curricula and training programmes, with a particular emphasis on the socio-cultural facets of PS.1 Restructuring HP education is challenging7 and a growing literature suggests it is occurring very slowly across the health professions.4 5 8–12

As efforts to include PS in HP education increase it is important to capture trainees’ and new HPs’ perspectives of their own PS knowledge and competence.1 There are a number of published studies that have examined trainees’ PS knowledge, skills and attitudes. Most of these studies have reported on assessments of the impact of specific PS curricular or training interventions.13–26 However, these studies have been largely confined to a single educational institution, and have been limited to a single HP group.

A much smaller set of studies have examined PS knowledge, skills and attitudes of more than one HP group.27–29 These studies have focused on areas such as methods of effectiveness for delivery of inter-professional education,29 learning uni-professionally or multi-professionally,28 or they have used qualitative approaches.30 31 Additionally, we were only able to find three studies that examined PS learning in the classroom and clinical settings. One was a small pilot study,23 one focused on the formal, informal and hidden aspects of curricula,32 and all three were limited to a single HP group.23 24 32 This paper reports on data concerning the safety educational experience gathered from newly licensed/registered nurses, pharmacists and physicians from across Canada's most populous province, Ontario.

Methods

We conducted a large, cross-sectional survey of new graduates in medicine, nursing and pharmacy using the HP Education in PS Survey (H-PEPSS).33 The H-PEPSS is a validated tool that measures HPs’ self-reported PS competence. The study was conducted in the fall of 2010 in the Canadian province of Ontario, which has six medical schools, 15 nursing schools and two training programmes for pharmacists, all of which are housed in publicly funded universities. At the time of data collection one of the pharmacy programmes was new and did not yet have graduates.

Sampling and procedures

The inclusion criteria for the sample were nurses, medical doctors and pharmacists who received registration/licensure with their relevant regulatory college in Ontario, Canada in 2010 (this was expanded to 2009 or 2010 for the pharmacy and postgraduate medicine groups to yield a sufficient sample for these two groups); completed their degree in a Canadian university during the preceding 1 year period; and provided their college with an email address. In the case of the physicians there were two groups: those who had just finished medical school (the MD group); and those who had just completed a residency (the postgraduate medicine group). The resulting sample consisted of 814 in the MD group, 965 in the postgraduate medicine group, 2196 registered nurses and 521 pharmacists. All 4496 members of the sample were invited to participate in this study.

The relevant professional college (the College of Physicians and Surgeons of Ontario, College of Nurses of Ontario, the Ontario College of Pharmacists) sent an email invitation to take part in the survey along with a link to the survey webpage. Following recommendations for electronic surveys34 two reminder invitations were sent out 1 and 2 weeks after the initial invitation. Survey invitations were drafted by the research team and were consistent for all three professional groups. To allow for anonymous survey completion, no trackable links were used. We were aware of the historically poor survey response rates from practicing HPs and in an attempt to ameliorate this we offered two draws, each one for an iPad. However, respondents had to provide their email address if they wished to be entered into one of these draws. Completed survey data were retrievable only by one of the study investigators (LG) who is not affiliated with any of the regulatory colleges. The study received approval from the Human Participants Review Committee in the Office of Research Ethics at the lead author's home institution (York University).

Survey instrument

Study data were gathered using the H-PEPSS.33 The H-PEPSS measures HPs’ self-reported PS competence and was designed to reflect six socio-cultural areas fundamental to PS and central to a number of key PS competency frameworks developed by various international professional bodies and WHO.5 35 36 A systematic review37 published just prior to publication of the H-PEPSS found no existing surveys that measured the breadth of content reflected in these safety competencies.

The psychometric properties of the H-PEPSS are described elsewhere33 and support a 16-item measure of self-reported PS competence that reflects six dimensions: Working in teams with other HPs (three items), Communicating effectively (three items), Managing safety risks (three items), Understanding human and environmental factors (two items), Recognising and responding to adverse events (two items), and Culture of safety (three items). For all six dimensions, αs range from 0.81 to 0.85. Items begin with the stem ‘I feel confident in what I learned about…’ and are answered using a five-point disagree–agree Likert-type scale and include a ‘don't know’ option. For each item, respondents are asked to respond separately about their confidence in what they learned in the classroom setting versus the clinical setting (given the structure of postgraduate medical training, this group is only asked about the clinical setting). Survey items in each dimension are shown in online appendix 1. Mean dimension scores are calculated from the items in each dimension for each learning setting (eg, you will have a score for confidence in learning around Communicating effectively based on education provided in the classroom and a separate score for confidence in learning around Communicating effectively based on education provided in the clinical setting). The H-PEPSS begins by asking about confidence in knowledge of four more clinical aspects of safety (eg, hand hygiene and infection control). These items are included in the H-PEPSS solely to help respondents distinguish between clinical and socio-cultural aspects of PS so they can focus on the latter.

Analysis

Study analyses were carried out using basic descriptive and inferential statistics: H-PEPSS dimension means were compared; one-way analysis of variance (ANOVA) was used to compare H-PEPSS dimension scores across the different HP groups in the study; and paired t tests were used to examine differences in confidence in PS learning in the class setting compared with learning in the clinical setting for each H-PEPSS dimension. All analyses were carried out using SPSS V.19.

Results

Respondents

Of the 4496 new graduates invited to complete the H-PEPSS, 180 had undeliverable email addresses or were terminated at the survey website for failing to meet the study eligibility criteria outlined above. A total of 1247 out of 4316 eligible new graduates submitted an online survey resulting in a response rate of 28.9% overall, with slight variation by respondent group (MD group=35%, postgraduate medicine 25%, pharmacy 29% and nursing 28%). Summary data provided by the relevant colleges indicate that the respondent group is representative of the target population in terms of age (mean age=27.5), gender (55% women for both medicine groups, 70% for pharmacy and 91% for nursing) and training institution, with one exception—those who trained in a Canadian jurisdiction outside of Ontario are slightly underrepresented in the respondent group. Data from 1102 responders who responded to more than 50% of the items in each H-PEPSS dimension are included in the analyses reported here.

Comparing H-PEPSS dimensions

To compare self-reported PS competence across the H-PEPSS dimensions, dimension means are compared and reported in table 1. Column B shows dimension means (as measured on a five-point scale ranging from 1 (‘strongly disagree’) to 5 (‘strongly agree’)) for self-reported confidence in learning in the classroom and clinical setting for all HP groups combined. Based on non-overlapping 95% CIs (not shown), respondents’ self-reported PS competence in the Communicating effectively dimension is significantly higher than self-reported PS competence in the Working in teams with other HPs dimension, the Managing safety risks dimension and the Understanding human and environmental factors that influence safety dimension. This pattern holds true for all HP groups and for confidence in learning in the classroom and clinical settings with one exception (for learning in the clinical setting pharmacists are equally confident in the learning around Effective communication and Understanding human and environmental factors that influence safety).

Table 1

Items in the managing safety risks dimension

	A	B	C	D	E	F
Dimension	Learning setting	All HP groups N=1102	Nursing N=520	Pharmacy N=120	MD N=236	Postgrad. med. N=226
Working in teams with other HPs	Clinical	3.72	3.62*	3.67	3.84	3.86
	Class	3.74	3.89**	3.33**	3.61	n/a
Communicating effectively	Clinical	4.15	4.19	4.08	4.09	4.18
	Class	4.12	4.26**	4.04	3.86	n/a
Managing safety risks	Clinical	3.82	3.95*	3.80	3.62	3.72
	Class	3.55	3.71**	3.40	3.27	n/a
Understanding human and environmental factors that influence PS	Clinical	3.84	4.04*	3.92	3.59	3.61
	Class	3.75	4.00**	3.32	3.42	n/a
Recognise and respond to remove immediate risks of harm	Clinical	4.02	4.14*	3.98	3.89	3.92
	Class	3.79	3.92**	3.71	3.54	n/a
Culture of safety (feel safe to speak up/understand system nature of PS problems)	Clinical	3.84	3.95*	3.85	3.69	3.73
	Class	3.91	4.07**	3.59	3.70	n/a

*Significant within learning setting compared with both medicine groups.

**Significant within learning setting compared with other HP groups.

HP, health professional; PS, patient safety.

Differences between HP groups on the H-PEPSS

One-way ANOVA was used to test for differences in self-reported PS competence across the different HP groups in the study. For learning in the classroom setting (comparing mean scores in table 1), nurses scored significantly higher than the pharmacy and MD groups on all dimensions F(df=2, 870) F ranges from 17.45 to 49.09, df=(2, 870), p=0.000). For learning in the clinical setting, nurses’ self-reported PS competence around Working in teams with other HPs is significantly lower than the MD and postgraduate medicine groups (F(df=3, 1091) F=7.008, df=(3, 1091), p=0.000); however, nurses’ scores are significantly higher than both physician groups on all other dimensions (F(3, 1091) F ranges from 7.03 to 17.64, df=(3, 1091), p=0.000) except Communicating effectively, for which there are no significant differences between any of the groups. The teamwork dimension is explored in more detail below.

Differences in self-reported PS competence in different learning settings

Paired t tests were used to compare confidence in PS learning in the class versus clinical setting. The trend in the data for the pharmacy and MD groups is that confidence in PS learning was significantly higher for the clinical setting compared with the classroom setting (p<0.01; see table 1, columns D and E) with a few exceptions shown in bold text: pharmacists’ scores based on the classroom and clinical setting learning did not differ on the Communicating effectively dimension, and MDs’ self-reported confidence in PS learning regarding a Culture of safety did not differ from the class to clinical setting. Nurses were the only group to report significantly lower levels of confidence in learning in the clinical setting than the classroom setting and this was true for the dimensions Working in teams with other HPs, Communicating effectively and Culture of safety (italic text in table 1) (t=−6.2 to −2.4, df=515, p=0.000 ti 0.017). Nurses’ self-reported confidence in PS learning on the Understanding human factors dimension did not differ for the classroom and clinical setting.

The trend of different levels of confidence for learning in the clinical setting compared with learning in the classroom setting was explored in greater detail by looking at specific items.20 For the Managing safety risks dimension table 2 shows that the clinical setting equips all groups with far more confidence in learning on each of these items. In addition, a comparison of the MD and postgraduate medicine scores suggests increasing confidence with a longer period in the clinical setting. This result is consistent with reported data for this dimension gathered from nursing students in different years of an undergraduate nursing programme.38

Table 2

Items in the Managing safety risks dimension

		% agree/strongly agree
‘I feel confident in what I learned about…’	Learning setting	Nursing	Pharmacy	MD	Postgraduate medicine
Q14. Recognising routine situations in which safety problems may arise	Class	73.4	61.0	53.0	76.4
	Clinical	83.5	82.5	69.9
Q15. Identifying and implementing safety solutions	Class	67.2	48.7	43.3	59.1
	Clinical	73.1	72.0	54.5
Q16. Anticipating and managing high-risk situations	Class	56.9	45.8	41.0	63.6
	Clinical	72.0	63.0	57.4

A focus on individual items is also useful for more closely examining the dimension Working in teams with other HPs. Table 1 indicates that the Working in teams with other HPs dimension scores lower than many or all dimensions across all four groups. In addition, the only instance when nurses’ level of confidence in learning is significantly lower than any of the other HP groups we examined is for learning in the clinical setting related to Working in teams with other HPs. Furthermore, nurses’ confidence in learning related to Working in teams with other HPs is not only significantly lower for learning in the clinical setting than for learning in the classroom setting but the magnitude of this difference is important (Cohen's d=0.33, which is a small to medium effect size).39 Figure 1 shows the percentage of respondents in each HP group who agree with each of the three items in this dimension. On items 9 and 10, 12% and 20% fewer nurses were confident following clinical learning than classroom learning. In contrast, pharmacists and postgraduates expressed far greater confidence in learning on all three items as they move from thinking about classroom learning to learning in the clinical setting. Online appendix 1 shows all of the items in each dimension and the percentage of respondents in each HP group who agreed or strongly agreed with each question.

Figure 1

Items in the Working in teams with other health professionals dimension. PGY1, postgraduate year 1.

Discussion

This study examined self-reported PS competence among newly registered/licensed nurses, pharmacists and physicians. In this study self-reported PS competence is operationalised as level of confidence in learning about six socio-cultural dimensions of PS. Here we discuss three key findings. First, the overall levels of confidence in PS learning we found are consistent with other research with physicians, indicating that more training in how to handle errors is required.40 In addition, our results show lower levels of confidence in PS learning around the dimensions of Managing safety risks and Understanding human and environmental factors that influence PS—a finding that underscores the need to introduce concepts from ‘safety science’ into HP education.41

Second, our results show nurses scored significantly higher than the pharmacy and MD groups on all H-PEPSS dimensions for learning in the classroom setting, and on most dimensions for learning in the clinical setting. Different studies that have asked different HP trainees the same question (about the extent to which ‘the culture of medicine makes it easy for providers to deal constructively with errors’) also show far more positive scores for nurse trainees25 26 compared with physician trainees.17 Other work27 has also found medical students’ self-reported baseline PS knowledge scores were significantly lower than those of nurses and respiratory therapists. While interesting, these kinds of absolute differences in levels of confidence in PS learning across different HP groups may reflect a number of factors, including differences in professional beliefs entrenched within the culture of learning42 43 or HP group differences in levels of ‘not knowing what one doesn't know’. Accordingly, this is an area in need of further study.

Third, our results showing differences in PS learning confidence across learning settings have interesting implications. The trend in our data was that the pharmacy and physician groups reported significantly higher confidence in PS learning following experience in the clinical setting compared with the classroom setting. This is consistent with work showing that opportunities for learners to practically apply PS material leads to greater PS confidence.16 However, our findings for nurses did not adhere to this pattern on the H-PEPSS dimensions Working in teams with other HPs, Communicating effectively, and Culture of safety. Instead, nurses’ confidence in learning on these dimensions worsened (while physician's confidence improved) as they moved from thinking about learning in the classroom setting to learning in the real world of the clinical setting. These findings suggest that, for nurses, there is an important contextual influence of training in the clinical setting that weakens their confidence in some of the key socio-cultural aspects of PS. These results fit with recent work which showed declining confidence in PS teamwork learning among nurses in later years of an undergraduate nursing programme38 and with work in intensive care units,44 operating rooms45 46 and general medicine wards,47 showing that nurses’ perceptions of collaboration and communication with physicians are less positive than physicians’ perceptions.

Indeed, the hierarchical nature of healthcare, power imbalances between physicians and nurses48 reflected in differing perceptions of and responses to conflict between these two HP groups,49 and aspects of the overall culture (recently described as disrespectful and demeaning in its treatment of nurses)50 may be important contributing factors to nurses’ deteriorating confidence in learning about Working in teams, Communicating effectively and the Culture of safety in the clinical setting. Similarly, the role and influence of the hidden curriculum may also help to explain why our nursing group had less confidence in their teamwork and communication skills after training in the clinical setting. The hidden curriculum, which is a set of influences or unintended messages that function at an institutional and cultural level,51 is well understood to be a potential source of the disconnect between what is learned in the classroom versus what is experienced in the clinical setting. Moreover, the literature is replete with instances of within-profession faculty-learner incivility in nursing52 and in medicine.53 54

The generally harmful nature of the hierarchical culture that typifies the clinical settings in which we train HPs is also well documented50 55–57 and may explain the lower levels of confidence in teamwork and communication learning among nurses as they moved into the clinical setting. Ultimately, HP trainees are highly influenced by their faculty preceptors’ attitudes towards PS and the PS culture of the teaching organisations in which they learn.20 58 Accordingly, it is crucial that the informal curriculum (eg, the ad hoc, personal form of teaching) and the hidden curriculum are well understood to try to exploit all educational venues and positively influence HP training programmes.32 59

Inter-professional education may be useful for responding to lower levels of confidence in teamwork and communication learning that nurses experience as they move into the clinical setting. Inter-professional training opportunities at different stages of the educational process can address harmful effects of hierarchical power dynamics that exist between nurses and physicians28 60 61 and can generate positive attitudes toward inter-professional collaboration early in training.62 While inter-professional learning opportunities remain limited,63 there is some recent empirical work that suggests trainees value learning in inter-professional environments,64 that it can improve teamwork knowledge and attitudes,29 62 and that it can resolve differences in PS viewpoints held by different groups of HP trainees.27 While some suggest that inter-professional education delivered using case studies and simulation are more likely to be successful than didactic lectures alone,41 65 more recent work found that costlier modalities of delivery were not necessarily found to be superior.29

Finally, as we consider approaches to more deeply and consistently embed learning about the socio-cultural aspects of PS it will be important to recognise that the clinical setting is complex and there are various factors that influence what is being learned and how.66 In addition, learning opportunities in the clinical setting rub up against the reality of competing educational demands9 and overburdened clinicians who have limited time to devote to teaching and learning.25 The need for greater numbers of clinical faculty who are familiar with quality improvement and PS content,1 4 9 67 68 and closer links between academic staff in universities and managers in healthcare settings around PS have been suggested and may help to achieve an appropriate balance of learning in these two settings.31 69

This study has some limitations. First, the self-reported nature of the data mean that some degree of social desirability bias is likely and the absolute levels of confidence in PS learning that are reported, when they are high, may reflect that respondents are unaware of what they do not know. Similarly, trainees may overestimate or underestimate their PS competence. For these reasons we focus more on the data patterns than the absolute levels of self-reported PS competence. Second, while response rates in the 30% range are typical in these kinds of surveys, the nature of any potential non-response bias remains unclear. Further research, perhaps qualitative, is required to better understand why nurses’ confidence in learning around inter-professional teamwork weakens when they learn in the clinical setting.

Conclusions

As efforts to enhance PS in HP education continue, it is important to recognise that PS in HP education still largely reflects clinical safety (eg, attention to infection control and medication safety31). This should not be surprising given that most HP curricula that include PS devote at most a few hours to the topic.9 70 Large-scale efforts are required to more deeply and consistently embed PS learning into HP education. However, such efforts require attention to a number of factors at the level of the learner, the faculty and the clinical learning setting. In particular, this study reveals that efforts to embed PS learning in HP education may be hampered by fairly profound deficiencies that persist in the culture of the clinical training environments in which we educate and, paradoxically, acculturate our new HPs.