Self-reported patient safety competence among Canadian medical students and postgraduate trainees: a cross-sectional survey.

BACKGROUND: Quality and patient safety (PS) are critical components of medical education. This study reports on the self-reported PS competence of medical students and postgraduate trainees.
METHODS: The Health Professional Education in Patient Safety Survey was administered to medical students and postgraduate trainees in January 2012. PS dimension scores were compared across learning settings (classroom and clinical) and year in programme.
RESULTS: Sixty-three percent (255/406) of medical students and 32% (141/436) of postgraduate trainees responded. In general, both groups were most confident in their learning of clinical safety skills (eg, hand hygiene) and least confident in learning about sociocultural aspects of safety (eg, understanding human factors). Medical students' confidence in most aspects of safety improved with years of training. For some of the more intangible dimensions (teamwork and culture), medical students in their final year had lower scores than students in earlier years. Thirty-eight percent of medical students felt they could approach someone engaging in unsafe practice, and the majority of medical students (85%) and postgraduate trainees (78%) agreed it was difficult to question authority.
CONCLUSIONS: Our results suggest the need to improve the overall content, structure and integration of PS concepts in both classroom and clinical learning environments. Decreased confidence in sociocultural aspects of PS among medical students in the final year of training may indicate that culture in clinical settings negatively affects students' perceived PS competence. Alternatively, as medical students spend more time in the clinical setting, they may develop a clearer sense of what they do not know. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Introduction

Patient safety is central and critical in providing quality healthcare, and the need for urgent patient safety reform is recognised on a global level.1–5 Effectively integrating patient safety science into the training programmes of healthcare professionals is essential for advancing this initiative. A need for greater emphasis on the sociocultural aspects of patient safety has been established,6 and recent literature suggests that we are slow to incorporate this into our curricula.5 7 8 The sociocultural perspective recognises that the interaction between knowledge and individual learning is dependent upon the cultural conditions under which learning takes place.9

The perspectives of medical students and postgraduate trainees (collectively referred to as learners) on patient safety curricula in the classroom and clinical setting is one necessary component for understanding the success with which patient safety concepts are integrated and actualised in medical education. How learners’ perspectives on patient safety learning evolve at various stages of training is also useful for reforming and developing patient safety education. While there is a growing literature showing that learners are not necessarily accurate at self-assessing performance when compared with external assessments, social cognitive theory suggests how self-assessment is important. Self-efficacy (an individual's perceived ability to succeed at a task), including academic self-efficacy,10 has been shown to influence behaviour and task performance.11

A recent study by Ginsburg et al measured healthcare professionals’ perceptions of patient safety competence at entry to practice (operationalised as confidence in patient safety learning).12 However, there have been few other studies that examine the learner's perspective on patient safety education and culture.13–16 The purpose of this study was to report on medical students’ and postgraduate trainees’ self-reported patient safety competence during their education programmes. The objectives were: (1) to describe and compare self-reported patient safety competence acquired in the classroom and clinical settings and (2) to compare self-reported patient safety competence across the programme years. The a priori hypotheses were: (1) medical students’ and postgraduate trainees’ self-reported patient safety competence will increase as they are increasingly exposed to the clinical setting and (2) medical students’ and postgraduate trainees’ self-reported patient safety competence will increase as they progress through their training.

Methods

Participants and setting

This cross-sectional study used a web-based survey with undergraduate medical students and postgraduate trainees enrolled at one Canadian university. All 406 students in the undergraduate medical programme and all 436 postgraduate trainees were eligible to participate. The undergraduate medical curriculum is a 4-year programme with a combination of classroom courses and clinical placements in all 4 years. The classroom setting includes lecture-style classes consisting of approximately 100 students and smaller breakout groups of 8–10 students. Clinical placements occur in a variety of settings, including acute, long-term and community care. In these settings, students are generally supervised by upper-year postgraduate trainees and board-certified physicians (eg, consultants). In years 3 and 4, students spend more time in clinical placements, however, they continue to spend several hours per week in the classroom setting. The postgraduate programme consists of several specialties that are usually 4 years in duration, and training primarily takes place in the clinical setting with approximately one half day per week in the classroom setting. The study was reviewed for ethical compliance by the Queen's University's Health Sciences and Affiliated Teaching Hospitals Research Ethics Board (REB #ANAE-194-11).

Measurement tool

In January 2012, medical students and postgraduate trainees were invited to complete a modified version of the Health Professional Education in Patient Safety Survey (H-PEPSS).12 This validated questionnaire was originally developed at York University, Canada, and was designed to provide new health professionals’ perspectives on their exposure to, and confidence in, learning about six sociocultural patient safety competencies. The six patient safety competencies were based on the Canadian Patient Safety Institute's Safety Competencies Framework,17 18 a tool developed to encompass the knowledge, skills and attitudes required by all healthcare professionals in order to be considered competent in patient safety. After validation, the original H-PEPSS was reduced from 23 to 16 items12 (box). An additional clinical skills dimension capturing confidence in learning about four clinical skills—hand hygiene, infection control, safe medication practices and safe clinical practice in general—was also included on the H-PEPSS. Respondents were asked to answer each question separately for the classroom and the clinical settings. The questionnaire stem was modified from the original questionnaire to reflect the present rather than the past tense. It reads, ‘I feel confident in what I am learning about…’, which would capture the perspective of current students rather than recent graduates. The questionnaire contained items on the six dimensions of patient safety (16 items, box), clinical skills (4 items), broader aspects of patient safety (7 items) and comfort when speaking up (4 items). All items were scored on a Likert scale of 1 (strongly disagree) to 5 (strongly agree). Additional questions asked about age, gender, year of study and programme of specialisation (for postgraduate students only). It took approximately 12 min to complete. The questionnaire was converted to a web-based format using the ‘Student Voice’ platform (http://www.studentvoice.com).

Contribute to a culture of patient safety (3 items).

Work in teams for patient safety (3 items).

Communicate effectively for patient safety (3 items).

Manage safety risks (3 items).

Optimise human and environmental factors (2 items).

Recognise, respond to and disclose adverse events and close calls (2 items).

Recruitment

Letters encouraging participation in the study were sent via email from the deans of the undergraduate and postgraduate medical education programmes. This was followed-up by an email invitation from a central contact in the respective offices containing information about the study and a link to the questionnaire. Two follow-up email reminders were sent; the first at 1 week after the initial email and a second one 1 week prior to the end of the 2-month period that the questionnaire was available for completion (January and March 2012). Participation in the survey was voluntary, and consent was implied by completion. All responses were anonymous. Participants were instructed on how to create a unique identification code in order to preserve anonymity and yet allow for linking responses if the questionnaire is replicated in future years. As an incentive for completing the questionnaire, participants were entered in a draw for one of two iPads if they were willing to provide their name and email address. The contact information was stored in a separate file from the questionnaire responses to ensure anonymity.

Statistical analysis

The primary analysis focused on responses provided by medical students. Analysis of postgraduate trainee data focused on self-reported patient safety competencies in the clinical setting. Demographic data were examined using univariate statistics. Based on the approach used in the initial validated tool, a mean score (SD) for each of the patient safety dimensions was calculated by averaging the items in each dimension. More students completed questions relative to the classroom setting than to the clinical setting. Therefore, to limit a reduction in sample size, the more conservative unpaired t test was used to assess for statistically significant differences between classroom and clinical scores. One-way analysis of variance was used to compare differences across programme years. The Bonferroni correction was used to test pairwise comparisons between the years (Bonferroni corrected p<0.007). Cohen's effect size was calculated for statistically significant pairwise comparisons. Like the initial validated tool, dimension scores (range 1–5) were also categorised into agree/strongly agree (>3.5) and neutral/disagree (≤3.5), and the χ2 test was used to assess differences between the classroom and the clinical setting. Statistical significance was set at p<0.05. All data were analysed using SPSS V.22. Given that the question stem we used was slightly different from the original survey instrument and that it was not validated with students, we revalidated the H-PEPSS factors structure using confirmatory factor analysis. Good model fit was achieved in both the medical student and postgraduate trainee samples using commonly accepted indices of fit (Confirmatory Factor Index >0.95, Root Mean Square Error of Approximation <0.06).19

Results

Sixty-three percent (255/406) of medical students and 32% (141/436) of postgraduate trainees responded to the survey. Due to the low response rate among postgraduate trainees, the results will focus on medical students, with a brief description of the postgraduate results. Demographic characteristics of the study sample and select characteristics of the total population are provided in table 1. The distribution of respondents across year in the programme, and programme of specialisation (postgraduate trainees only) was not significantly different between the sample and target population.

Table 1

Demographic characteristics of medical student and postgraduate participants

	Medical student sample (n=255)		Postgraduate trainee sample (n=141)
Age (mean (SD))	24.7 (2.8)		30.6 (4.3)
Gender (n (%))
Female	132 (51.8)		76 (53.9)
Male	123 (48.2)		65 (46.1)

*χ2=0.65, p=0.88.

†χ2=0.54, p=0.97.

‡χ2=7.32, p=0.50.

§Includes cardiology, critical care, radiology, gastroenterology, haematology, nephrology, neurology, obstetrics and gynaecology, palliative care, paediatrics, physical and rehabilitation, psychiatry, public health and preventative medicine, radiation oncology, respirology, rheumatology, urology.

¶Includes six missing programme information.

n/a, not available.

Patient safety dimensions

Medical students were most confident in what they were learning about clinical safety skills and communicating effectively (table 2). They were least confident in what they were learning about managing safety risks, understanding human and environmental factors that contribute to safety, responding to remove immediate risks of harm and culture of safety. While there were statistically significant differences in mean patient safety dimension scores between the classroom and clinical setting, in most cases, the effect size was very small and, therefore, of low clinical significance. In terms of the proportion of respondents who were confident about what they were learning, close to three-quarters of respondents ‘agreed’ they were confident in what they were learning about clinical safety and communicating effectively for patient safety; however, approximately half or fewer medical students agreed they were confident in what they were learning about the other five sociocultural dimensions of patient safety (table 2).

Table 2

Classroom and clinical self-reported patient safety dimension scores for medical students (reduced H-PEPSS model*)

			Score (1–5)1 (strongly disagree) to 5 (strongly agree)				Agree/strongly agree
Patient safety dimensions	Setting	n	Mean	SD	Effect size	Unpaired t testp Value	n	Per cent
Clinical safety skills	Class	255	3.9	0.7	0.03	0.69	182	71
	Clinical	235	3.9	0.7			168	72
Work in teams with other Health professionals	Class	257	3.6	0.8	0.13	0.67	146	55
	Clinical	233	3.7	0.8			135	50
Communicating effectively	Class	252	3.8	0.9	0.11	0.21	173	69
	Clinical	234	3.9	0.7			177	76
Managing safety risks	Class	247	3.2	0.9	0.26	<0.01	100	41
	Clinical	226	3.4	0.9			116	51
Understanding human and environmental factors	Class	243	3.1	1.0	0.20	0.04	111	41
	Clinical	222	3.3	1.0			122	46
Recognise and respond to remove immediate risks	Class	242	3.4	0.9	0.12	0.11	133	50
	Clinical	214	3.5	0.8			135	50
Culture of safety	Class	244	3.5	0.9	0.00	0.58	139	52
	Clinical	216	3.5	0.9			122	46

*Reduced H-PEPSS contains 16 items measuring the six patient safety domains.

H-PEPSS, Health Professional Education in Patient Safety Survey.

Statistically significant differences in mean patient safety dimension scores by year of the programme are reported in online supplementary appendix 1. There were significant differences across years with medium to large effect sizes in the classroom setting for clinical safety skills, working in teams and culture of safety. For learning in the clinical setting, statistically significant differences by year were found in clinical safety skills, managing safety risks, and human and environmental factors. First-year students had significantly lower scores for the dimension of clinical safety in the classroom setting compared with all other years, and lower scores than fourth-year students in the clinical setting. Students in the second year were significantly less confident about what they were learning about managing safety risk in the clinical setting and understanding human and environmental factors in the clinical setting compared with fourth-year students. In other areas, confidence in learning scores were lower in later years of the programme: first-year students reported more confidence about working in teams in the classroom setting compared with years 2–4, and fourth-year students reported significantly less confidence in what they were learning about the culture of safety in the classroom setting compared with first-year students.

Broader aspects of safety and comfort when speaking up

Questions on broader aspects of safety and comfort when speaking up about patient safety are presented in table 3. Most medical students agreed ‘Clinical aspects of patient safety (eg, hand hygiene, transferring patients, medication safety) are well covered in our programme’ (63% agree/strongly agree). In a number of areas pertaining to broader aspects of patient safety in health professional education, medical student agreement levels, that is, agree or strongly agree, were below 50%: scope of practice is very clear (48%), there is consistency in how patient safety issues are dealt with by different preceptors (35%), understanding that reporting can lead to change and improvement (44%), patient safety is well integrated in overall training (46%) and ‘system’ aspects were well covered in the programme (eg, aspects of the organisation, management, or the work environment including policies, resources, communication and other processes) (24%). A comparison of responses from medical students in different years of the programme (not shown) shows that significantly more first-year students than students in years 2–4 agreed that system aspects were well covered in their programme (1st year Mean (M)=3.4, 2nd=2.6, 3rd=2.6, 4th=2.7, Statistic (F)=9.3, p<0.01). First-year and second-year medical students also reported significantly more consistency in how patient safety issues are dealt with by different preceptors compared with students in years 3 and 4 (1st year M=3.4, 2nd=3.2, 3rd=2.9, 4th=2.7, F=6.1, p<0.01). Items with significant differences across the year in programme are shown with an asterisk in table 3.

Table 3

Broader aspects of patient safety and comfort in speaking up about patient safety for medical students

Medical student (n=255)	Mean (SD)1 (strongly disagree) to 5 (strongly agree)	Agree/strongly agree n (%)
Broader aspects of patient safety
As a student, my scope of practice is very clear to me	3.2 (1.0)	123 (48.4)
There is consistency in how patient safety issues are dealt with by different preceptors in the clinical/simulation setting	3.0 (1.0)*	89 (35.2)
I have sufficient opportunity to learn and interact with members of interdisciplinary teams	3.4 (1.0)*	143 (56.5)
I am gaining a solid understanding that reporting adverse events and close calls can lead to change and can reduce recurrence of events	3.2 (1.1)	111 (44.0)
Patient safety is well integrated into the overall programme	3.3 (1.0)	116 (45.8)
Clinical aspects of patient safety (eg, hand hygiene, transferring patients, medication safety) are well covered in our programme	3.7 (1.0)	158 (62.7)
‘System’ aspects of patient safety are well covered in our programme (eg, aspects of the organisation, management, or the work environment including policies, resources, communication and other processes)	2.8 (1.0)*	62 (24.4)
Comfort in speaking up about patient safety
If I see someone engaging in unsafe care practice in the clinical setting, I feel I can approach them	3.0 (1.0)*	96 (37.8)
If I make a serious error, I worry that I will face disciplinary action	3.6 (1.0)	169 (66.8)
It is difficult to question the decisions or actions of those with more authority	4.1 (0.8)	215 (85.0)

*Significant differences (p<0.05) across the year in the programme.

Table 3 also presents data on comfort when speaking up about patient safety issues. Only 38% of medical students felt they could approach someone engaging in unsafe practice. Similarly, 85% of medical students agree it is difficult to question the decisions or actions of those with more authority and the majority (67%) worry they will face disciplinary action if they make a serious error. Medical students in year 3 reported the lowest level of comfort in approaching someone they see engaging in unsafe care practice in the clinical setting and they were significantly less comfortable than students in year 1 who reported the highest level of comfort (1st year M=3.3, 2nd=3.1, 3rd=2.8, 4th=3.1, F=3.1, p=0.03—results not shown).

Postgraduate trainees

In spite of the low response rate for postgraduate trainees, the demographic characteristics of the study sample (n=141) were reflective of the population frame, with the majority of respondents enrolled in family medicine (36%). The majority of postgraduate trainees reported confidence in the patient safety competencies gained in the clinical setting—clinical safety skills (84%), communicating effectively (81%), recognising and responding to adverse events (75%), working in teams (74%), managing safety risks (67%), culture of safety (63%) and understanding human and environmental factors (61%). Seventy-eight percent reported having sufficient opportunity to learn and interact with members of interdisciplinary teams. However, like their undergraduate peers, only 39% agreed there was consistency in how patient safety issues were dealt with by different preceptors. Forty-seven percent agreed ‘system’ aspects were well covered in the programme. Seventy-eight percent agreed it was difficult to question the decisions or actions of those with more authority and 59% worried they will face disciplinary action if they make a serious error (detailed data available upon request).

Discussion

To our knowledge, this is the first study to examine self-reported patient safety competence in medical education from the perspective of medical students and only the second study to explore this issue in postgraduate trainees.20 Generally, medical students were most comfortable with aspects of clinical safety, such as hand hygiene, infection control and safe medication practices. They were less confident in what they learned about sociocultural or contextually dependent aspects of patient safety, such as teamwork, managing safety risk and culture of safety. Confidence in most aspects of patient safety competence improved with years of training, with a few exceptions. Upper-year students were less confident in their learning around competencies related to working in teams and the culture of safety, compared with lower-year students. The majority of medical students (85%) and postgraduate trainees (78%) reported difficulty questioning the decisions or actions of those with more authority and approximately two-thirds of medical students and one-third of postgraduate trainees did not feel they could approach someone engaging in unsafe patient care.

The findings that medical students reported higher patient safety competence about aspects of clinical safety than about sociocultural aspects of safety are consistent with reports in nursing students15 and recent graduates.16 Additionally, the focus on public and institutional campaigns on hand hygiene and medication safety21–23 may influence confidence in these areas. The findings are also consistent with other recent work showing that when asked about aspects of patient safety that they teach, clinical faculty overwhelmingly identify clinical safety to the exclusion of sociocultural aspects of patient safety.24

Our finding that self-reported patient safety competence improved with years of training is also consistent with reports in the literature.15 However, learning on some of the more intangible dimensions (teamwork and culture) was lower for upper-year medical students. Upper-year students spend the majority of their time in the clinical setting where confidence in teamwork and cultural aspects of safety are more strongly influenced by the patient safety culture in those clinical settings. The key implication is that the culture in the clinical settings where training takes place is important and, at present, may be negatively affecting students’ self-reported patient safety competence.16 25 It is also conceivable that this trend in the data reflects that, as they spend more time in the clinical setting, students gain greater awareness about what they do not know.

Our finding about the uncertainty and apprehension around reporting of errors is a consistent theme in patient safety research, and more training is required in this area.16 In a prospective study involving 92 second-year medical students in the USA, the majority reported they were more likely to disclose errors to peers, followed by reporting to postgraduate trainees, and least likely to report to faculty or to use an electronic reporting tool.13 Comfort in speaking up about patient safety is most persistently entrenched in culture and attitudes in the clinical setting, as it requires that errors be thought of as the interplay of system issues rather than the result of individual incompetence.14 26 ‘System’ aspects of patient safety were considered not well covered according to both medical students and postgraduate trainees. This is consistent with a recent systematic review where only 39% of studies (n=16/41) reported that ‘systems thinking’ was part of the education curriculum for medical students and postgraduate trainees.27 There is also a need to improve the overall integration and implementation of patient safety concepts in the classroom and clinical setting, as well as the integration between what is being taught in both settings. This might help to address the inconsistencies in how patient safety issues are dealt with by different preceptors, as reported by both medical students and postgraduate trainees. The importance of preceptor or teacher factors has been well documented in the literature. Important factors include adequate number of faculty and involvement of role models with expertise in patient safety, as well as buy-in and adequate time to teach the curriculum.27

Corporate culture, especially support to openly address errors without assigning blame, also impacts on the confidence of learners to speak up.28 29 A dearth of knowledge and expertise on safety science and/or hierarchical leadership models in the clinical setting, as well as the resources required to conduct exhaustive analysis of errors, may all contribute to the one-dimensional focus on individual responsibility. Whether the effort required to capture, report and analyse errors is the best method to contribute to safer care has also been called into question.30 However, our data do suggest that appreciation of system factors improves with years of training. For instance, postgraduate trainees were almost twice as likely as medical students to feel that system aspects of patient safety were well covered in their programme. Similarly, more than 60% of postgraduate trainees reported that discussion of adverse events takes a system focus rather than a blame focus compared with fewer than 40% of medical students. The findings regarding postgraduate trainees are consistent with the report in the UK where 66% of junior doctors understood the role of healthcare organisations in error management.20

The strength of the current study is the use of a validated measurement tool, which has been used with other healthcare students, postgraduate trainees and recent graduates. It also includes a relatively large sample (n=255) of medical students with a response rate (63%) that was higher than most reports in the literature.31–34 However, the response rate was only 32% in the postgraduate sample, which is similar to the 28% response rate reported in a study of postgraduate trainees in the UK.20 Therefore, the postgraduate trainee results should be viewed with caution given the risk for bias in the sample. For example, the results could be biased towards lower levels of self-reported patient safety competency if respondents were more likely to be sensitive to, or knowledgeable about, factors that influence patient safety, and they may, therefore, be more critical and less confident in their patient safety competency. Study limitations include the cross-sectional nature of the study which does not allow for examination of changes over time and the inclusion of only one academic teaching centre. The findings may differ across academic centres; however, our findings are consistent with other reports in the literature.13 15 16 20

The findings of this study have implications for design and implementation of patient safety content in medical education. Successful implementation of sociocultural patient safety competencies in medical education requires incorporation of concepts in the classroom, supported by actions and modelling of the concepts in a consistent manner by faculty working in a safety-oriented clinical setting.25 A true understanding of the system's nature of patient safety problems allowing medical students and postgraduate trainees to feel comfortable speaking up will only result with appropriate support and modelling in the institutional setting.