Improving preparedness of medical students and junior doctors to manage patients with diabetes.

OBJECTIVE: New medical graduates are the front-line staff in many hospital settings and manage patients with diabetes frequently. Prescribing is an area of concern for junior doctors, however, with insulin prescribing reported as a particular weakness. This study aimed to produce an educational intervention which aimed to improve preparedness to manage patients with diabetes and evaluate it using a mixed methods approach. RESEARCH DESIGN AND METHODS: An e-resource (http://www.diabetesscenariosforjuniordoctors.co.uk) was created to contain commonplace and authentic diabetes decision-making scenarios. -32 junior doctors (n=20) and year 5 students (n=12) in South West England worked through the scenarios while 'thinking aloud' and then undertook a semistructured interview. Qualitative data were transcribed verbatim and analyzed thematically. Participant confidence to manage patients with diabetes before, immediately after, and 6 weeks after the educational intervention was also measured using a self-rating scale.
RESULTS: Participants reported that patients with diabetes were daunting to manage because of the wide array of insulin products, their lack of confidence with chronic disease management and the difficulty of applying theory to practice. The e-resource was described as authentic, practical, and appropriate for the target audience. Junior doctors' self-rated confidence to manage patients with diabetes increased from 4.7 (of 10) before using the e-resource, to 6.4 immediately afterwards, and 6.8 6 weeks later. Medical students' confidence increased from 5.1 before, to 6.4 immediately afterwards, and 6.4 6 weeks later.
CONCLUSIONS: Providing opportunities to work with authentic scenarios in a safe environment can help to ameliorate junior doctors' lack of confidence to manage patients with diabetes.

Junior doctors reported that managing patients with diabetes was a common, but daunting, task.

An educational intervention that involved interactive, authentic diabetes management scenarios and access to expert thinking improved confidence.

Junior doctor confidence to manage patients with diabetes appeared to be lower outside the acute setting.

Junior doctors often appeared unfamiliar with national guidelines and some felt they were unwieldy to use.

Introduction

According to the WHO, approximately 1.5 million deaths were caused by diabetes in 2012,1 and the global prevalence of diabetes was around 9% adults in 2014.2 With such a prevalent disease, junior doctors who are the front-line staff in many hospital settings encounter patients with diabetes frequently. As such, they need to make basic decisions about managing patients with diabetes and know when to seek help from more senior or specialist staff.

Junior doctors, unfortunately, do not feel well prepared to prescribe in general, or for diabetes in particular.3–5 In 2011, George et al5 undertook a national online survey of trainee doctors (foundation and specialist trainees) in the UK. They assessed the confidence of respondents using the Royal College of Physicians ‘Confidence Rating’ four-point scale (CR1: ‘not confident’, CR2: ‘satisfactory but lacking confidence’, CR3: ‘confident in some cases’, and CR4: ‘fully confident in most cases’). Confidence in diabetes management ranged from 55% for diagnosing and managing hypoglycemia, to 18% for altering diabetes therapy prior to surgery. Reported confidence across all diabetes-related domains were lower than in the two similar areas of medicine studied with 66% fully confident in the management of angina and 65% fully confident in the management of asthma, irrespective of stage of training.

These findings are worrying given that prescribing insulin is considered a high-risk activity, being error prone, and with errors having potentially serious consequences for patient safety. Yet 75% of foundation year 1 (FY1) doctors in a UK study failed an applied knowledge test on this topic.6 There are many existing resources that aim to impart knowledge to junior doctors about diabetes,7 but very few focus on practical skills, or encourage learners to articulate how they would manage different scenarios or to visualize themselves in role, despite good evidence that medical trainees learn best through doing.8

One exception is the study by Conn et al,3 which involved 15 junior doctors (midway through their first postgraduation year) who undertook two 1 h simulated scenarios workshops on the practical skills required to manage glycemic control of insulin-treated patients. They assessed performance and confidence before, during and 3 months after the workshop. Participants rated confidence in performing the tasks specified in simulated exercises and their overall confidence in managing patients with diabetes requiring insulin treatment. Confidence was rated on five-point scales where 0 was ‘not very confident at all’ and 4 was ‘very confident’. Their pretest score for overall confidence in ‘managing patients with diabetes’ was 2.73 of a maximum 4. This went up to 2.80 immediately after the intervention and to 2.93 3 months afterwards. Although these are modest effects, it encourages further research in this area.

We set out to explore the self-assessed preparedness of senior medical students and junior doctors in the UK to manage patients with diabetes; and produce and evaluate an educational intervention which aimed to improve preparedness.

Research design and methods

Study design

Bleakley9 describes the development of new models of apprenticeship that support novices not just to do the job but to ‘think and recount the job’ (p.154)—a so-called cognitive apprenticeship. We have used this idea to shape the development of the educational intervention, an e-resource which provides insights into the cognitive processes of experts, and the evaluation, which uses a ‘think aloud’ methodology to provide insights into the cognitive processes of participants. The research team comprised three authors, plus a clinical stakeholder group. NAAK, who carried out the data collection, is a psychologist by background with qualitative research expertise, and had no teaching, assessment, or clinical involvement with the study participants. KGB is a medical educator with a pharmacology background and expertise in technology-enhanced learning. KLM is a medical educator, with a background as a clinical scientist in microbiology, and substantial experience in both quantitative and qualitative research. The clinical stakeholder group comprised five individuals: a consultant diabetologist, a consultant in acute medicine, a pharmacist with expertise in patient safety, a junior doctor, and a diabetes specialist nurse.

Developing the educational intervention

Members of the clinical stakeholder group were interviewed to understand the range, and relative frequency, of ways in which junior doctors encounter patients with diabetes. The aim of the educational intervention was to provide learners with common and important scenarios relevant to the management of patients with diabetes, appropriate to training stage, in order to increase confidence to manage those situations. The specific objectives were to develop an open-access e-resource, featuring authentic scenarios and encompassing work-based activities supported by relevant resources, with embedded feedback that provides insights into expert cognition. A story board for an e-resource was developed and shared with the clinical stakeholder group for feedback. A suite of 13 scenarios were created (eg, acute presentation of diabetic complications, new treatments, managing high blood sugars, taking down intravenous infusion, nasogastric feed and insulin, perioperative diabetes control, managing a hypo, preparing for discharge, different types of insulin), using video/audio clips as a means of engaging the user and providing contextual cues. An open-access e-resource was then created by an independent web developer: http://www.diabetesscenariosforjuniordoctors.co.uk (figure 1). The interactive online scenarios required the learner to state what they would do in certain situations (in line with the cognitive apprenticeship approach taken), since this commitment to a specific course of action should encourage more meaningful engagement with the subsequent expert response and thereby improve learning. Sometimes the user had a task to do, such as completing a drug chart, which was relevant to actual ward-based tasks that they would need to undertake. The activities were supported by information resources such as national guidelines, which were the same resources likely to be used for ward-based decision-making, thus providing authenticity and raised awareness. After submitting the responses, the learner could review an ‘expert response’ to the questions (again, in line with a cognitive apprenticeship approach) and see their exemplar completed drug charts.

Figure 1

The landing page of the open-access e-resource, available at http://www.diabetesscenariosforjuniordoctors.co.uk.

Evaluating the educational intervention

The aim of the evaluation was to determine to what extent the e-resource was successful in its aims, namely to improve confidence in managing patients with diabetes by providing relevant and appropriate pitched learning opportunities, that encouraged participation in authentic tasks and access to expert cognition. Twenty FY1 doctors, in their first year postgraduation, working at a hospital trust in South West England, and 12 final year medical students studying at a medical school in South West England were recruited by email and paper flyer to participate in an evaluation of the e-resource. This participant group was selected as those needing to be prepared to manage the scenarios covered by the resource. Each evaluation session comprised the participant working through five selected scenarios within the e-resource, while thinking aloud followed by a short interview. The choice of interview questions was informed by the clinical stakeholder group. Participants were first shown a worked example from the e-resource to demonstrate its use and then completed the five scenarios themselves in a prespecified order. A think aloud approach was used to enable participants to verbalize their thought processes,10 11 which provided insight into their experience of using the e-resource, as well as their clinical reasoning processes in working through the scenarios. The researcher played no part in this activity, though participants were able to ask clarifying questions and the researcher prompted where necessary to ensure the task was on track and the participant was thinking aloud. After working through the scenarios, the researcher asked open-ended questions of the participants to evaluate the e-resource, including ease of use, quality of content, relevance, and usefulness. The evaluation session was audiotaped and transcribed verbatim. Thematic analysis of the qualitative data was undertaken by one researcher (NAAK) with independent review by a second researcher (KLM) and the final coding framework agreed through consensus. Participants were also asked to complete a one-question scale three times during their involvement in the project: before the evaluation session, immediately afterwards, and approximately 6 weeks later. The question asked the respondent to rate, on a scale of 1–10 with 10 representing the highest level, how well prepared they felt to manage diabetes in hospital settings.

Ethics approval

The project was approved by a fully constituted Research Ethics Committee and by the research and development department of the hospital trust involved.

Results

Participants

Demographic data for the 20 FY1 doctors and 12 fifth year medical students are given in table 1.

Table 1

Participant demographics

	Year 5 medical students (n=12)	FY1 doctors (n=20)
Age	Range 22–40	Range 23–42
	Median=24	Median=25
	Mode=24	Mode=24
Gender	7 female	11 female
	5 male	9 male

FY1, foundation year.

Impact of the e-resource on confidence

Junior doctors involved in the evaluation scored their confidence to manage patients with diabetes as 4.7 of 10 before using the resource, 6.4 immediately afterwards, and 6.8 6 weeks later (table 2).

Table 2

Changes in mean scale score reflecting confidence over time in FY1 doctors (n=20) and medical students (n=12): before using the e-resource, immediately after the session, and after 6 weeks

	Mean score
	FY1: all (n=20)	FY1: male (n=9)	FY1: female (n=11)	Students: all (n=12)	Students: male (n=5)	Students: female (n=7)
Presession	4.7	4.7	4.7	5.1	5.6	4.7
Postsession	6.4	5.9	6.8	6.4	6.2	6.6
6-week follow-up*	6.8	7.1	6.4	6.4	6.4	6.3

*Three female FY1s and one female medical student did not complete a 6-week follow-up questionnaire.

FY1, foundation year.

Medical students scored their confidence to manage patients with diabetes as 5.1 of 10 before using the resource, 6.4 immediately afterwards, and 6.4 6 weeks later (table 2). Around half of the participants rated their confidence as two points higher after working with the e-resource. This suggests that using the e-resource improved confidence of medical students and junior doctors to manage patients with diabetes, and that this impact lasted at least 6 weeks. The qualitative data helped to elucidate the reasons underpinning this improved confidence (see below). Three female FY1s and one female medical student did not complete a 6-week follow-up questionnaire. The data show that the change in scores between presession and postsession for female participants were generally greater than for the male participants. However, the mean female participants’ scores decreased between the postsession and 6-week follow-up, whereas the male participants’ scores increased (table 2).

The qualitative data revealed three main themes (table 3).

Table 3

Thematic index from qualitative interview data, based on data that were self-reported as part of the interview discussions

Interview theme	Interview subtheme
1. Challenges of patients with diabetes	1.1 Insulin products
	1.2 Responsibility
	1.3 Acute and chronic presentations
	1.4 Theory practice gap
2. Reactions to the e-resource	2.1 Overall
	2.2 Scenarios
	2.3 Supporting resources
	2.4 Expert responses
	2.5 Tasks
3. Longer term impact of e-resource	3.1 Increased confidence
	3.2 Behavior change

The challenges of managing patients with diabetes

There was broad consensus among participants that the prospect, and reality, of managing patients with diabetes was daunting:

Diabetes is something that scares me and, I think, from my medical school, it wasn't taught very well and it always seems to be quite a complicated subject with, lots of…different options and lots of different ways to do it. Female F1; F1Fh.

Prescribing insulin and prescribing diabetic medications is a common task for everyone and I feel [we] are vastly unprepared for that task. Male F1; F1Mb.

Part of the challenge of managing patients with diabetes seemed to be the wide array of insulin products and devices:

One of the big stumbling blocks and things that I find hard is…the vast arrays of insulin, pens and products and different brandings out there…and, you know, the…seeming lack of guidance on when one should choose to use a human insulin, an analogue or an animal, is there any difference or do you just pick the one because you feel like it? Female F1; F1Fd.

Similarly with the pen devices I don’t have any knowledge of that, pen devices and needles. Female Medical Student; MedFc.

Another theme was a higher degree of confidence to manage acute cases than chronic situations:

The acute stuff they [medical schools] are quite good at: DKA, hypo, but the long term care and management is not that well done and I think we had a few lectures on it but because there's so many different types of insulin and all the words are not very, like Levemir and Humulin…maybe I just switched off a bit. Male F1; F1Mf.

Presenting in hospital it's normally an acute presentation of something like DKA or hypoglycaemia erm for, then you've got the other scenarios which I didn't really know anything about erm, where you are having like surgery erm and sliding scales. Female Medical Student; MedFa.

The difference between learning something in a lecture theater and applying it with patients was highlighted by participants:Participants were predominantly positive about the e-resource. They found it simple to access and easy to use. The scenarios were deemed relevant and realistic, focusing on common situations that junior doctors will face. This appears to justify the choice of scenarios and highlights the motivation provided by learning materials that are clearly relevant to work-based tasks. The scenarios were pitched at an appropriate level for fifth year medical students and FY1 doctors:

You get this theoretical knowledge but medicine's essentially a practical thing and until you've actually done it [a medical task] you know I don’t know where things are kept, the kit, insulin. Female F1; F1Fc.

I've got a basic knowledge of, sort of, hypos and DKAs but I don’t think, if I actually was faced with a patient, I would be able to competently manage. Erm I've got sort of the theoretical knowledge but not sort of the practical knowledge of writing erm the charts. Female Medical Student; MedFb.

2. Participant reactions to the e-resource

Yes I think they [the scenarios] are really appropriate to what you actually get in the hospital ‘cos those are things that have been coming up over the last few weeks. Female F1; F1Fg.

…when you do the questions you start to think about things and umm, imagining situations you've either been in or you could be in umm, and just that process of thinking through makes you more confident. And the way that the questions most of the questions were structured really helped as well. Male F1; F1Mc.

The range of scenarios was broader than many participants had envisaged. The questions and tasks embedded in each task were reported to help the participants visualize themselves in the role of managing a patient who presented in that way, which suggests the cognitive apprenticeship principles were successful in encouraging participants to ‘think and recount the job’:

…there's a lot more scenarios which I hadn't imagined me dealing with as an F1 and so the breadth of scenarios has increased my knowledge to cover all those scenarios. Male F1; F1Mb.

Well it certainly gets you thinking umm, taking the step from thinking you know something to actually trying to fill in a chart is quite telling. Female F1; F1Fc.

The national guidelines, which were provided as links within the supporting resources section of each scenario, received mixed reviews. Many participants were unaware of them, which was an important finding given our intention to raise awareness of the national guidelines, but concerns were expressed about their length and usefulness at the point of care:

I think that the national guidelines uhh, they rely on, are interesting, uhh, nicely done, but yeah, twenty pages to go through, it's a lot to do on the moment. Female Medical Student; MedFX.

The expert response, which responded to the questions posed within each scenario, were extremely positively reviewed and described as straightforward, insightful, and concise. Participants felt they gave clear explanations and good examples:The overwhelming majority felt using the e-resource had increased their confidence to manage patients with diabetes and made them think about what they would actually do in such a situation. Working with the scenarios prompted many participants to identify gaps in their preparedness to manage patients with diabetes.

…the expert answers were, really nice, concise, to the point, and sort of snippets of all the information you needed. Female F1; F1Fe.

…what was most interesting was,…was the way that the expert gave the answer was brilliant because I had never seen that before. Male F1; F1Mc.

3. Longer term impact of the e-resource

Because I now realise that there are many scenarios…that I have no clue about and no previous training about…in terms of managing them on the ward and around surgery and discharge and new diagnosis. Male F1; F1Mb

Yeah I didn't know anything about sliding scale and insulin and about managing patients with diabetes who are having surgery. I mean I haven't really read about that at all so that's something, that's an area that it's identified. Male Medical Student; MedMa.

Some commented specifically that this new-found knowledge might change their behavior or way of working:

I feel more confident about asking other healthcare professionals, whereas before I was even more hesitant about doing that as I think my knowledge was at such a level that I didn't really think I could ask that question. Male F1; F1Md.

Following the session, I felt more confident in my own skills at managing diabetes and when to escalate treatment to seniors. I have been referring to the information supplied on the back of ‘diabetic drug charts’ more frequently since. Female F1; F1Fe.

Participants were asked if they were likely to use the e-resource again and the majority felt they would, though responses were not necessarily stated with conviction, using such expressions as ‘I think so’, or ‘probably’:

I think I would recommend it to my colleagues I am not sure if everyone would use them. I would definitely recommend it for medical students I think something like that as a student it would …have made me feel a lot more confident in terms of coming into the job as a junior doc. Female F1; F1Fe.

I think it's very good…resources like this unfortunately they aren't…always taken up as well but this is one that's actually quite useful and is applicable to a lot of things we do and it doubles up not as a learning resource but as a revision tool. Male Medical Student; MedMc.

Discussion

The aim of this study was to explore the self-assessed preparedness of senior medical students and junior doctors to manage patients with diabetes; and produce and evaluate an educational intervention to improve preparedness. In common with previous studies,3–5 participants described feeling unprepared to manage patients with diabetes. Our mixed methods approach, using cognitive apprenticeship as the guiding principle, enabled this study to provide insights into the reasons for this perceived lack of preparedness and ways in which it might be remedied. It also shows that confidence was particularly low outside the acute setting and was often associated with the wide array of insulin devices and products. The educational intervention that we created received positive evaluations from participants, who felt the scenarios it provided were relevant, authentic, and provided insights into the cognitive processes of experts that were otherwise rarely shared. Previous work supports this desire of junior doctors to understand the decision-making processes of their seniors.12

The junior doctors in our study were less confident in managing patients with diabetes than in other studies,3 but this difference probably represents the earlier stage of training in our study (at the start of the FY1), compared with the Conn et al study (midway through) rather than geographical variation. Like our study, Conn et al and George et al's research indicate that preparedness to manage patients with diabetes is a real issue and not just at the start of training. The fact that junior doctors who are already working in the clinical environment identify a number of areas where they feel ill prepared to manage patients with diabetes suggests that further training and support is urgently required to ensure patient safety. This could include formal undergraduate and postgraduate medical education, together with a more supported transition into the clinical environment, both initially and when junior doctors rotate to a new ward environment. A recent paper by Taylor et al13 demonstrated that short focused interventions can help improve confidence and patient safety and our work complements this approach. Future research will be required to explore how a learner's intention to engage with further study can be facilitated to translate into activity, since we suspect that our participants are not alone in letting intention succumb to time pressure. It will also be important to examine whether increased confidence in managing patients with diabetes correlates with increased competence, which was beyond the scope of the current study but has important safety implications.

The strengths of our research are in the careful design of an educational intervention targeting learners at a critically important transition in medical education in an area which, if successful, will have direct benefits for the safety of a large patient population. We believe the use of ‘cognitive apprenticeship’ as a framework to inform the design of the intervention and the evaluation lends rigor and coherence to the process undertaken. Furthermore, the intervention created is openly available to readers, and we hope the e-resource will be widely used, extended, and updated.

As with all research, there are also some weaknesses. With studies of this sort, there is always the potential for participants to respond in a socially desirable way, for example, to give positive evaluations of an e-resource or report increasing confidence after exposure to an educational intervention.14 We aimed to minimize this through separation of the e-resource creation and evaluation process, which involved different members of the research team, and through the 6-week follow-up which was not face-to-face. There has also been significant critique of self-assessment, including self-report inventories, in the medical education literature.15 The feedback provided by the e-resource was not tailored to individual participants, since this would have required significant ongoing funding. Although the advantages of authentic scenarios and tasks are clear from this study, participants could struggle to transfer their learning to other scenarios and tasks. This is a wider challenge for instructional methods (eg, problem-based learning, case-based teaching), rather than specific to this resource. Confidence at the 6-week follow-up could have been affected by many things other than the e-resource. Finally, the participants were from a fairly restricted geographical area (two counties in South West England) and the sample size was necessarily small in order to facilitate the detailed think aloud evaluation process. Despite these limitations, we believe this study makes a significant contribution to the literature and will inform the design of further education and support interventions that will ensure safe care for patients with diabetes.

The recommendations arising from our study are: (1) that the educational intervention described here, or other resources that incorporate authentic scenarios and practical activities, is used within facilitated teaching sessions in a safe environment, perhaps with additional materials tailored to the specific setting, since uptake as an independent learning resource may be low; (2) that senior medical staff consider how they can make their rationales and decision-making processes more available to junior doctors; and (3) that organizations creating guidelines consider how they might be applied by junior doctors within a busy clinical environment, since our participants found them unwieldy.