Clinical decision making in a high-risk primary care environment: a qualitative study in the UK.

OBJECTIVE: Examine clinical reasoning and decision making in an out of hours (OOH) primary care setting to gain insights into how general practitioners (GPs) make clinical decisions and manage risk in this environment.
DESIGN: Semi-structured interviews using open-ended questions.
SETTING: A 2-month qualitative interview study conducted in Oxfordshire, UK. PARTICIPANTS: 21 GPs working in OOH primary care.
RESULTS: The most powerful themes to emerge related to dealing with urgent potentially high-risk cases, keeping patients safe and responding to their needs, while trying to keep patients out of hospital and the concept of 'fire fighting'. There were a number of well-defined characteristics that GPs reported making presentations easy or difficult to deal with. Severely ill patients were straightforward, while the older people, with complex multisystem diseases, were often difficult. GPs stopped collecting clinical information and came to clinical decisions when high-risk disease and severe illness requiring hospital attention has been excluded; they had responded directly to the patient's needs and there was a reliable safety net in place. Learning points that GPs identified as important for trainees in the OOH setting included the importance of developing rapport in spite of time pressures, learning to deal with uncertainty and learning about common presentations with a focus on critical cues to exclude severe illness.
CONCLUSIONS: The findings support suggestions that improvements in primary care OOH could be achieved by including automated and regular timely feedback system for GPs and individual peer and expert clinician support for GPs with regular meetings to discuss recent cases. In addition, trainee support and mentoring to focus on clinical skills, knowledge and risk management issues specific to OOH is currently required. Investigating the stopping rules used for diagnostic closure may provide new insights into the root causes of clinical error in such a high-risk setting.

Introduction

Primary care in the UK is provided during weekends, evenings and overnight by various types of out of hours (OOH) services, and it has become a core component of round-the-clock primary care in many countries. Accessible and high-quality OOH services provide an important mechanism to reduce service pressures and associated costs from inappropriate use of Emergency Department services.1 The majority of OOH care in the UK is provided by general practitioners (GPs), often in conjunction with emergency medical and nursing practitioners, and organised by private providers or primary care trusts. Patients typically initiate contact through a telephone triage system. Those requiring face-to-face consultations, but not urgent hospital referral, are seen by a GP at a local clinic or in the patient's home.

Patients who contact OOH services are more likely to have acute problems than those who are seen in daytime primary care. Therefore, GPs working in OOH are likely to meet a higher number of acutely ill patients than in their routine practice. Mostly these are ‘new’ patients, that is, presenting to GPs who are unlikely to know them and with little or no access to their GP or hospital records. The combination of lack of continuity and high incidence of acute illness provides a potentially hazardous practice environment, in which errors can occur.2 Although the incidence of significant clinical error is unknown, high profile adverse incidents have received significant media attention3 and prompted a fundamental review of OOH service provision in the UK.4 Recommendations focused on clinical governance, more effective performance management, better team work and making patient records more accessible. In addition, they have highlighted staff development issues such as the possible use of checklists, regular feedback, improving GP training for the specific demands of OOH and individualised audits.

However, the ways in which GPs make clinical decisions in OOH has received little attention. Yet, how clinical decisions are made under these circumstances is currently unknown, and little is known of the factors that affect safety and referral patterns. We therefore examined clinical reasoning (CR) and decision making in an OOH setting, with the aim of gaining insights into how GPs manage risk while keeping patients safe in this environment and focusing on factors they may improve the delivery of services.

Methodology

Studies of CR describe the reasoning processes employed during a consultation.5 We have previously published a model of CR based on our work with GPs.6 The assumption behind the model was that understanding the reasoning process was essential for fruitful reflection on clinical practice. The model highlights key areas of knowledge and critical value judgements that are used in the clinical encounter. We used this model, which is derived from the dual theory of cognition,7 as the framework for a qualitative study of OOH practice.8 We are not aware of other studies that have tested the applicability of similar theory-based models to real-life consultations, though generic theoretical models have been proposed.9

The dual theory posits two systems interacting with each other. The non-analytic system 1 provides the fast response, while the reflective system 2 is slow and may override the original response. There is rapid recognition of a salient feature of the case,10 which may stem from the context provided by what is already known about the patient, physical appearance or critical cues specific to a presentation. This links to pre-existing mental representations, based on theory and experience built up of multiple exposures to similar cases. While some aspects of the salient feature can be described by the clinician, others may appear no more than a ‘sense of alarm or reassurance’, described as ‘gut feelings’ by Stolper et al,11 which impact on the reasoning and decision-making process of GPs. This combination of recognition and gut feelings leads to a limited search for more cues. System 2 represents reasoning through the initial response vis-à-vis the rules provided by the theory of the profession and may lead to correcting or overriding system 1. In cases where there are no salient features perceived, system 2 takes over in place of the absent fast response.7 The initial judgements provide the frame for the direction of the search for further information. Drivers prioritise the search, generally aimed at risk avoidance with the focus on ruling diagnostic options out or in, through searching for a few highly critical cues that support or negate the original frame. A necessary aspect of this kind of reasoning is a stopping rule for data collection, but we have not been able to find any research that describes them in a clinical setting.12

Setting and cohort

We conducted a 2-month study in Oxfordshire, UK, where OOH care is provided by a combination of different services. Most weekend, evening and holiday care is provided by GPs employed by the primary care trust. Overnight services are provided by a private provider. Initially, we invited all 62 GPs who regularly worked in the OOH service organised by the primary care trust. We asked them to participate in 30 min interviews about two recently seen OOH cases. We chose 30 min to minimise the pressure on busy work schedules. From past experience, most GPs would be interviewed before or after consulting and often during their brief lunch breaks. The time allocated was deemed sufficient to capture the data on the issues on which we were focused. Information about the study was kept to a minimum to reduce respondent bias. We obtained positive responses from 29 (47%) GPs and interviewed 21 (34%) GPs, who provided a total of 42 cases. Four GPs had been in practice <5 years, nine for 5–20 years and eight for >20 years (range <1–37 years). In our experience and in reports in the literature, busy professionals are relatively difficult to persuade to find time for an interview. Therefore, one is generally unlikely to find random cohorts, the participants coming through word of mouth and professional contacts. The only variable we would expect to have an impact for this study was length of experience, and in this respect, our cohort was relatively well represented.

Design and data collection

To provide face validity to our findings, we used recently seen cases, rather than case vignettes in a laboratory setting. Participants were asked to describe the presentation of two new patients who had consulted with them during their most recent OOH session and interviewed within 48 h of their session. They were asked to discuss a straightforward case and a more demanding one. After the first few interviews, it became apparent all straightforward cases offered were young children with diarrhoea or raised temperature. Subsequent interviews excluded this group of presentations. Most GPs referred to computer printouts of the cases during the interviews. All interviews, conducted by a single researcher (JB), were analysed, audiotaped and transcribed. The GPs were asked to describe the presentation of the patient and told that there would be interruptions to clarify what went on in their mind during the consultation. The semi-structured interviews used open-ended questions and focused on each step of the model (appendix 1).6

The literature suggests that the greatest impact on decision making occurs at the first step which frames the direction of the process and at the final step of diagnostic closure when immediate data collection stops and where most of the errors are likely to occur.13 14 Our focus for the first step was to what extent GPs could describe the salient feature that would lead to recognition of the case. We specifically did not attempt to dig deeper or look for gut feelings, at the same time acknowledging their prominence in the diagnostic process. On the other hand, we explored these instinctive aspects at the stage of closure. This relates to our particular interest in this step. The scope of the study would not allow us a full exploration of all facets of each step.

Data analysis

The audiotapes were used to develop brief typed summaries of each case to gain a sense of the processes; these were then used to build a structure for the analysis. This was carried out with NVivo software. Two researchers searched the text independently for underlying themes in relation to our conceptual framework. Categories were coded according to emerging themes and added to or changed as new concepts emerged.15 Any differences in interpretation were discussed, and consensus was reached on clustering to common themes.

We divide the results into five areas: (1) the participants' perceptions of the OOH environment and its impact on practice, (2) characteristics of a case to make it straightforward or demanding, (3) the process of reasoning in this context, (4) prerequisites for diagnostic closure and (5) learning points from OOH consultations for GP trainees. Within each area, we report on common themes raised by the participants.

Results

All participants indicated that our questions were easy to understand and many remarked that it was a good way for them to reflect on their practice. The analysis was consistent with previous findings on the dual process of reasoning, where instant recognition, followed by reasoning and value judgements lead to a search for a limited number of cues prior to diagnostic closure. (Direct quotes from the interviews appear in italics.)

Key perceptions of OOH environment and its impact on practice

The most powerful themes to emerge related to dealing with urgent potentially high-risk cases, trying to keep patients out of hospital, the concept of ‘fire fighting’, time pressures and working in an unfamiliar environment (table 1). This challenging environment impacted on their practice in a number of ways. It was often difficult to work out whose agenda they had to deal with, the patient's or concerned relatives or carers who had contacted the OOH service. Their approach to the patient was to deal with immediate problems only, rather than the usual holistic approach, as described by one GP: basically you've got to decide whether this chap is safe. Lack of feedback and follow-up was the rule: you don't really tend to get any feedback at all unless something has gone horrendously wrong (table 2).

Table 1

Participants' perceptions of out of hours (OOH) environment (quotes in italics, numeric refers to interviewee ID). Emerging themes to question: How is the OOH environment different from normal practice and how does it impact on your practice? Frequently these comments came up spontaneously

Theme	Example
Avoiding hospital referral	avoid hospital…indication [to refer only if] either further assessment is required or there is some definite intervention that the hospital can provide (11); intravenous fluids which of course can't be given in the GP context (13);
Fire fighting	only deal with the immediate situation that's sort [of] a bit of fire fighting (12); what can we do to make sure he's safe to be seen by his GP in the morning or do we need to admit him tonight (3);
Time pressure	not doing the other patients a service if they are sitting out there waiting (19);
Practising in an unfamiliar environment	easier to practise medicine in your own environment (5);
High-risk patients	see a high proportion of high risk patient in out of hours…frightening sometimes (21);

Table 2

Impact of out of hours (OOH) environment on practice

Theme	Examples
Lack of background information and lack of clarity about agenda	Challenging because you don't know the patients (9); whose agenda? (6); you don't have access to the records (16);
Approach to patient	I'm not quite so patient centred…a bit of fire fighting …don't want to miss critical things and if there's a red flag symptom that comes up…[it] doesn't need to be acted upon (15);
Lack of follow-up	The sad thing about out of hours is that you don't follow-up on most of your patients (18);
Lack of feedback	in reality as an out of hours presentation… it is quite difficult to find out what the discharge diagnosis was (17);
Attitude to work	a rewarding part of my work…best provided by experienced general practitioners (11);

What makes a case straightforward or demanding?

There were a number of well-defined characteristics thath made presentations easy or difficult to deal with. The focus was very much on illness severity, and a severely ill patient was seen as straightforward, as they just required hospital transfer: It's easy when somebody is terribly unwell…going to admit them to hospital. Oldery patients were often difficult because of the complex nature of multiple pre-existing diseases in the absence of background information. Children and young adults were seen as straightforward with well-defined single conditions: Easy cases tend to be younger people with less complicated medical histories and where serious things are less common. Familiarity with the condition helped, while falling between the two extremes tended to lead to lack of confidence (table 3).

Table 3

Key themes to differentiate between straightforward and demanding presentations. Question: Can you explain why you felt that Case 1 was straightforward and second one more demanding or difficult? (S: straightforward, D: demanding)

Theme	Example
Salient feature rapidly recognised or unclear	S: fine within about one minute of seeing the baby just by looking (3); D: vague and there was sort of lack of symptoms a lack of I suppose a good history (4);
Familiar or not	S: just the run of the mill you know straight forward diagnosis that you see commonly everyday (10); D: uncomfortable because I had not seen a rash quite like that before (7);
Age group	S: easy presentations occur in young people with no medical history and whom you don't have to worry at all about referring to old notes (15); D: chronic ongoing problems where lots of doctors (16);
Degree of ongoing uncertainty	S: it's about the degree of risk that I take…it is about the degree of confidence that I have whether I can help them or not (14); D: I had a degree of uncertainty about the second case that I didn't have about the first one (12);
Severity of illness	S: Global impression that child is well [or the other] be dead in a few hours (1); If not serious, may or may not know the diagnosis but does not matter (6); you cannot make a definite diagnosis, no red flags (13); D: more difficult I think the fact that there was a longer differential…the stakes were higher…I had a degree of uncertainty (12);

The process of CR

In most cases, there were instantaneous formulations, for instance, on seeing a 2-year-old child with 2 days of fever, runny nose, coughing and pulling her left ear: the likelihood is that it was probably a upper respiratory tract infection…with otitis media. The salient features that brought these to mind were generally recognised: had all the signs of a grizzly toddler tugging at the left ear, yep have to exclude an otitis media. An inability to make such instantaneous judgement made the case difficult and required data clarification, which was not always achieved. Once presentation issues were clarified, early formulations were associated with brief lists, generally headed by the most high-risk clinical conditions. This drove the search for one or two critical cues to rule out these. There was a particular focus on red flags which potentially indicated severe illness: she's quite dry…her oxygen saturation is a little bit low… I'm really thinking about, is this lady compromised because of her diarrhoea? (table 4).

Table 4

Process of reasoning in this context. Question: Can you describe what was the main driver of the process? Often these statements were made spontaneously during the interview. (Digit after stop refers to 1st or 2nd case for interviewee.)

Themes	Example
Clarify issues and whose agenda to deal with first	S: think firstly you have to identify what the real issue is (6.1); D: firstly, what real issue is, in OOH may be very problematic (1.2); too vague to formulate any thoughts (4.2);
Looking for critical cues and red flags	S: my main concern then was about her hydration (15.1); D: Especially fast resp rate without infection (1.2); wanted to establish that she was not confused (2.2);
Process of ruling out serious illness first	S: I sort of work backwards rather than forwards and so I want to rule out the serious things (16.2); I want to make sure they are systemically well (15.1); D: First thing was likely to be the usual just rule out anything serious (1.1);
Process of ruling in common and safe condition	S: rule in, since nil high risk (4.1); purpose was to confirm the diagnosis which she had given me looking for evidence to support…hypothesis (12.1); D: initially I started with a net looking for the common causes (17.2); getting to the correct diagnosis and keeping her safe (20.2);
Reassure patient and carers	S: addressing the patient's concerns (15.1); D: my main concern was safety,…how could I keep her as safe as possible while trying to respect her autonomy (10.2);
Risk assessment	S: [ketotic pregnant woman]: probably 90% certain that it was a urine infection, I would say 100% certain in this particular case that admission was needed (13.) ; I was just slightly concerned that he was starting something more serious so I said if his temperature went up persistently…or if he became breathless…he should contact back again and that there was somebody on the out of hours all weekend (14.1); D: err on the side of caution…threshold to me feels like a fairly nebulous thing…the clinical state of the patient was surprisingly good (5.2); I was reasonably confident…there was this possibility that we would be able to [treat] her orally (11.2);
Used another opinion	well I actually got somebody else in to have a look at it (7.2); speaking to the medical registrar at the hospital…the other deciding factor (10.2);

Prerequisites for diagnostic closure

Diagnostic closure occurs when the clinician stops immediate data collection and makes a management decision based on the available information. A rule of thumb used by one GP defined the issue: dividing people into two groups…on the basis of one question: is it serious or potentially serious or not? Confidence to stop and make a decision was then assessed using a number of criteria as in table 5.

Table 5

Prerequisites for closure of diagnostic process. Question: Can you try and explain what made you stop the process at this particular stage?

Themes	Example
Perception of patient systemically well or unwell	Well: I didn't think she was medically kind of unwell (4); vital signs were good (13); Unwell: She looked ill and she was vomiting profusely (19);
Ruled out urgent or severe illness	urgent problems…excluded…don't really know what's going on now, but I didn't think it was anything serious (3); I felt I'd done more excluding than I had done confirming (17);
Ruled in non-urgent diagnosis with satisfactory confirmation	know it's to be one of the simpler things but you have to ask the questions, and I guess how much is enough I mean at the point when I'm satisfied that there is significantly more confirmation of one of the simpler things than one of the nasties (5); the history and the examination coalesce on a simple diagnosis (11);
Red flags present or absent	ketotic…don't think I could have done anything else (13); her gait was staggering (21);
Set up good safety net	safety netting because everyone makes mistakes, (1 9); I normally very scrupulous about safety netting because I'm paranoid about missing things (7); a gold star safety net in this case…driver knew the patient (2);
Responded to patient's needs	…in terms of the diagnosis I think I was less than 50% sure…, but in terms of what was best to do for the patient…towards 100% (18); I hadn't succeeded in allaying his anxieties (6); Stop since good explanation that is rule in and also consistent with parent expectations (7);

Confidence that appropriate threshold had been reached was rarely stated numerically by participants, and when asked to assess this way, it was generally over 90%, but some were not willing to go beyond a descriptive assessment. As a prerequisite to closure, GPs depended on a combination of three criteria being met: (1) Confidence in ruling out a list of diseases, with clarity about illness severity: I'd reached the end of what I could do. (2) Meeting patient's needs: I'd relieved the patient's anxiety that was the main issue at stake. (3) An ability to set up a good safety net: a gold star safety net in this case. This is described as particularly important in emergency settings where diagnostic uncertainty is prominent and the patient's condition may deteriorate unexpectedly.16 It was consistently stressed during our interviews: mum was,…supportive,…safety netting was pretty major, and as a consequence, GPs went to great lengths to set one up if necessary.

Learning points for GP trainees

This was the last question we asked about each case. It turned out to be particularly useful as it gave GPs time to pause and reflect on what they regarded as particularly significant training issues relating to that consultation. Emerging themes divided into (1) training needs on how to deal with patients, stressing the need to develop rapport despite the intense time pressures; (2) learning to deal with uncertainty: think about how certain am I, how certain do I need to be to send him home and (3) learning about common presentations with a focus on critical cues to exclude severe illness and a few high-risk diseases: what are the danger signs you would look out for; specific symptom questions you want to ask (table 6).

Table 6

Trainee take home messages. Answers in response to question: What sort of message would you like your trainee to take home from this consultation?

Theme	Example
Patient-related issues	Take time to establish a rapport (2); listen to the patient's concerns (15);
Case-specific learning	this is how chest infections present in the elderly (3); common things that can cause sudden deterioration in elderly patients (13); you know these kinds of vague symptoms in an elderly patient probably you would want to rule out anything sinister or acute (4); I'd want them to sort of look up investigations for unexplained bone pain and what would be the key questions for red flags and what blood tests they might want (9);
The process	looking at the global perspective of the child (1); I said to him look I really don't find signs of anything serious or anything potentially serious just now (6); the take home message is the safety net (5);
Dealing with uncertainty	uncertainty…sometimes you don't know…talk them through the probability side of things I would discuss the things that lead me towards one way or another (7);

Discussion

To our knowledge, this is the first study to show how the context of OOH practice impacts on the CR process in primary care. Participants were aware that their behaviours and decisions were affected by the environment. In line with the perceived purpose of OOH care, objectives were to keep patients safe, respond to their needs and reduce unnecessary hospital attendances. Their expectation was to deal with urgent and potentially high-risk issues only, avoiding hospital admissions unless patients were at unacceptable risk.

GPs found practice in OOH rewarding but identified a number of issues that made this a high-risk clinical environment. The most prominent of these include a lack of background information in the form of medical records and complex multisystem chronic disease presentations where it was difficult to define the pertinent issues. Success depended on clinical skills to develop rapport with patients and recognise their concerns. Lack of regular feedback on clinical performance was raised repeatedly.

Our findings were consistent with the dual theory of cognition. In line with system 1, there was instant recognition on finding salient features of the presentation. Even before GPs knew the presenting complaint, they were constantly on the lookout for severe illness (tables 1 and 3), using red flags (table 4) and well-tried markers of systemic illness (table 5). Lists and markers were derived from personal experience, and the novice would struggle in their absence (table 6). These early cues would frame the immediate search for further cues. The major driver was to exclude serious disease and severe illness (table 4). Ruling out is commonly used in everyday practice,17 but it was practically uniform in OOH. It was surprising how often this was seen as ‘working backwards rather than forwards’. System 2 that represents correcting or overriding system 1 was used in many but not all cases. For instance, ruling in first was used only occasionally, in straightforward familiar presentations, and then followed by ruling out in most cases. Once serious disease, severe illness or the risk of abrupt deterioration were excluded, the precise diagnosis became less important and could be left for another day. Such decisions, as they often required reflection, represented system 2. Diagnostic closure that occurs after a clinician has weighed up all the options and needs to choose a specific course of action, in all but the most clear-cut presentation would involve such reflection (table 5). This is the point where clinical errors crystallise, and therefore, understanding the rules and thresholds for stopping provides insights into the causes of diagnostic error. Little is known about how diagnostic closure occurs in most clinical specialties and settings, including primary care. The OOH setting provided a unique opportunity to gain insights into diagnostic closure, as clinical care is associated with clear-cut end points, not often seen in other practice settings. The stopping rules used by GPs involved three criteria: first, have high-risk conditions and severe illness that require urgent hospital attention been excluded? Second, have I responded directly to the patient's needs? Third, is there a reliable safety net in place, given the high-risk environment where error may be more likely than in a familiar situation?

The basis of individualised judgements about acceptable confidence levels for the stopping rules were difficult to define and as one GP put it ‘feels like a nebulous thing’. This is consistent with a previous study, which used an end point of variation in hospital referral rates and concluded that the individual nature of learning from experience leads to a variation in approach to risk tolerance and associated confidence levels for decision making.18 We believe that our findings are consistent with the work of Gabbay and le May,19 who suggested that multiple exposures to clinical experience, embedded in a social context lead to a set of ‘internalised, collectively reinforced and often tacit guidelines…malleable to deal both with individual patients' needs and the multifarious factors that come into reckoning when making decisions.’

Strengths and limitations of the study

This is the first study that we are aware of that has explicitly explored the diagnostic reasoning that occurs in OOH primary care and is likely to be relevant to other countries with similar systems of primary care. The design of the study is based on a strong theoretical framework provided by the dual theory of cognition. This has been shown to have generalisability to human reasoning irrespective of the domain.7 There are also good theoretical models suggesting that clinical thinking can be modelled within this framework.9 The study also has face validity using recently seen cases, where participants were not made aware of the content and purpose of the interviews.

Limitations relate to a number of issues. (1) Sampling: participants were not chosen at random but consisted of self-selected individuals who made up a third of a cohort we approached. This is an almost universal problem with studies that involve busy and senior clinicians. We asked participants to select the two most recently seen patients who met the selection criteria. We cannot be certain if these rules were always applied. We have not performed similar patient-based studies in other settings in the UK or in countries with different health systems, and to this stage, the research has not included other specialty groups. (2) There are concerns regarding the validity of think-aloud protocols, but they have been considered appropriate for studies of cognition in natural settings as used here.20 (3) We designed the study to allow us to focus on specified steps of the CR process, for example, closure. This means that some steps could not have full coverage of issues, such as some of the crucial intuitive aspects of the early stages of diagnosis. This will require further qualitative work. (4) Another issue relates to triangulation. These qualitative studies lead to the development of inventories suitable for triangulation through quantitative studies.

We believe that the dual theory of cognition is the closest model to how people actually think in real-life scenarios and this makes it an appropriate way to study reasoning in a clinical setting.13 On account of the strong theoretical base, our findings, albeit with different emphases, are likely to have some degree of generalisability to other specialties and health systems.

Implications for practice

High-quality OOH care is important to the functioning of the health service and particularly the interface between primary care, emergency care and acute hospital care. While the results of our investigation provide a unique insight into the clinical decision making that occurs in OOH, we have several practical recommendations based on our research which could potentially be implemented with minimal additional resources.

Recognising the limitations of our study, we also find that our conclusions are consistent with the literature on the importance of feedback on performance and settings for fruitful reflection on experience. We believe that given this congruence between the relevant literature and our proposals, our recommendations merit cautious acceptance and evaluation.

First, at the organisational level, an automated, regular and timely system of feedback to GPs and other clinical staff practising in the OOH setting is essential. Providing clinical care in an environment almost devoid of clinical feedback (apart from when there are complaints or serious events) is contrary to the type of practice that GPs are trained to work in. One of the mainstays of primary care practice is the longitudinal relationship with patients, so that information gathering and clinical decision making occur over several encounters. In the OOH setting, this process becomes compressed and is compounded by high patient flow. Since the vast majority of OOH care is provided by local GPs and other staff, an initial step would be to provide routine feedback on the outcomes of their clinical decisions. Since most OOH services are now computerised, this could be implemented with little additional resource. These systems need to be implemented with sensitivity, recognising the risk of creating anxiety and dislike of being monitored. Feedback needs to be seen as non-judgemental and to be used by the recipient and trusted colleagues only. The importance of regular feedback on performance to gain expertise is well accepted in the literature.21

Second, for the individual, in line with the principles of fruitful learning from experience, we suggest that peer and expert clinician support with regular meetings and feedback on recent cases should be implemented. The educational literature provides good evidence that practice with feedback enhances deep learning, which goes with improved understanding of subject matter.22 GPs would then be in a position to reflect, calibrate their decision making and update their mental constructs, all necessary for good practice. Some GPs noted that they previously had used informal coffee breaks to do this, but the increasing numbers of patients attending OOH now precluded this. However, such informal systems are probably inadequate, and we propose that OOH practice should experiment with formal opportunities for reflection in a safe and supportive environment.

Third, our research has implications for training GPs, whose requirement for OOH training and experience is fairly modest. Individual mentoring and group discussions need to focus on (1) how to develop rapport with new and anxious patients; (2) how to deal with uncertainty and (3) common presentations with a focus on critical cues to exclude severe illness and the relatively few high-risk diseases. We believe that the model of deliberate practice, as described by Ericsson,23 is likely to be a suitable model for training. The clinician seeks out cases in areas of perceived weakness, and practice is associated with immediate individual feedback and group discussions. To this stage, excellent improvements have been demonstrated in interventional specialties but not in primary care. However, in view of the confluence of the principles of deliberate practice and deep learning approaches, we recommend the evaluation of deliberate practice training models for trainees.

We also draw attention to the ongoing difficulties in accessing GP or hospital records from OOH settings; a situation that we acknowledge is unlikely to change without the continued significant investment in the NHS IT infrastructure.

Implications for future research

We recognise that our study is a starting point for further research in this area. To explore the generalisability of our findings to other OOH or alternative emergency settings, our studies will need to be replicated with appropriate cohorts of GPs and other clinicians working in such environments. We also propose to develop a quantitative questionnaire based on our qualitative studies and on leads that we find in the literature. Such studies, extending beyond our cohort, may then provide data to allow greater generalisability of the results. We do not know if the practice changes that we propose will lead to better clinical care and this needs separate evaluation. Finally, investigation of the rules clinicians use for diagnostic closure may give us insights into the root causes of clinical error. We are at this stage in the process of developing a qualitative study, examining diagnostic error in general practice.

Conclusions

This study provides the first evidence of how working in an OOH environment impacts on CR and decision making. GPs identify OOH as a potentially high-risk clinical environment and adapt their practice approach to this setting. Lack of access to GP or hospital records is a major concern. Our study provides further support to the literature on reflective practice and educational programmes in clinical settings. We suggest that improving feedback to GPs about their clinical decisions and providing opportunities for reflection on OOH practice may be valuable for ongoing review and improvement of clinical practice. The findings provide guidance to the development of GP training programmes to cover the specific needs of OOH. Exploring further the rules and criteria that clinicians use for diagnostic closure may give us insights into the root causes of clinical error.