Subarachnoid haemorrhage guidelines and clinical practice: a cross-sectional study of emergency department consultants' and neurospecialists' views and risk tolerances.

OBJECTIVES: To establish if emergency medicine and neuroscience specialist consultants have different risk tolerances for investigation of suspected spontaneous subarachnoid haemorrhage (SAH), and to establish if their risk-benefit appraisals concur with current guidelines.
SETTING: 4 major neuroscience centres in London. PARTICIPANTS: 58 consultants in emergency medicine and neuroscience specialities (neurology, neurosurgery and neuroradiology) participated in an anonymous survey. PRIMARY AND SECONDARY OUTCOME MEASURES: The primary outcome measure was the highest stated acceptable risk of missing SAH in the neurologically intact patient presenting with sudden onset headache. Secondary outcome measures included agreement with guideline recommendations, risk/benefit appraisal and required performance of diagnostic tests, including lumbar puncture.
RESULTS: Emergency department clinicians accepted almost 3 times the risk of a missed SAH diagnosis compared with the neuroscience specialists (2.8% vs 1.1%; p=0.02), were more likely to accept a higher risk of missed diagnosis for the benefit of a non-invasive test (p=0.04) and were more likely to disagree with current published guidelines stipulating the need for LP in all CT-negative cases (p=0.001).
CONCLUSIONS: Divergence from recognised procedures for SAH investigation is often criticised and attributed to a lack of knowledge of guidelines. This study indicates that divergence from guidelines may be explained by alternative risk-benefit appraisals made by doctors with their patients. Guideline recommendations may gain wider acceptance if they accommodate the requirements of the doctors and patients using them. Further study of clinical risk tolerance may help explain patterns of diagnostic test use and other variations in healthcare delivery. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/

Novel survey addressing a controversial clinical scenario commonly encountered by all acute medical physicians.

Explanation for variation in clinical practice that has been widely criticised and has a profound impact on patients, doctors and healthcare institutions.

Pilot survey with a modest sample size (n=58).

No suitable validated instrument was available to measure clinicians' risk tolerance.

Potential participation bias due to low response rate.

Introduction

Controversies in the investigation of suspected spontaneous subarachnoid haemorrhage (SAH) principally concern the question of whether or not a lumbar puncture (LP) is needed following a normal CT scan. Missing the chance to treat acute aneurysmal SAH can have devastating consequences due to high associated mortality and morbidity.1 2 Although an imperfect test,3 4 the LP is held as a gold standard in the diagnosis of SAH, and when combined with CT has been shown to detect SAH reliably.5 Current international and national guidelines assert that SAH cannot be excluded based on a negative CT scan. Investigation is deemed incomplete unless negative imaging is followed by a lumbar puncture with spectrophotometric assessment of the supernatant for cerebrospinal fluid bilirubin.2 6–9

UK-based studies have found frequent deviation from guidelines. LP omission rates of 40–50% are common and are only slightly lower in teaching hospitals compared with District General Hospitals.10–13 UK figures are in line with international practice where rates of LP omission range between 40% and 60%.14–16

Although guidelines are not ‘railroads’ for clinicians to follow,17 omission of LP in suspected SAH diagnosis has been widely criticised,8 10 11 18 and attributed to a lack of awareness and education.10 14 However, the role of LP has been called into question due to reported low specificity, low clinical impact, low diagnostic yield and improved sensitivity of modern CT.3 4 19–23 Doctors are also known to vary in their tolerance of risk and uncertainty which may influence management decisions.24–26 The low LP rate in suspected SAH may therefore represent widespread disagreement with guidelines and greater tolerance of the risk of misdiagnosis. A fear of LP-related complications may also dissuade clinicians from performing or recommending the procedure.21

Guidelines for SAH investigation have been shown to be relatively risk intolerant compared with recommended investigations of other potentially fatal conditions presenting to the emergency department (ED), for example, pulmonary embolism (PE) and acute coronary syndrome (ACS). Guidelines for suspected PE and ACS allow small but non-zero calculated risk end points, while SAH guidelines afford no misses.27 ED clinicians habituated to accept small risks of misdiagnosis for common emergency presentations (such as PE and ACS), may be more willing to accept similar risks in SAH work-up compared with their neurospeciality colleagues (ie, neurologists, neurosurgeons and neuroradiologists). Given that patients with suspected SAH are typically managed by emergency physicians,28 any such variation in risk appraisal could help explain the frequency of LP omission.

The purpose of this study is therefore twofold:

to survey clinicians' analysis of risk and benefit related to investigation of suspected acute spontaneous SAH;

to establish whether published guidelines reflect current professional opinion, and to characterise any variation between ED clinicians and neurospecialists.

The null hypothesis is that ED clinicians are no more risk tolerant than neurospecialist consultants.

Given an LP omission rate of between 40% and 50% in the UK, at least 40% of ED clinicians are expected to disagree with the guideline stating that LP is required in CT-negative cases.

Methods

No appropriate validated instruments were available to measure risk tolerance for missed SAH diagnosis. We therefore elected to design a questionnaire for this purpose according to principles described in detail elsewhere (see online supplementary material).29 30 The questionnaire underwent initial trial use by consultant and trainee clinicians in the relevant specialities. Changes were made to wording, question format and layout following this exercise.

Each question was devised to investigate clinicians' perceptions about aspects of the diagnostic process: clinical risk factors; LP complications; risk tolerance for misdiagnosis; sensitivity requirements of diagnostic tests; risk–benefit appraisal for diagnostic tests and investigative inertia, that is, the propensity to pursue a diagnosis due to referral bias rather than personal appraisal. While piloting the questionnaire, it became apparent that respondents could be identified if they provided their age, gender and speciality. Demographic details were therefore limited to include age range (<40, 41–50, 51–60, >60) and speciality but not gender.

The primary outcome of risk tolerance was measured using a short vignette describing an otherwise well adult aged 40 years, presenting with a worst-ever ‘thunderclap headache’. Participants were asked to give the post-test probability of missed SAH that they would accept before stopping further investigations. Respondents were told that the pretest probability for patients was 10%, a conservative estimate and in line with similar clinical scenarios in the published literature.31

The primary objective of this study was to investigate patterns of risk tolerance and guideline agreement between neurospecialists and emergency physicians. A lack of validated instruments and relevant preliminary data prohibited a power calculation. We opted to sample two groups (neurospecialists and emergency physicians) comprising a quota target of 50 respondents, half from each group. Consultant neurosurgeons, neurologists, neuroradiologists and emergency clinicians were approached at four large NHS Trusts with tertiary neuroscience services: UCLH, Kings College Hospital, Queens Hospital Romford and Charing Cross/St Mary's Hospital. London Trusts were chosen to reduce the influence of regional variations.

A written questionnaire was devised in preference to an electronic survey because of concerns that electronic correspondence would be missed or ignored among the large volumes of information received electronically by NHS consultants.

Permission to conduct the survey was then sought from the clinical lead in each department. Invitations to participate were communicated by telephone, email and in person wherever possible in order to optimise participation. Secretarial staff were approached to help disseminate the questionnaires and email reminders were sent to eligible consultants where necessary. Data collection was carried out between October 2015 and February 2016.

Statistical analysis

Data were entered into an Excel spreadsheet (Excel for Mac, 2011). The significance of group differences was calculated using χ2 statistical tests within Excel. The statistical package (StatPlus) was used for descriptive statistics and for the t-test statistic where appropriate.

Patient involvement

No patients were involved in setting the research question or the outcome measures, nor were they involved in developing plans for recruitment, design or implementation of the study. No patients were asked to advise on interpretation or the writing up of results.

Results

A total of 58 consultants completed the questionnaire, including 23 ED clinicians and 35 neuroscience specialists. The overall response rate was 34% with proportional participation by both subgroups (35% for the neurospecialists compared with 32% of ED physicians). The median age range was 41–50 years and consultants had a median of 16–25 years of postgraduate experience. The gender of participants was not available for reasons described above. A total of 17 questions (1.5%) were omitted or ignored by the 58 respondents. A further seven responses were excluded from analysis because they were incompatible with the question asked, indicating a misunderstanding or misreading.

Seventeen ED clinicians and 30 neurospecialists indicated their risk tolerance for missed SAH diagnosis by recording the highest post-test probability at which they would stop investigations to diagnose SAH (Question 10). There was a significant difference in the mean scores between groups (p=0.03) with the ED clinicians' risk tolerance almost three times higher than the neurospecialists' (2.8% (SD 3.3) vs 1.1% (SD 1.9)). Neurospecialists were also more likely to advocate routine LPs compared with ED clinicians (74% vs 39%, p=0.01).

Only 70% of consultants agreed with current guidelines stipulating that an LP is mandatory in suspected SAH when an initial CT is negative (table 1). There was a significant difference between the ED and neurospecialist groups in this regard with a majority of the ED group disagreeing with guidelines (57%) compared with a minority (11%) of neurospecialist consultants (p<0.001). ED clinicians were more inclined to omit the LP if a negative CT had been obtained within 6 hours of headache onset (35% vs 3%, p=0.002).

Table 1

Agreement with guidelines

Question/dimension	All clinicians combined (n=58)	ED physicians (n=23)	Neurospecialists (n=35)	p Value
LP mandatory ‘No’	17 (30%)	13 (57%)*	4 (11%)
LP mandatory ‘Yes’	40 (70%)	9 (39%)*	31 (89%)	0.0001
LP omitted when CT <6 hours ‘No’	46 (84%)	15 (65%)	31 (89%)†
LP omitted when CT <6 hours ‘Yes’	9 (16%)	8 (35%)	1 (3%)†	0.0017

*Question omitted by one consultant.

†Question omitted by three consultants.

LP, lumbar puncture.

ED consultants also required a higher pick-up rate to justify the routine use of LP compared with their neurospecialist colleagues, but this difference did not achieve statistical significance (table 2).

Table 2

Clinicians' risk–benefit appraisals

Question/dimension	All clinicians combined (n=58)	ED physicians (n=23)	Neurospecialists (n=35)	p Value
Routine LP justified? ‘No’	23 (40%)	14 (61%)	9 (26%)
Routine LP justified? ‘Yes’	35 (60%)	9 (39%)	26 (74%)	0.007
Investigative inertia ‘No’	45 (78%)	20 (87%)	25 (71%)
Investigative inertia ‘Yes’	13 (22%)	3 (13%)	10 (29%)	0.2
Required LP pick-up rate: ≥1 SAH every 100 LPs	25 (44%)	12 (52%)	13 (38%)*
1 SAH every 101–500 LPs	19 (33%)	8 (35%)	11 (32%)*
1 SAH for every >500 LPs	13 (23%)	3 (13%)	10 (29%)*	0.3
Risk–benefit trade off†
Non-invasive test
Yes	12 (21%)	8 (38%)‡	4 (11%)	0.018
No	44 (79%)	13 (62%)‡	31 (89%)
Quicker test
Yes	5 (9%)	2 (10%)‡	3 (9%)*	0.9
No	50 (89%)	19 (90%)‡	31 (89%)*
Cheaper test
Yes	5 (9%)	2 (10%)‡	3 (9%)*	0.9
No	50 (89%)	19 (90%)‡	31 (89%)*
Experience of previous missed SAH ‘No’	5 (9%)	2 (9%)	3 (9%)*
Experience of previous missed SAH ‘Yes’	52 (91%)	21 (91%)	31 (91%)*	0.9

*Question omitted by one consultant.

†Participants were asked if they would accept a higher risk of missed SAH in the given scenarios: quicker, cheaper or non-invasive test.

‡Question omitted by two consultants.

LP, lumbar puncture.

There was no significant group difference between the clinicians' personal experiences of LP-related complications with both groups median estimate falling in the ‘infrequent’ (1 in 200) category (figure 1).

Figure 1

Clinicians' estimated frequency of lumbar puncture complications. Ordinal scale provided: Very rare is <1 in 200; Rare ∼1 in 500; Infrequent ∼1 in 200; Common ∼1 in 100; Very common ∼1 in 20.

Fewer than 10% of respondents in each group indicated a willingness to substitute the LP in favour of a cheaper or quicker test if it carried an increased risk of missed diagnosis. However, ED clinicians were more likely to accept an increased risk of misdiagnosis for the benefit of a non-invasive test (38% vs 11%, p=0.02).

Almost all clinicians in both groups reported direct personal experience of missed SAH due to incomplete investigation (91% in both groups). Fifty-five per cent of clinicians had given evidence in a medicolegal capacity and although this was more common among ED clinicians (65% vs 55%), the difference was not statistically significant.

Local practices and policies were influential in deciding work-up of SAH; however, personal experience was the most commonly cited factor by clinicians (illustrated in figure 2). Nevertheless, 22% of clinicians reported that they would feel obliged to investigate SAH if it had been raised and documented as a potential diagnosis, irrespective of their own clinical judgement.

Figure 2

Factors reported to influence work up of subarachnoid haemorrhage.

Although free text answers were not requested from participants, some responses were annotated. Where relevant, these are discussed below.

Discussion

Although SAH is a relatively rare disease, its misdiagnosis was a familiar scenario for more than 90% of consultants. The consequence of missed diagnosis can be devastating for patients and carries significant medicolegal implications for doctors and healthcare institutions. It is therefore difficult to understand the frequency with which clinicians stray from recommended practice, especially considering the defensive nature of modern medicine.32

There is an extensive body of published research evaluating the appropriateness of different strategies for SAH diagnosis. While there is some controversy regarding the utility of LP in the diagnostic pathway, current national and international guidelines unanimously recommend the procedure for patients with negative initial imaging. A recent survey of UK Emergency and Acute medicine clinicians confirms a wide variation in practice with only 74% of respondents following the recommended CT-LP pathway.33

To the best of our knowledge, this is the first study to investigate whether clinicians' risk tolerance might account for the observed variation in practice and to survey clinicians' appraisals of the relevant recommendations directly.

Almost all respondents had a personal experience of missed SAH diagnosis; however, only 70% agreed with the guidelines that LP was mandatory for all CT-negative cases. When ED clinicians were considered separately, only 39% held this view. Previous attempts to explain deviation from guidelines have suggested lack of awareness as a possible cause. Taken together, our findings suggest that clinicians are well aware of the perils of missed diagnosis, but are nevertheless sceptical of the benefits of LP in the diagnostic pathway.

It is typically the emergency and acute medical physicians who are compelled to perform an LP before referring or discharging patients with suspected SAH. Neurospecialists are more likely to be involved in an advisory capacity until a diagnosis has been made. The LP may therefore serve as a filter reducing unnecessary referral to specialist services.3 ED consultants were more willing to trade off benefits of a non-invasive diagnostic test against the increased risk of missed diagnosis. Neurospecialists did not tend to share this view, which may reflect a concern that referrals would increase if alternative tests were more readily performed.

This survey was not designed to test clinicians' knowledge of guidelines, epidemiology of SAH or the performance of diagnostic tests. Nevertheless, some responses did highlight common misconceptions, which are worth addressing here. Although the LP is taken as a gold standard test, the actual sensitivity in CT-negative SAH is unknown.28 The commonly quoted sensitivity of photospectrometry for xanthochromia comes from Vermeulen et al34 who reported 100% sensitivity up until 2 weeks after ictus (n=32). These data were derived from LPs performed in patients with CT-positive SAH.34 The sensitivity of LP in CT-negative patients would be a more useful measure since this is the context in which the LP is employed. Unfortunately, this is difficult to quantify since there is no confirmatory test against which the LP can be measured.

The high frequency of equivocal photospectrometry findings is another shortcoming of the LP. In a large observational study by Sayer et al,4 more than 15% of 1898 LPs were inconclusive by Clinical Biochemistry Guidelines.35 This large contemporary sample is arguably more representative of current clinical practice than the Vermulen cohort and demonstrates that even when applying the CT-LP paradigm, SAH cannot be entirely excluded. This may explain why ED clinicians are less likely to consider the LP as a prerequisite for SAH diagnosis.

Recent research has suggested that CT approaches 100% sensitivity in acute SAH if performed within 6 hours of ictus. This has led to calls for a change of practice.36–39 Participants were specifically asked about this scenario. ED clinicians were more willing to adopt this policy than their neurospecialist counterparts, which may reflect a desire to reduce pressure on EDs.

The concept of safe discharge has been deemed ‘irrational unhelpful and unachievable’ because it raises unrealistic expectations of a risk-free environment.40 Conversely, accepting an arbitrary threshold for risk may seem callous or even negligent as it consigns some patients to an adverse outcome.41 Interestingly, five neurospecialists stated that they would only stop investigations for SAH when the risk of missed diagnosis was reduced to zero. None of the ED consultants shared this view. ED clinicians' reluctance to resort to LP may therefore reflect the greater tolerance of risk in this group.

It is arguably inappropriate—for patients and health services—that a medical intervention should be determined entirely by the risk threshold of an individual doctor.41 However, it is equally undesirable for guidelines to impose an arbitrary risk threshold that is based on opinion rather than evidence. To be accepted by clinicians and patients, guidelines may need to accommodate a range of possible risk tolerance preferences. This may apply to a variety of clinical situations—involving different specialities and healthcare systems—where there is a fine balance between clinical risk and benefit.

Our findings suggest that the current guidelines do not closely reflect ED clinicians' risk tolerance for SAH investigation—despite the fact they usually manage patients presenting with the acute sudden onset headache. Incidentally, the Royal College of Emergency Medicine was not represented in the formulation of the current RCP Stroke guidelines (confirmed by personal correspondence).

Limitations of this study

The lack of an appropriate instrument to measure clinicians' risk tolerance necessitated the development of a novel questionnaire. Despite our best efforts, the responses indicated some shortcomings of this tool that limit the strength of our data. Seven respondents were known to have misinterpreted questions because the answers they provided were incompatible with the question asked. Although these responses could be excluded from the final analysis, other misrepresentative responses may still exist within the data set. The development and validation of an instrument to reliably measure risk tolerance was outside the scope of this study. Although we did not feel the lack of such a tool precluded investigation, our findings require cautious interpretation as a result.

A targeted approach to recruitment (by email, telephone or in person) was adopted to optimise participation. However, this may have introduced a degree of bias since not all respondents were contacted in exactly the same manner or at the same time. An online survey may have generated more responses and would have been easier to conduct at a national level but would not have overcome participation bias. We felt a targeted pilot survey was an appropriate starting point that could be used to inform future study.

The survey is limited by a poor response rate, which introduces the possibility of participation bias. Although response rates were similar between the two groups, different factors may have influenced each group. Response rates were not balanced across institutions (ranging between 17% and 42%) and as such, our results are skewed towards the NHS Trust that provided the most responses.

As with any survey of this type, hypothetical scenarios can never accurately reflect real-life practice. This problem is exaggerated when considering abstract concepts such as risk. We asked respondents to quantify their acceptable risk of missed diagnosis. This is a particularly challenging task—reflected in the fact that four consultants did not provide a response. Clinicians may have felt uncomfortable characterising their clinical judgement in statistical terms. However, this is a common scenario faced by clinicians in their daily practice when patients ask them to quantify the chance of complications from an invasive procedure, diagnostic test or surgery. Furthermore, incorporating relevant quantitative research findings into clinical decision-making is necessary to deliver evidence-based medicine (EBM). Therefore, although a simulation, the scenario we presented to clinicians is a realistic challenge similar to that commonly faced in the healthcare setting.

One respondent indicated that they would ask the patient to determine their own level of acceptable risk rather than assume this responsibility for themselves. It is possible that other participants shared a similar view and felt our question was incompatible with their own professional approach. This raises interesting questions about the doctor–patient relationship, the didactic nature of guidelines and the role of patients in guideline development. Clearly, clinical risks need to be acceptable to patients first and foremost and patient centredness has become a dominant paradigm in modern medicine.42 However, transferring responsibility to the patient can only lead to empowerment if patients are supported in the decision-making process and the best evidence is made easily accessible to them.

Patient participation is encouraged in guideline development43 on the assumption that active patient involvement will enhance the quality of guidelines. However, it has been argued that this mechanism is ill suited to achieving patient-centred care at the individual level. As described by van de Bovenkamp and Trappenburg,42 training and supporting patients to participate as full members in guideline development is a double-edged sword. Patients who have been adequately trained and supported become fellow academics and may no longer be able to contribute the experiential knowledge for which they were asked to participate in the first place. Patients who were not properly trained do contribute this experiential knowledge, but studies have shown that it is difficult to incorporate this in EBM guidelines.

Clinicians continue to play a key role in guiding patients through complex medical decisions. Although challenging, further study of clinicians' risk–benefit appraisals could improve our understanding of clinical behaviours and ensure that guidelines and doctors work together to deliver patient-centred care.

Knowledge of clinicians' risk tolerance could also inform diagnostic test development, given that the usefulness of a test is determined by more than its accuracy alone. For example, our findings suggest that some clinicians would prefer a non-invasive alternative to LP, despite a lower sensitivity. Conversely, a diagnostic test offering improved accuracy may fail to gain acceptance if other characteristics were less tolerable to doctors and patients. This is borne out by the growing use of non-invasive vascular imaging techniques in SAH diagnosis. Considerable advances in non-invasive vascular imaging methods such as CT angiography (CTA) and MR angiography (MRA) have led to their use in clinical practice. A CT/CTA diagnostic paradigm has been reported to have a sensitivity of 99% for aneurysmal SAH diagnosis and has been proposed as a substitute for LP.44 Opponents criticise this approach for moving from the diagnosis of haemorrhage to the detection of aneurysms, which may be coincidental.45 Nevertheless, given the advantages afforded by non-invasive tests, the use of CTA and MRA may need to be incorporated into future guidelines for the management of patients with suspected SAH.

Conclusion

This study provides a possible explanation for the observed variation in practice of SAH investigation. Opinions vary significantly between ED clinicians and neurospecialists with respect to the utility of LP and their tolerances for the risk of missed diagnosis. Omission of the LP from the diagnostic work-up may reflect scepticism about its utility rather than an unawareness of the risk of missed diagnosis.

Guidelines provide essential, expert evidence summaries and can marshal clinicians through a complex and ever-changing scientific literature. Nevertheless, guidelines, healthcare institutions, clinicians and patients may continue to contradict each other when they are informed by different tolerances of risk. Guidelines may be used more widely if they accommodate different preferences where appropriate.

Future study of risk tolerance in the clinical setting may enhance our understanding of a range of clinical behaviours, including referral patterns and diagnostic test use. Development of validated instruments to measure risk tolerance is needed, so that robust conclusions can be drawn from future research.