Assessment of acute stroke cerebral CT examinations by anaesthesiologists.

BACKGROUND AND
PURPOSE: It is essential to diagnose ischaemic stroke as soon as possible after symptom onset, so that thrombolytic treatment can be initiated as quickly as possible. This might be greatly facilitated if cerebral CT could be carried out in a pre-hospital setting. The aim of this study was to evaluate if anaesthesiologists, who in Norway provide pre-hospital medical care, could be trained to assess cerebral CT scans to exclude radiological contraindications for thrombolytic stroke treatment.
METHODS: Thirteen anaesthesiologists attended an 8-h course in acute stroke assessment, including a 2-h introduction to the neuroradiology of acute stroke. Each participant then assessed 12 non-contrast cerebral CT examinations of acute stroke patients with specific regard to radiological contraindications for thrombolytic therapy. Test results were compared with those of three experienced neuroradiologists. Inter-rater agreement between anaesthesiologists and neuroradiologists was calculated using Cohen's Kappa statistics. Robustness of the results was assessed using the non-parametric bootstrap.
RESULTS: Among the neuroradiologists, Kappa was 1 for detecting radiological contraindications for thrombolytic therapy. Twelve of the 13 anaesthesiologists showed good or excellent agreement (Kappa > 0.60) with the neuroradiologists. The anaesthesiologists spent a median time of 2 min and 18 s on each CT scan.
CONCLUSIONS: This study suggests that anaesthesiologists who are experienced in pre-hospital care may be quickly trained to assess cerebral CT examinations in acute stroke patients with regard to radiological contraindications for thrombolytic therapy.

The current study found that the anaesthesiologists not familiar with cerebral CT examinations can assess adequately CT scans in acute stroke patients after short‐term training. This suggests that thrombolytic therapy may be administered without delay if indicated in stroke.

Stroke affects approximately 15 million people worldwide each year and is the third leading cause of death in most western countries. It is also the major cause of adult disability leaving two‐thirds of stroke survivors with moderate to severe disability.1 In acute ischaemic stroke, intravenous thrombolysis may, if given early and preferably within 90 min from symptom onset, significantly reduces disability.2 At present, cerebral ischaemia cannot be differentiated from cerebral haemorrhage without the use of neuroradiology as treatment of a patient with acute cerebral haemorrhage with thrombolysis may be fatal.3 Intravenous thrombolysis for cerebral ischaemia must therefore currently be performed in hospital following an initial cerebral computer tomography (CT) or magnetic resonance imaging (MRI) examination.

Cerebral CT is the standard radiological examination of acute stroke patients and is used to exclude contraindications for intravenous thrombolytic therapy.4 The absolute CT contraindications are: intracranial haemorrhage, acute hypodensity involving one‐third or more of the middle cerebral artery (MCA) territory or a brain lesion with a high likelihood of haemorrhage (tumour, abscess, large vascular malformations). A CT scan is highly sensitive for excluding haemorrhage and other major contraindications, while relatively insensitive in detecting early signs of infarction corresponding to the initial cytotoxic oedema. Minor acute ischaemic changes include subtle or small areas of hypodensity, loss of grey–white distinction, or the presence of a hyperdense artery sign and are not a contraindication to thrombolytic treatment. However, when vasogenic oedema develops within the first few hours, a CT scan will show hypodense areas in the affected part of the brain. With no radiological or clinical contraindications, thrombolytic treatment with thrombolysis may be given up to 4.5 h after symptom onset.4

Time is the crucial factor in acute stroke management. Very few ischaemic stroke patients are at present treated within “the golden first 60 min” after symptom onset.5 A logical solution to this time delay is to establish the diagnosis of ischaemic stroke as close to symptom start as possible.6 This may lead to earlier and more effective thrombolytic therapy for a larger number of patients. The Phantom study demonstrated that pre‐hospital stroke diagnostics may reduce time to thrombolytic treatment without an increase in adverse events.7

Pre‐hospital delay in stroke is a crucial factor limiting patient access to thrombolytic therapy.8 Both patient delay (time from symptom onset to alarm) and transportation delay (response to alarm, transportation time and logistics) are time‐consuming factors detrimental for brain cells. Due to large distances and a sparsely populated countryside, Norway has established a national helicopter emergency medical service (HEMS), which is responsible for acute medical life support, transportation of critically ill and rescue operations. All crews of the 17 Norwegian HEMS bases include both an anaesthesiologist trained in pre‐hospital critical care and a HEMS paramedic. At present, the contribution to stroke patients by the Norwegian HEMS is limited to rapid transportation and stabilizing medical treatment. The ability to treat stroke patients would be greatly facilitated if anaesthesiologists could carry out pre‐hospital assessments including a cerebral CT examination. Previous studies on pre‐hospital acute stroke studies using CT have either been based on a stroke physician and a neuroradiologist in the stroke ambulance,9 or by using on‐line telemedicine assessments between the stroke ambulance and the admitting hospital.10

The aim of this study was to assess if anaesthesiologists from the Norwegian HEMS could safely carry out radiological stroke diagnostics especially with regard to the exclusion of contraindications for thrombolytic treatment in acute ischaemic stroke.

Methods

The anaesthesiologists were recruited from the national Norwegian HEMS and included in a study having an observational cohort design. Anaesthesiologists working in the Norwegian HEMS in the southern part of Norway were invited and 13 of these participated. The anaesthesiologists had median 9.5 years of experience in anaesthesiology. Eleven (85%) worked at present in pre‐hospital critical care and two in intensive care units. None of the anaesthesiologists had any specific training in stroke CT diagnostics. Their experience with stroke CT interpretation was limited to occasionally contact with stroke patients in their daily work.

Training in cerebral CT scan interpretation

All participants attended an 8‐h course in the clinical and neuroradiological assessment of acute stroke developed by stroke neurologists and neuroradiologists from the Departments of Neurology and Neuroradiology at Oslo University Hospital. The first part of the course focused on cerebrovascular pathophysiology, cerebral vascular anatomy and clinical assessments using the NIHSS (National Institute of Stroke Scale). The course was completed with a 2‐h teaching in neuroradiology. This included the basic principles of CT imaging and the interpretation of cerebral CT in acute stroke. CT scans were selected based upon common radiological findings in a cohort of patients with clinical stroke. CT examinations that showed contraindications to iv. thrombolysis were slightly overrepresented in the test. This selection was made to assess whether the anaesthesiologists were able to detect the most serious contraindications to thrombolytic therapy.

Cerebral CT scan assessment

After the course, the anaesthesiologists were presented with 12 CT examinations, taken within 4.5 h from symptom start from patients with hemispheric symptoms due to an acute stroke. The CT examinations were divided into four diagnostic categories by an experienced neuroradiologist: (1) intracranial haemorrhage, (2) acute hypodensity involving one‐third or more of the MCA territory, (3) brain lesion with high likelihood of haemorrhage (tumour, abscess, large vascular malformations) and (4) normal, subtle or small areas of hypodensity, loss of grey–white distinction or the presence of a hyperdense artery sign. No specific clinical or anamnestic information was given.

The cerebral CT examinations were presented as axial non‐enhanced images covering the whole brain (22–49 slides, slice thickness 3–5 mm). Five CT examinations demonstrated contraindications to thrombolysis (three intracranial bleedings, one acute cerebral infarction with a volume greater than one‐third of the MCA territory and one with findings consistent with a tumour). The remaining seven CT examinations were either normal or showed subtle or small areas of acute hypodensity, loss of grey–white distinction or the presence of a hyperdense artery sign. These seven patients were regarded eligible for intravenous thrombolysis based on CT examinations.

The test was handed to each participant on a memory stick with the 12 scans as separate power point files. The participants conducted the test without any external communication and had no reference work available. The CT scans presented to the participants were identical and all test results were given in a standardized study form (Fig. 1).

Figure 1

Flow chart of acute stroke cerebral CT examination test.

The anaesthesiologists could scroll between the slides, but window width (WW) and window level (WL) were set (50–65 WW, 40 WL). All scans presented in the study were anonymous, with no patient information, clinical history or physical findings. The anaesthesiologists where instructed to answer “conservative treatment” if they in a real setting would have consulted an in‐hospital specialist.

An experienced senior consultant in neuroradiology developed the educational module and the test. Two independent neuroradiologists from Oslo University Hospital did also complete the same CT assessment as the anaesthesiologists. The three neuroradiologists test results were then set as the reference when assessing the results of the anaesthesiologists.

Statistical methods

Data are presented as numbers (%). Agreement among the neuroradiologists and between the anaesthesiologists and the neuroradiologists was assessed using Cohen's Kappa for inter‐rater agreement.11, 12 The Kappa statistic can take values from 1 (indicating perfect agreement) through 0 (representing no agreement beyond what can be expected by chance), to −1(indicating perfect disagreement). When interpreting results, Kappa values < 0.2 represent negligible improvement over chance, 0.2–0.4 minimal, 0.4–0.6 fair, 0.6–0.8 good and > 0.8 excellent.13 Robustness of the results was assessed using the non‐parametric bootstrap.14

Results

Test results of the CT scan examinations are shown in Table 1 and 2. The anaesthesiologists completed the cerebral CT test with a median (range) total time spent of 25 (16–40) min, with the median (range) time spent on each scan being 2:18 (1:33–3:33) min.

Table 1

Agreement on acute CT scan contraindications to thrombolysis for neuroradiologists vs. anaesthesiologists

Neuroradiologists	Anaesthesiologists
	Thrombolysis	Contraindication	Total
Thrombolysis	83	8	91
Contraindication	4	61	65
Total	87	69	156

Table 2

Diagnostic diversity between the anaesthesiologists on acute stroke CT scan interpretation

CT scan	No significant pathology	Intracranial haemorrhage	Hypodensity > 1/3 MCA	Significant pathologya	Total
1	11	1	1	0
2	1	12	0	0
3	12	1	0	0
4	1	7	1	4
5	12	0	1	0
6	11	0	0	2
7	0	0	13	0
8	12	0	1	0
9	2	1	3	7
10	12	1	0	0
11	13	0	0	0
12	0	8	3	2
Total	87	31	23	15	156

Brain lesion with high likelihood of haemorrhage (brain tumour, abscess, vascular malformations).

Radiological contraindications of thrombolytic therapy

Inter‐rater agreement between the three neuroradiologists had a Kappa of 1 on classifying the radiological contraindications for thrombolytic treatment.

Inter‐rater agreement for the anaesthesiologists in finding radiological contraindications for thrombolytic therapy was generally high. Twelve (92%) of the 13 anaesthesiologists had a kappa > 0.6 agreement with the neuroradiologists, and 10 (77%) of the anaesthesiologists had a kappa > 0.8 (Fig. 2). The anaesthesiologists missed a total of 4 (2.6%) scans categorized by the neuroradiologists as a contraindication for thrombolysis (Table 1).

Figure 2

Radiological contraindications of thrombolysis in acute stroke.

Median Kappa in the data was 0.83. Performing robustness analysis indicates that this might be an underestimate of the true value, as the bootstrap median (95% CI) was 1 (0.57–1).

Diagnostic accuracy

The diagnostic diversity between the anaesthesiologists is shown in Table 2. When testing the ability to set a correct radiological diagnosis based on the CT examinations alone, agreement between the neuroradiologists had a Kappa of 0.92. In the group of anaesthesiologists, 54% had a kappa > 0.61 and 46% had a kappa > 0.4–0.61 (Fig. 3).

Figure 3

Diagnostic accuracy in cerebral CT interpretation.

Discussion

The aim of this study was to investigate if anaesthesiologists trained in pre‐hospital critical care could assess radiological contraindications for thrombolytic treatment in acute ischaemic stroke as a possible way to reduce pre‐hospital time delay from symptom onset to diagnostic and subsequently therapeutic decisions.

The results suggest that anaesthesiologists, after limited but specific training, are able with a high level of agreement compared with a panel of three neuroradiologists, to detect cerebral CT contraindications for thrombolytic stroke therapy.

Thrombolytic therapy is the only approved casual therapy for acute ischaemic stroke.15 However, the treatment effect of thrombolysis is highly time‐dependent, with number needed to treat (NNT) to achieve good outcome increasing from 4.5 patients up to 1.5 h after symptom onset to 14.1 patients after 3.0–4.5 h.16 Contraindications for thrombolytic therapy must therefore be assessed as quickly as possible after symptom onset. Pre‐hospital delay is the main reason why at present only a small number of patients receive thrombolytic treatment.17 Several studies have been carried out which address the time‐consuming delays in the chain of acute stroke treatment. Norwegian data from 2012 showed that median time from symptom onset to admission was 3.0 h (179 min; range 77–542 min).17 Decision delay defined as hesitation to contact medical services accounted for 55.1% of the delays. A Swedish study assessed if the clinical skills of ambulance nurses could help to shorten diagnostic delay in the assessment of acute stroke.18 The intension was to bypass the emergency physician's examination on arrival to hospital and let the ambulance nurse decide whether the patients should go directly to the CT scan or not. The study concluded that there was poor agreement (kappa = 0.22) between the nurses and the physicians, and that additional tools must be included in pre‐hospital decision‐making before the physician judgement can be by‐passed.

In clinical studies, where comparisons between groups are made with regard to “agreement”, the level of agreement can be presented as the overall per cent (%) agreement, but statistically this is considered inadequate.13 This does not show the prevalence of the findings in the subjects studied it also does not show how considerable agreement would be expected by chance alone, and it does not tell the extent to which the agreement improves by chance. Calculating kappa statistical measures allows the determination of the extent to which agreement exceeds that expected by chance.13 As the consultant neuroradiologist and the two independent neuroradiologists in our study attained a kappa of 1, indicating excellent agreement, we considered their test results as the “gold standard” for our further calculations. The overall agreement (kappa 92%) between anaesthesiologists and neuroradiologists in estimating radiological contraindications for thrombolytic therapy in acute ischaemic stroke in our study was good (kappa > 0.60).

The anaesthesiologists missed contraindications in only a very few scans (n = 4). However, as thrombolytic therapy may be very harmful to the patient if given when contraindicated, there must always be as few radiological misinterpretations as possible. In the Stroke DOC Telestroke Network (2013), overall agreement on radiological contraindications for thrombolytic therapy between stroke neurologists and radiologists was tested. The study concluded that vascular neurologists are able to effectively and immediately interpret cerebral CT scans sent by teleradiology.19 To minimize the number of radiological misinterpretations, the anaesthesiologists should in a real life setting always consult a radiologist or a stroke neurologist using teleradiology when in doubt about the interpretation. Misinterpretations by the anaesthesiologists in this study may, however, have been influenced by the lack of clinical information given. In a clinical setting, the anaesthesiologists will of course do a complete diagnostic work up including a neurological examination (NIHSS), history and laboratory tests before concluding on thrombolytic therapy or not.

To test diagnostic accuracy, all results of the cerebral CT examinations were categorized into four standardized radiological diagnoses. We found significantly less agreement in diagnostic accuracy between the anaesthesiologists and the neuroradiologists. Also among the neuroradiologists, there was a slight tendency of diagnostic disagreement. This variation in level of agreement in diagnostic decision‐making may show a common challenge of testing reliability in clinical medicine. It is often seen that diagnostic testing requiring human judgement seems to fall in the fair or good range.19

In the setting of in‐hospital stroke care, other studies regarding the interpretation of cerebral CT in acute stroke have showed great variations in diagnostic agreement among physicians treating stroke patients. In a US trial from 1998, emergency physicians, neurologists and radiologists interpreted CT scans showing acute cerebral infarctions, intracerebral haemorrhages, intracerebral calcifications, old cerebral infarctions and normal findings.20 The study showed an overall percept agreement between the physicians and a “gold standard” of 77% (95% confidence interval, 74–80%) in correct score when identifying intracranial haemorrhage. No kappa statistics were, however, performed. Emergency physicians had an accuracy of 67% of the presented scans, neurologists and radiologists could detect the correct diagnose in 83% of the cases.20 This study concluded that sensitivity of detecting haemorrhage was fairly good for all three groups of specialists. The emergency physicians in the study did not achieve a level of sensitivity sufficient to permit safe selection of candidates to thrombolytic therapy. As a consequence of this study, the investigators of the ECASS II study trained the investigating physicians for 4 h in cerebral CT assessment of acute stroke and concluded that this training significantly improved the number of correct interpretations (P < 0.0001).21 Considering the results of the ECASS II study and the experience from our own study, some more specific radiological training should also be implemented for the Norwegian HEMS anaesthesiologists before entering a clinical study phase. A general accepted experience in pre‐hospital emergency medicine is that diagnostic specificity will seldom reach fully up to the in‐hospital quality level. However, a slightly reduced diagnostic specificity may be fully counterbalanced by a higher treatment efficacy because of the highly time‐dependent treatment effect for most medical emergencies.

Our study has some limitations due to its theoretical design. The series of CT scans presented in the test was handpicked with a planned slightly overrepresentation of pathological findings. The CT scans were presented as a power point file, thereby diminishing the possibility to adjust the window frame and image resolution, which may limit the ability to detect subtle findings as early infarction changes. Despite this, the neuroradiologists showed a 100% overall agreement in finding the contraindications of thrombolytic therapy.

Conclusions

Introducing acute ischaemic stroke diagnostics in the pre‐hospital field may allow more patients to obtain earlier and more efficient thrombolytic therapy. Our concept of teaching HEMS anaesthesiologists to interpret pre‐hospital CT scans may reduce technical and communicative problems related to teleradiology and dependency upon in‐hospital medical specialists. It may also diminish the need to bring in‐hospital specialists such as neurologists into the pre‐hospital room. A confirmation of these findings in future clinical studies would suggest the possibility of a new concept for treating acute ischaemic stroke patients in the pre‐hospital setting.