Comparison of a basic lung scanning protocol against formally reported chest x-ray in the diagnosis of pulmonary oedema.

Introduction: Brief lung scan protocols have been recommended as a useful adjunct to identify pulmonary oedema in the breathless elderly patient. Some papers quote diagnostic accuracies above that of chest x-ray. Method: We recruited a prospective convenience sample of patients over sixty years of age reporting any breathlessness on presentation to the emergency department. Those who received both bedside lung scan and chest x-ray later had their case notes audited by an expert cardiologist for the cause of their breathlessness at presentation. Admission diagnosis was also extracted.
Results: 204 comparative data sets were collected. Compared with cardiologist chart review, delayed expert radiology report had a diagnostic accuracy of 92.2% (95%CI 87.6 to 95.1). Bedside interpretation of lung scan protocol had a diagnostic accuracy of 85.3% (95%CI 79.8 to 89.5). The difference of 6.9% between the two accuracy measures was significant (95%CI 0.69 to 13.1). Admission diagnosis accuracy, which encompasses inexpert x-ray interpretation was 70.2%(95%CI 62.9 to 76.6), significantly less than either lung scan or expert chest x-ray report.
Conclusion: For identifying heart failure in breathless patients, urgent chest x-ray with delayed formal report has not been shown to be redundant. Basic lung scan protocols should not yet replace chest x-ray, but may be more reliable in the interim than inexpert clinician interpretation of chest x-rays.

Introduction

Breathlessness in the older population is a very sensitive predictor of heart failure but unfortunately non specific. , For this reason, many people who present with breathlessness require more complex investigation to determine its cause. Specific clinical and radiographic signs of heart failure are rare – so patients undergo a barrage of tests with mediocre precision. This is an expensive business. In 2008–09 in the USA, a presentation of dyspnoea cost an average of US$6958 per patient with any history of acute coronary syndrome. Subsequent admission costs averaged US$20,693 per patient.

One test that is coming under increased scrutiny is chest x‐ray.

Chest x‐ray is a recommended standard in the investigation of dyspnoea. – , . However in acute heart failure, chest x‐ray has a tainted reputation. , , When used to investigate breathless patients, it may be interpreted as negative in up to 20% of patients with heart failure. , Alternatively, when non‐radiologist x‐ray interpretation is added to clinical assessment, the diagnostic formulation is incorrect in up to 25% of breathlessness presentations. , ,

The cost of a chest x‐ray in Australia is approximated conservatively by the Medicare schedule fee of $47.15 while an after hours hospital radiographer costs at least $188.88 for the first call back. The question arises, is this financial outlay warranted for a test of mediocre precision?

Not surprisingly, clinicians are examining low cost modalities that improve diagnostic precision, in particular B‐type natriuretic peptide , , , and bedside lung ultrasound. , – The first lung ultrasound studies (LUS) were conducted in intensive care on patients with pulmonary oedema of various aetiologies, and aimed to identify radiographic changes labelled ‘alveolar interstitial syndrome’. – Lung ultrasound for interstitial syndrome produced sensitivities and specificities , , , , that suggested it may surpass chest x‐ray. , , – Some research groups now recommend that a negative lung scan protocol be used as a screening tool to reduce redundant chest x‐ray in breathless patients. ,

Simple economics will push clinicians to use lung sonography instead of chest x‐ray. More research is warranted to confirm that this substitution is wise, safe and justified.

This paper approaches the question from another angle. It asks ‘has urgent chest x‐ray added to the sum of clinical knowledge?’ It will be up to individual clinical managers to decide if the additional information gleaned is cost effective.

Method

Participants

This is a prospective observational study of a convenience sample of patients aged 60 years and over, presenting to an urban district Emergency Department (ED) with any complaint that included shortness of breath.

To recruit, the senior ED coordinating the shift monitored the descriptive fields on the computer triage screen, looking for the words ‘breathlessness’, ‘dyspnoea’ or acronyms such as ‘SOB’ (short of breath) or ‘WOB’ (work of breathing).

Patients were not eligible for the trial if LUS interfered with active resuscitation or if symptoms were related to trauma. Patients were excluded during the trial if a chest x‐ray was not performed within one hour of LUS, or within two hours providing that no active fluid management occurred in the intervening period. Post recruitment exclusions occurred due to missing data.

As far as possible, an MO not directly involved in managing the patient was delegated to perform lung scans soon after patient arrival, and the findings were recorded separately to the clinical record. In all other respects, patient management continued unchanged. As an ethical requirement, the scanning MO did not inform the treating MO of findings unless an incidental finding required emergent management.

Recruiting occurred from 08.00 to 23.30 hours when both radiography and a LUS trained medical officer (MO) were available. The project commenced at Ipswich Hospital Emergency Department in March 2011 and concluded February 2012. Further details of demographics and sample characteristics have been published.

Ethical considerations

Waiver of consent was obtained from the Queensland Civil and Administrative Tribunal before commencement of study, and ethics approval was granted by the West Moreton Health Service District Human Research Ethics Committee.

Sample size was initially calculated at 90 cases using Lichtenstein's 20% improvement (alpha error of 5% and power of 80%) however interim analysis showed a smaller pulmonary oedema group so, with permission, the sample size was expanded to 220.

Test methods

For each form of test or diagnosis, the results were categorised by the way the result would influence initial fluid management. This required three groups. A test or diagnosis that suggested pulmonary oedema or congestive cardiac failure was categorised as ‘wet’. Any other result was categorised as ‘dry’. Results that gave no guidance to fluid management were regarded as indeterminate. The LUS result and the auditor result were prospectively defined, to minimise indeterminate results. ED diagnosis codes and chest x‐ray reports were categorised retrospectively, with unexpected diagnoses categorised on a case‐by‐case basis.

Reference test

The reference test for the cause of breathlessness was blinded post‐discharge chart audit. A specialist cardiologist with experience in emergency assessment and echocardiography (AT) considered all notes, reports, inpatient tests and images with the exception of the LUS result. The auditor was asked to report an opinion as to the cause of breathlessness at the ED presentation, specifically whether the patient would have benefitted from fluid‐reducing therapy (‘wet’) or not (‘dry’). Patients with combined cardiac and respiratory pathology, with no indicator of primary cause of breathlessness throughout the admission were labelled as indeterminate. This standard is the one used in recent literature – , , as it includes but is not limited by radiologist x‐ray report. Alternative gold standards were not available (immediate echocardiography) or not clinically warranted due to radiation dose (CT chest).

Lung ultrasound scan

The index test was an eight‐view lung scan (LUS) collected on a GE Logic‐e portable ultrasound (Shanghai, China), using a 2–5MHz curved probe with a low dynamic range, and the focus at the pleural line. Harmonics and crossbeam were switched off to maximise the B‐line artefact. B lines are an hyperechoic artefact extending from pleura to the base of the screen, reminiscent of a searchlight. The sonographic artefact was originally named a ‘comet‐tail’ artefact, and indicated an area that would show either interstitial markings or ‘ground glass’ changes on chest x‐ray. These were first described by Lichtenstein , using chest x‐ray as a reference, and subsequently confirmed with region by region comparison using CT scanning as the reference standard.

The patient was scanned in their position of comfort, usually semi‐reclined. We used a modification of the Volpicelli, et al scanning protocol, since ratified by the international consensus group. This requires 4 pictures each side, one anterosuperior then one anteroinferior in the mid clavicular line (sagittal plane), one anterior axillary (oblique) and one laterobasal (coronal) view (Figure 1).

Figure 1

In older patients, lung region 2 frequently appears more rostral than illustrated in the international consensus guidelines modified from Volpicelli et al. This may be in part due to longitudinal orientation of a curved probe.

To replicate the early learning curve, sonologists used only the dichotomous version of the basic LUS protocol. The patient was considered to have a positive scan if two or more regions on each side demonstrated three or more B lines simultaneously per view. Pulmonary oedema is inferred from the widespread distribution of the B lines. For the purpose of our study, we considered a positive scan result to imply pulmonary oedema rather than radiographic ‘interstitial syndrome’ , as have other studies investigating the more clinically relevant form of the protocol. – , This accepts a small false positive rate caused by diffuse fibrosing or inflammatory lung conditions. We recorded a bedside interpretation of the protocol, and then a blinded interpretation of the saved scans. The blinded re‐interpretation was performed by a radiology registrar with prior Master of Applied Science (Medical Ultrasound) and echocardiology credentialing (GS).

One experienced sonologist and eleven novice sonologists performed the scans. Novice sonologists were required to attend any general ultrasound course, followed by a specific four‐hour lung scanning workshop, and then collect ten proctored scans. Inter‐rater agreement was examined and reported.

ED Diagnosis (Clinical acumen)

The second data sampling point, the Emergency Department Information System (EDIS) diagnosis code, represents ‘clinical acumen’. In our hospital, the ED diagnosis is the diagnosis of the admitting doctor, and incorporates initial tests and the clinician assessment of the chest x‐ray. The code is entered either by the treating ED doctor, or the shift coordinating senior clinical nurse reading the diagnostic formulation at the conclusion of the admission notes. Potential diagnoses were not constrained, but staff were kept aware of the importance of correct coding to the project.

Diagnosis codes were divided into ‘wet’ (heart failure, left ventricular failure, ‘CCF’ or pulmonary oedema) and ‘dry’ (asthma, COPD, chest infection, pulmonary embolism, pulmonary fibrosis, sepsis, anaemia, anxiety, social and other). Indeterminate codes were acute coronary syndrome, myocardial infarction, any arrhythmia and pleural effusion. These cases were excluded due to difficulty in categorisation. These conditions may or may not be associated with pulmonary oedema, but this complication is not reliably entered on EDIS. Secondly, when these conditions are considered the primary diagnosis, the physician focuses on condition‐specific treatments before fluid management.

Formal Chest X‐ray Report

The third piece of information was the formal chest x‐ray report. Chest x‐rays were reported as per normal practice in the hospital, with some additions.

An historic scoring system was also displayed in the radiology reporting room for use as a reference and could be used as a reference for difficult x‐rays. Radiologists were reminded regularly to express an opinion as to the presence or absence of pulmonary oedema.

Reports were classified as ‘wet’ if the radiologist reported that any degree of pulmonary oedema or congestive cardiac failure was present or likely to be present. Cardiac failure could be diagnosed by the presence of pleural effusion and a change in the heart size, without requiring any increase in peribronchial markings, upper lobe vascular redistribution or interstitial fluid.

Chest x‐rays were classified as ‘dry’ if an alternative diagnosis was reported, such diagnoses being asthma, chronic obstructive pulmonary disease, focal infection, hyperinflation, pneumothorax, fibrosing lung disease and ‘normal’.

GS and a senior consultant reviewed chest x‐rays in two circumstances. Firstly, the chest x‐ray was reviewed for a probable cause where an abnormality was reported (for example ‘fluid in the fissure’, or ‘pleural effusion’), but no opinion given for the causative agent. Secondly, the films were reviewed if the initial report suggested dual pathologies that required opposing fluid strategies (for example ‘left lower lobe pneumonia and pulmonary oedema’, or ‘hyperinflation with a large heart and bilateral pleural effusions’). If a primary cause could not be discerned radiologically from the subsequent evolution of inpatient x‐rays, the ED chest x‐ray was labelled ‘indeterminate’.

To allow comparison with prior studies, we extracted age, sex, length of stay and triage category. For post hoc sub group analysis, we recorded incidental findings on LUS, the complete chest x‐ray report, and the actual diagnosis made by the auditor.

Statistical methods

Summary data were analysed using Excel (14.2.5 Office for Mac 2011, Microsoft, Seattle, USA). Sensitivity, specificity, diagnostic accuracy, difference in proportions and predictive values were obtained with the VassarStat online calculator with Newcombe's method for the 95% confidence intervals.

Diagnostic accuracy was calculated as per previous studies, by cross tabulating the index with the reference test, summating the concordant cells then dividing by the sum of all cells, including the indeterminate results in the denominator. , ,

Results

The participants (n = 204) had had a median age of 76 years (IQR 69 to 83 years), and 46% (n = 93) were female. Although many presentations were multifactorial, the principal audit diagnosis was asthma/COPD in 30%, heart failure in 20%, chest infection in 19%, other cardiac in 13% and miscellaneous in 18%.

Of 230 patients recruited, 204 complete data sets were obtained. Thirteen patients were excluded initially. Six did not fulfil inclusion criteria, three had scans reported at the bedside but not saved, two had inadequate saved scans and two were not scanned at all (reason unknown). There were thirteen post‐enrolment exclusions for missing audits (9) and indeterminate reference tests (4). As this was a convenience sample for pilot study, we did not keep a screening log.

Estimates

A single cardiologist auditor was used for the reference test without re‐audit. For the index tests, multiple practitioners acquired the LUS and read the chest x‐rays as we aimed to identify robust test characteristics in a situation as close to clinical practice as possible. The agreement between bedside and blinded interpretation of LUS scans has been reported previously.

Thirty‐six EDIS results were excluded, as the inability to categorise them was due to study design, not the tool itself. For the same reason, four indeterminate reference tests were among the original 26 exclusions. Indeterminate LUS and chest x‐ray results were not excluded because they represent a small resource waste that detracts from the value of either test.

Test results

Bedside reporting of the LUS protocol had a sensitivity of 65% (95%CI 48.3 to 78.9) and a specificity of 91.9% (95%CI 86.3 to 95.5) with an overall diagnostic accuracy of 85.3%(95%CI 79.8 to 89.5). Three cases were labelled indeterminate as we were not aware at the time of the method for quantifying confluent B lines.

The EDIS diagnosis, representing clinical acumen of the admitting doctor was compared with the reference test, with a sensitivity of 71.4% (95%CI 51.1 to 86), specificity of 70% (95%CI 61.6 to 77.3) and diagnostic accuracy of 70.2% (95%CI 62.9 to 76.6%).

Delayed formal chest x‐ray report had a sensitivity of 76.9% (95%CI 60.3 to 88.3) a specificity of 97.5% (95%CI 93.4 to 99.2) and a diagnostic accuracy of 92.2% (95%CI 87.6 to 95.1). Four chest x‐rays remained ‘indeterminate’ after review. Two of these had significant dual pathology that would push fluid management in opposite directions, and two had minimal pathology without indication of the cause. These four chest x‐rays have been designated as ‘indeterminate’, because the report could not be used to guide fluid management.

When the complete chest x‐ray reports were reviewed, 43 out of 204 reported a localised pathology other than pleural effusion. Of the 43 localised findings on chest x‐ray, the auditor confirmed 20 as having a primary diagnosis of pneumonia. Conversely, the auditor diagnosed 41 cases of acute lower respiratory tract infection, of which only 20 were recognised as such in the chest x‐ray. These figures are given only as an indication, as pneumonia was not targeted by this project.

Figure 2a

This is an example of a strongly positive eight view LUS protocol, performed prior to the chest x‐ray depicted in Figure 2b.

Discussion

Urgent, unreported chest x‐ray of breathless patients added diagnostic information, but this information did not benefit the emergency physician. There is not yet evidence to justify replacing urgent chest x‐ray with a dichotomous lung ultrasound protocol, but there is evidence to support the benefit of added lung ultrasound.

Table 1 summarises the evidence available in the literature of estimates of the diagnostic accuracies for clinical and radiological data available at relevant time points to this study. Studies on the accuracy of chest x‐ray in heart failure have pointed out the significant differences between bedside clinician interpretation, routine radiologist interpretation and forewarned expert review.

Table 1

Summary of the accuracy of information available from tests during the breathless patient's episode of care.

Patient Journey	Test Result Available	Calculated Diagnostic Accuracy	Study
In ED	LUS	85% 84% (28 scan protocol)	This study Gargani (16)‡
Within 4 hours	History, Exam, ECG, Blood tests, CXR (non radiologist report)	74% (‘80% sure‘)	McCullough (7)
		70.2%	This study
Same day CXR	CXR report
	Radiologist/Cardiologist	61%‡	Fonseca† (3)
	Radiologist	69%	Mueller Lenke § (9)
Days later	Forewarned CXR report With specific tool	78–79%‡ (‘probable’)	Studler § (2)
Several days later	Delayed CXR report with series.	92%	This study §

‡ Amalgamated from raw data given in text of article

† Reference standard independent from CXR report

§ Potential of CXR report to influence reference standard.

Our study suggests a further small improvement conferred by delayed expert radiology report that includes retrospective comparisons.

In a systematic review, Wang calculated the accuracy of clinical acumen (‘gestalt’‐including clinician x‐ray review) as having a positive likelihood ratio of 4.4 and a negative likelihood ratio of 0.45. These figures are supported by the findings of the Breathing Not Properly multinational study, where bedside clinician judgement ‘80% certain’ using chest x‐ray, had a sensitivity of 49%, a specificity of 96% and an overall diagnostic accuracy of 74% for identifying the presence or absence of heart failure. Our own ‘clinical acumen’ results were commensurate with these figures.

In the EPICA study, blinded cardiologist and radiologist reporting of chest x‐rays for possible heart failure demonstrated a diagnostic accuracy of 61% (recalculated from the raw figures). In the BASEL study, immediate routine radiologist reports had a sensitivity of 53.3% and a specificity of 86.3% with an overall diagnostic accuracy of 68.8% for the diagnosis of heart failure.

When experts reviewed the BASEL chest x‐rays, with forewarning, reporting ‘probable heart failure’ using a specific tool, sensitivity and specificity improved to give an overall test accuracy with an area under ROC between 0.855 and 0.857, although for the data points tabulated, a diagnostic accuracies of 79.7% and 78.2% can be inferred, the difference due to patient positioning. These results pertain to groups with a higher rate of heart failure (51%). Finally, our delayed radiologist reported films had the highest diagnostic accuracy, at 92.2%. There is both a reason for this, and one potential bias.

In our small hospital, reporting of emergency department chest x‐ray is often delayed by several days. It is the prerogative of the radiologists to examine subsequent imaging for changes, before reporting on the original presenting chest x‐ray. In these cases their precision is honed by information not available at the time of image capture. The situation is akin to interval collection of serological biomarkers, the most important information derived from the change recorded from the baseline test.

Chest x‐ray conferred another benefit above the dichotomous LUS protocol. Localised acute disease was demonstrated in around 10% of x‐rays. This was incidental to our study aim but must be counted in favour of chest x‐ray. However, the accuracy of chest x‐ray in the diagnosis of pneumonia may be as low as 65%, when compared against CT. Until sonologists can use LUS to identify local disease, chest x‐ray continues to add value, but further LUS techniques may rapidly overtake chest x‐ray in the diagnosis of pneumonia. – A steep learning curve has been demonstrated for identification of pneumonia using LUS. ,

When deciding if lung ultrasound should replace chest x‐ray, the decision facing health managers is whether the estimated 7% improvement in diagnostic accuracy justifies a $50 test, or a $189 call in fee. Unfortunately, the value added by our chest x‐ray reporting does not benefit the emergency department management of patients, although it can inform inpatient teams and subsequent presentations.

Limitations

This study is biased to favour chest x‐ray accuracy in several ways. Firstly, our recruiting did not extend overnight, potentially missing the higher acuity patients with ‘flash’ pulmonary oedema. It is in these patients that chest x‐ray may lose sensitivity. Figures 2a and b give an example of a strongly positive LUS scan coincident with a ‘clear’ chest x‐ray.

Figure 2b

Chest x‐ray – anteroposterior view of patient imaged in Reported as ‘lungs appear essentially clear. Heart may be enlarged.

Secondly, the auditor was blinded to LUS but not to the chest x‐ray report, and only reviewed films in controversial cases. This incorporation bias will improve the apparent diagnostic accuracy of chest x‐ray. In the absence of more informative tests (CT and echocardiography), we chose not to withhold this important information from the auditor. Although our reference test may have been influenced by the radiologist report, it was only one factor in the auditor's diagnostic synthesis. Both the BASEL , and the Breathing Not Properly , studies appear unblinded in this same respect, meaning that our estimates are comparable and we assume the improvement can be attributed to the paired sample effect.

A second limitation is the use of existing hospital information systems for data collection, rather than using a prospectively prepared tool. We attempted to compensate for this by regular reminders to medical and senior nursing staff of the importance of an accurate EDIS diagnosis to the research question. Furthermore, we counted only the most distinct of the diagnosis codes. We justify our results by pointing to similar ‘clinical acumen’ rates across larger studies in different continents. , ,

Future research

Diagnostic accuracy studies compare investigations. To truly validate LUS as a useful test, further research needs to demonstrate that a LUS incorporated into practice results in better patient outcomes, perhaps by measuring length of inpatient and emergency room stay. Our unpublished linear regression of these outcomes suggests that a very large study will be required to show any significant difference in length of stay in those diagnosed correctly versus those diagnosed incorrectly.

Several modifications could improve the precision of this study. We strongly recommend echocardiography at presentation to inform the auditor providing the reference standard, and an ED diagnosis recorded and prospectively categorised by the treating doctor rather than extracted from the hospital information system.

Blinding the auditor to formal chest x‐ray report will remove the interdependence of the tests, but will weaken the auditor's data by removing an expert opinion. We would not recommend this separation until immediate echocardiography and CT chest scanning are available to the auditor.

The potential of lung scanning to identify pneumonia as well as heart failure may soon raise its comparability to that of formal chest x‐ray. This valuable study will be possible when more sonologists have acquired experience in both pulmonary oedema and pneumonia recognition. The difficulty is to balance the complexity of the protocols against the skills of the sonologists.

Conclusion

This study recommends that emergency physicians should continue to order urgent chest x‐rays for breathless older patients, budget permitting. However, physicians should commence fluid management based on their LUS findings. When practitioners gain confidence and experience in lung scanning, the role of chest x‐ray must be re‐examined.