Diagnostic accuracy of basic lung ultrasound in breathless patients over 60 years of age; stressing the protocol.

Introduction: Emergency department differentiation of pulmonary oedema from chronic obstructive airways disease causing acute breathlessness is inaccurate 25% of the time despite clinical acumen, clinician-reported chest x-ray and ECG. This research investigates whether a basic lung ultrasound protocol (LUS) could improve identification of pulmonary oedema in breathless elderly patients. Method: Researchers prospectively sampled patients over 60 years, describing any breathlessness on presentation to a suburban emergency department. LUS studies were acquired by experienced or novice sonologists, interpreted by a blinded reviewer and compared with cardiologist chart audit for diagnosis at admission (gold standard). The admitting doctor's diagnosis, blinded to LUS, was compared with the chart audit result.
Results: 204 LUS were collected, 145 by experienced sonologist and 59 by inexperienced. Diagnostic accuracy compared to cardiologist audit was 86.2% (95% CI 80.9 to 90.3), significantly higher than 70.2%, diagnostic accuracy for admission diagnosis, difference in proportion of 16% (95%CI 7.7 to 24.4%).
Conclusion: A simple lung scanning protocol can help exclude pulmonary oedema in any breathless elderly patient.

Introduction

This research was instigated to see how an internationally recognised lung ultrasound protocol (LUS) performs in the differentiation of pulmonary oedema from other causes of breathlessness in elderly patients presenting to the emergency department. This protocol has been tested in exclusive groups with high rates of pulmonary oedema – but has yet to be challenged by a widened spectrum of disease. If specificity remains high it can guide diuresis and non‐invasive ventilation in the acutely ill, and if sensitivity is better than current measures it may reduce unnecessary investigation or inappropriate medical therapy in the elderly.

In current practice, signs, symptoms, bedside tests and chest x‐rays are combined in the investigation of dyspnoea, and diagnosis is the art of evaluating interdependent likelihood ratios. – One in four elderly patients who present with breathlessness may be misdiagnosed , and possibly mismanaged in the emergency department. Rapid reliable tests evoke interest as pressure grows from increased geriatric presentations, while National Emergency Access Targets reduce time available for investigation. However, thorough understanding of test characteristics should precede intelligent implementation of any investigation.

LUS can be regarded as a ‘FAST’ equivalent for the lungs, identifying fluid and distribution, but not nature or chronicity. Controversy arises between those who seek a test that clearly separates the target condition from background disease, , and pragmatists who accept some imprecision providing that there is improvement on current practice. , ,

It is time to validate LUS under conditions replicating clinical practice.

It is predicted that the sensitivity of the recommended protocol will fall as the inclusion criterion widens. Specificity is likely to reduce when the target condition changes from radiographically defined ‘interstitial syndrome’ to the more clinically relevant ‘pulmonary oedema.’ , – , This study challenges the protocol and generates robust test characteristics by using practitioners with varying skill, asking the most relevant question of an inclusive population.

Methods

Participants

This is a prospective observational study of a convenience sample of patients aged 60 years and over, presenting to an urban district Emergency Department (ED) with any complaint that included shortness of breath. Patients were excluded prospectively if LUS interfered with active resuscitation or if symptoms related to trauma. Patients were excluded retrospectively if a chest x‐ray was not performed within one hour of LUS, or within two hours providing that no fluid bolus, diuretic or non invasive ventilation was instituted in the intervening period. Recruiting occurred from 08.00 to 2330hours when a LUS trained medical officer (MO) was available. The project commenced at Ipswich Emergency Department in March 2011 and concluded February 2012.

Senior ED clinicians monitored the descriptive screens on the computer triage field, during the study period looking for the words ‘breathlessness’, ‘dyspnoea’ or acronyms such as ‘SOB’ (short of breath) or ‘WOB’ (work of breathing). As far as possible, an MO not directly involved in managing the patient was delegated to perform lung scans soon after patient arrival, and the findings were recorded separately to the clinical record. As an ethical requirement, the scanning MO did not inform the treating MO of findings unless a significant incidental finding required emergent management.

Ethical considerations

Waiver of consent was obtained from the Queensland Civil and Administrative Tribunal before commencement of study, and ethics approval was granted by the West Moreton Health Service District Human Research Ethics Committee.

Test methods

The reference test for the diagnosis at presentation was blinded post‐discharge chart audit. A specialist cardiologist with experience in emergency assessment and echocardiography (AT) considered all notes, reports, inpatient tests and images with the exception of the LUS result. The auditor was asked to report an opinion as to the cause of breathlessness at the ED presentation, specifically whether the patient would have benefitted from fluid‐reducing therapy (‘wet’) or not (‘dry’). Patients with combined cardiac and respiratory pathology, with no indicator of primary cause at presentation were labelled as indeterminate. This standard is the one used in recent literature , , , as it includes but is not limited by radiologist x‐ray report. Alternative gold standards were not available (immediate echocardiography) or not clinically warranted due to radiation dose (CT chest).

The index LUS test was an eight‐view lung scan collected on a GE Logic‐e portable ultrasound (China), using a 2–5MHz curved probe with a low dynamic range, and the focus at the pleural line. Harmonics and crossbeam were switched off to maximise the B‐line artefact. The 2012 consensus paper defines B lines as “discrete laser‐like vertical hyperechoic reverberation artifacts that arise from the pleural line (previously described as ”comet tails';), extend to the bottom of the screen without fading, and move synchronously with lung sliding“.

The protocol is used in recent studies, , – with minor adaptions to suit the resources. Probe placement on the chest wall is illustrated in Figure 1 in the next article.

Figure 1

Comparison of triage acuity at presentation in patients with and without target condition.

Sonologists were allowed to save a cardiac, IVC view and comments for piloting of future work. These were not made available to the reviewer or auditor.

Scan sets were collected by an experienced physician sonologist (KB) or one of eleven novice physician sonologists. Details of qualifications, teaching and agreement are have been described.

De‐identified scans were saved electronically on a portable data device without any other patient details. The device was given to single blinded reviewer (GS) who interpreted them as ‘wet’, ‘dry’ or ‘indeterminate’. The reviewer is a registrar training in radiology, with a Master of Applied Science (Medical Ultrasound) and echocardiography credentialing.

Ultrasound scan sets were reported as ‘wet’ if there were three or more simultaneous B lines present in two or more zones on each side of the chest. , The reviewer was bound to the dichotomous form of the protocol, and told to disregard findings that suggested other causes. In this simplest form, the protocol considers all B‐lines equal, and all bilateral positive scans to represent pulmonary oedema. Rare diffuse fibrosing conditions or pneumonitis were considered to be unavoidable false positives. A view with confluent B lines (‘white’ or ‘shining’ lung) was considered wet.

To imitate clinical practice, we predominantly saved stills rather than cine loops. The hospital imaging system does not store ultrasound loops.

To allow comparison with prior studies, we extracted age, sex, length of stay, triage category, delayed expert radiology report and the ED diagnosis code for the patient presentation from the hospital information systems. The ED diagnosis is the diagnosis of the admitting doctor, either registrar or consultant, and incorporates initial tests and chest x‐ray. Diagnosis codes were divided into ‘wet’ (heart failure, left ventricular failure, ‘CCF’ or pulmonary oedema), ‘dry’ (asthma, COPD, chest infection, pulmonary embolism, pulmonary fibrosis, sepsis, anaemia, anxiety, social and other) and ‘indeterminate’.

Indeterminate codes were acute coronary syndrome, myocardial infarction, any arrhythmia and pleural effusion. These indeterminate cases were excluded due to inability to categorise. Such conditions may or may not be associated with pulmonary oedema, but additional diagnoses are not reliably entered on EDIS. Secondly, when these conditions are considered the primary diagnosis, the physician focuses on condition‐specific treatments before fluid management.

Statistical methods

Summary data were analysed using Excel (14.2.5 Word for Mac 2011, Microsoft, Seattle, USA). Sensitivity, specificity, diagnostic accuracy, difference in proportions and predictive values were obtained with the VassarStat online calculator with Newcombe's method for the 95% confidence intervals.

Results

The participants (n = 204) had had a median age of 76 years (IQR 69 to 83 years), and 46% (n = 93) were female. Symptom severity is implied by the spread of triage category at presentation (Figure 1). The auditor diagnosed some degree of pulmonary oedema in 20% of patients, and these presented with a significantly higher triage acuity than patients without pulmonary oedema (relative risk 2.267, exact P = 0.002).

Of 230 eligible patients 13 were excluded initially. Six were incorrectly recruited, three had scans reported at the bedside but not saved, two had inadequate saved scans and two were not scanned at all (reason unknown). There were thirteen post‐enrolment exclusions for missing audits (9) and indeterminate reference tests (4). Recruitment is illustrated using the recommended Standards for Reporting of Diagnostic Accuracy flow diagram (Figure 2). None of the 26 exclusions was included in this statistical analysis, although the indeterminate reference test results are illustrated in the cross tabulations.

Figure 2

Recruitment flow diagram.

Within the subgroup of ED diagnosis codes, 36 further data sets were excluded because there was insufficient information to indicate presence or absence of pulmonary oedema. The ED diagnosis calculations were made with a group of 168 data sets.

Test results

Although many presentations were multifactorial, the principal diagnosis was asthma/COPD in 30%, heart failure in 20%, chest infection in 19%, other cardiac in 13% and miscellaneous in 18%.

Cross tabulation of the index test against the reference shows a sensitivity of 72.5% (CI 55.9 to 84.9%)and a specificity of 90.2% (CI 84.3 to 94.1%) for identifying cardiac pulmonary oedema, with an overall diagnostic accuracy of 86.2% (95% CI 80.9 to 90.3%) for identifying the presence or absence of cardiac pulmonary oedema (Table 1). Positive predictive value was 64.4% (95%CI 48.7 to 77.7%) and negative predictive value was 93.0% (95%CI 87.6 to 96.3%).

Table 1

Cross tabulation of blinded LUS interpretation versus cardiologist chart audit.

	Cardiologist	Chart Audit	Diagnosis
	‘wet’ on audit	Indeterminate audit	‘dry’ on audit	TOTAL
‘wet’ on LUS	29	2	16	47
indeterminate	0	0	1	1
‘dry’ on LUS	11	2	147	160
TOTAL	40	4	164	208

The admitting doctor diagnosis was compared with the reference test (Table 2), and shows a sensitivity of 71.4% (CI 51.1 to 86%), specificity of 70% (CI 61.6 to 77.3%) for identifying pulmonary oedema, congestive cardiac failure, left ventricular failure or heart failure, with a diagnostic accuracy of 70.2% (95%CI 62.9 to 76.6%). Positive predictive value was 32.3% (95%CI 21.3 to 45.5%) and negative predictive value was 92.5% (95%CI 85.2 to 96.4%).

Table 2

Cross tabulation of emergency department diagnosis† against audit diagnosis.

		Cardiology audit diagnosis
		‘wet’	‘dry’	TOTAL
Emergency Department Diagnosis‡	‘wet’	20	42	62
	‘Dry’	8	98	106
	TOTAL	28	140	168

†Indeterminate results are excluded due to difficulty in categorising, rather than inconclusive result.

‡Recorded on the Emergency Department Information System

LUS diagnostic accuracy in identifying the presence or absence of cardiac pulmonary oedema showed a significant difference from the EDIS derived clinician diagnosis, with a 16% improvement (95%CI 7.7 to 24.4%) using blinded interpretation of all protocols (LUS 86.2%, EDIS 70.2%), and a 16.2% (95%CI 3.5 to 26%) improvement using novice interpretation of scans (novice LUS 86.4%, EDIS 70.2%).

No adverse event was reported from performance of the index test.

Estimates

Inter‐rater variability was minimised by the use of a single blinded LUS reviewer and single chart auditor. The blinded reviewer had a similar diagnostic accuracy for reports from novice scans versus expert but different sensitivity and specificity. (Table 3).

Table 3

Comparison of blinded reports generated from the experienced and novice acquisitions, and bedside interpretations by experienced and novice sonologists.

	Sensitivity (95% CI)	Specificity (95%CI)	Diagnostic accuracy (95%CI)	n
Blind interpretation of scans by experienced	76.7% (57.3 to 89.4)	87.0% (79.1 to 92.2)	84.8% (78.1 to 89.8)	145
Blind interpretation of scans by novices	60% (27.4 to 86.3)	97.9% (87.5 to 99.9)	89.8% (79.5 to 95.3)	59
Aggregate	72.5% (55.9 to 84.9)	90.2% (84.3 to 94.1)	86.2% (80.9 to 90.3)	204†
Experienced interpreting own scans	70% (50.4 to 84.6)	91.2% (83.8 to 95.4)	84.8% (78.1 to 89.8)	145
Novice interpreting own scans	50% (20.1 to 79.9)	93.9% (82.1 to 98.4)	86.4% (75.5 to 93)	59
Aggregate	65% (48.3 to 78.9)	91.9% (86.3 to 95.5)	85.2% (79.8 to 89.5)	204†

†Excludes indeterminate results from reference test.

Inter‐rater agreement between bedside and blinded reviewer was ‘excellent’ (kappa = 0.82, 95% CI 0.72 to 0.92) and ‘good’ for novices (kappa = 0.70, 95% CI 0.45 to 0.95).

Due to a paucity of trained clinician sonographers and background workload, only a small portion of breathless patients who presented were recruited. A formal screening log was not kept. A retrospective report was generated from the Emergency Department Information System for the study period, using the study's inclusion criteria, and ‘dyspnoea’ as the primary presenting complaint. The system does not allow search of the free triage field to allow inclusion of those with breathlessness as a secondary complaint. Patients with reported trauma or shock (i.e. requiring active resuscitation) were excluded. The remaining diagnosis codes were regrouped to match the auditor's principle diagnosis categories. The diagnoses of the patients suffering breathlessness in the audit were similar to those in the trial group. There were 454 presentations (study group n = 230), Asthma/COPD was diagnosed in 37.7% (study group proportion − 30%), heart failure in 16.5% (20%), chest infection in 20.2% (19%), other cardiac conditions in 8.4%(13%) and miscellaneous in 17.2% (18%).

Discussion

This prospective blinded trial demonstrates a diagnostic accuracy of 86% for a simple lung ultrasound protocol when compared to expert clinical audit. This improves the diagnostic accuracy of ED assessment of breathlessness, previously estimated at 75%11,12 and 70.2% in our results. The most useful test characteristics are the high specificity for novices and the high negative predictive values that are maintained in the widened spectrum of disease. The high negative predictive value means that practitioners can use a fluid negative LUS to withhold diuretics and consider fluid bolus in distressed patients awaiting chest x‐ray

This study is a logical extension on lung ultrasound research to date. Early research was conducted on very sick inpatients, using microconvex or cardiac probes, interrogating many lung regions. , , Lung ultrasound has become a precision tool in the hands of physicians and intensivists. , , Interested emergency physicians subsequently simplified and transferred the technique to selected emergency presentations, using the more ubiquitous curved abdominal (3–5MHz) probe. , – , , At this point, controversy has arisen, with some groups recommending the extended tangential view of pleura7,23,25and others a longitudinal view that includes a rib shadow, counting B‐lines within a single interspace2 or in the whole field.

We chose to study the more recognisable longitudinal view. Our priority was a robust protocol that would act as a safe initiation for novices. We decided to count all B‐lines per screen since the longitudinal view shows less pleural surface than the previously studied tangential, and we intended to recruit the less severe end of the disease spectrum.

Since our research, international consensus guidelines have been released recommending that a positive field be considered as three or more B‐lines, shown between two ribs in a longitudinal plane, yet references the tangential studies. We have presented data that supports, quantifies and yet modifies the evidence upon which these guidelines are based. , ,

This research demonstrates that the eight‐view LUS protocol improves diagnostic accuracy for the assessment of undifferentiated breathlessness in a wider cohort of elderly patients presenting to in an urban district ED. The improvement is predominantly in specificity. The high specificity suggests that novices can use a negative scan to withhold pulmonary oedema treatment while awaiting chest x‐ray for the acutely distressed patient.

The precision of lung ultrasound decreases when applied to a less acute population. Acknowledging differing protocols, there is a contrast between the high PPV of 87.9% in Cibinel's study and the 62% PPV in Gargani's paper. The proportion identified with pulmonary oedema was 48% in the former and 25% in the latter. While our study protocol was very similar to Cibinel, the PPV and prevalence of heart failure was consistent with Gargani's group. Combining this information with the spread of diagnoses illustrated in Figure 1, the implication is that the more distressed the patient on presentation, the higher the prevalence of pulmonary oedema. Condensing a population observation to the individual, the PPV of LUS should improve in the sicker elderly patient. Conversely, chest x‐ray has been reported as negative in up to 25% of decompensated pulmonary oedema.

The low sensitivity means that the protocol will miss some heart failure patients. These are likely to be in the mild disease spectrum, as evidenced by the lower triage acuities in our study, and extravascular lung water quantification by other groups. , Other factors may have contributed to the lowered sensitivity. Semi‐erect positioning of the patient could effect fluid distribution. Novice operator limitations were demonstrated in post‐hoc review where the novice operator reported a positive field but was unable to capture all the B lines on a still field for the blinded reviewer. The cine loop and focus are important functions to enhance B line definition.

This study supports current recommendations stating that a negative protocol be used as a screening tool for reducing redundant chest x‐rays, , but that B lines indicate abnormal tissue in all but the lung bases and further imaging should be guided by the severity of symptoms rather than cutpoints in screening protocols.

Resources limited the methodology of this research in three ways. Low staffing precluded overnight recruitment, thereby missing some acute presentations. This is the group potentially able to benefit from LUS, in the absence of senior medical staff and expert radiology reporting. However we felt that there was inadequate consensus on safety, reproducibility and novice use of LUS to make the protocol available to this vulnerable group, particularly since we could not ensure blinding of the treating doctor to the LUS results.

The second resource limitation pertains to the gold standard reference test. We used retrospective audit with inpatient investigation, as per previous studies , – as our small hospital cannot provide 24‐hour echocardiography. Future studies should aim for consistent early echocardiography to allow correlation of LUS with systolic and diastolic dysfunction.

The third resource limitation was that those tested in this study was a fraction of all people presenting with breathlessness, due to a paucity of trained clinician sonographers and background workload. We do not believe this has introduced major selection bias, as the proportions of diagnoses in our sample was very similar to those of all older people presenting with breathlessness, as reported by the retrospective EDIS report.

A further limitation arises from the simplistic nature of the protocol. The intention of this study was to investigate a practical, auditable beginner's protocol, hence the choice of dichotomous decision, limited views and still image storage. The storage of still images compels compliance with the international recommendations (three separate B lines) but led to some disagreements between blinded reviewer and bedside sonologists who were inexperienced with the cine‐loop function. Further analysis of the false negative and false positive subgroups of scans will be performed.

Finally, extensive exclusions from the ED diagnosis group widened the confidence interval in this subset. We justify inclusion of these results as they are consistent with other reported studies. –

The challenge for future research is more accurate identification of true positive results in subacute illness. The demonstrated drop in sensitivity was expected, as prior studies have shown that the absolute number of B lines correlates strongly with the volume of extravascular lung water. , , , Subclinical disease might be expected to manifest fewer B lines. Using alternative strategies, groups report mixed success using fewer B lines or pleural effusions to identify pulmonary oedema. , , Urgent echocardiography would be very useful, to inform both the sonologist and the reference standard.

Researchers seeking to improve protocol performance in the broader populations are starting to look beyond the cutpoints, to pleural line abnormalities to indicate chronic lung disease, and cardiac views – to differentiate right from left heart failure. Pleural effusions remain equivocal. , The challenge to researchers will be in optimising the trade‐off between protocol simplicity and accuracy.

In the same way as FAST scans have initiated physicians to abdominal scanning, LUS is a safe way to introduce novices to lung interrogation. This study reiterates the safety of a very basic dichotomous protocol, which incorporates skills that can later be used to identify local changes such as early pneumonia and effusion. The ED is the ideal opportunity for learning lung ultrasound because findings can be compared rapidly to chest x‐ray.

Conclusion

Emergency department LUS is a safe, convenient and feasible investigation. After brief training, it is accurate enough to guide emergency management or for use as rapid screening tool. With a diagnostic accuracy of 86%, a basic lung ultrasound protocol improves the current diagnostic paradigm of breathlessness in the elderly, beyond standard clinical assessment and baseline investigations.