Accuracy of ED Bedside Ultrasound for Identification of gallstones: retrospective analysis of 575 studies.

STUDY
OBJECTIVE: To determine the ability of emergency department (ED) physicians to diagnose cholelithiasis with bedside ultrasound.
METHODS: ED gallbladder ultrasounds recorded over 37 months were compared to radiology ultrasound interpretation.
RESULTS: Of 1,690 ED gallbladder ultrasound scans performed during this period, radiology ultrasound was performed in 575/1690 (34%) cases. ED physician bedside interpretation was 88% sensitive [95% CI, 84-91] and 87% specific [95% CI, 82-91], while positive predictive value (PPV) was 91% [88-94%] and negative predictive value (NPV) was 83% [78-87%], using radiology interpretation as the criterion reference.
CONCLUSION: ED physician ultrasound of the gallbladder for cholelithiasis is both sensitive and specific.

INTRODUCTION

As less expensive, portable ultrasound machines became available in the mid-1980s, ultrasound made its debut in the emergency department (ED) as a diagnostic tool.1,2 Ultrasound is now an accepted use in the ED by emergency physicians (EPs).3–13 Training in ultrasound is part of the curriculum in 96% of emergency medicine (EM) residencies, and there are a growing number of fellowships in emergency ultrasound for EM–trained physicians.14

Approximately 20 million people in the United States have gallstones, with an additional one million diagnosed each year.15 Traditionally, evaluation involves a right upper quadrant (RUQ) ultrasound by an ultrasound technician and interpretation by a radiologist. However, radiology ultrasound is not universally available to ED patients. ED ultrasonography of the RUQ can result in shorter stays, lower costs, and better patient satisfaction.16,17

Over three years we performed 2,321 RUQ ultrasounds. With this large experience, we attempt to assess the ability of emergency physicians to diagnosis cholelithiasis. This study is the largest sample to date of ED right upper quadrant ultrasound with image confirmation and comparison to radiology criterion reference.

METHODS

We Performed the study in a 35-bed Level I trauma center ED, with a census of 41,000 patients per year, supporting a Post Graduate Year 1–3 EM residency. Thirteen attending emergency physicians and one ultrasound fellow had various levels of expertise with ultrasound. Three attendings were credentialed by the American Registry of Diagnostic Medical Sonography (RDMS) and seven (including the fellow) had sufficient experience to qualify to take the RDMS exam. The remaining three rarely performed ultrasound. Obtaining hospital privileges first required 300 ED ultrasound examinations, over-read by the Director of Emergency Ultrasound, who had completed an emergency ultrasound fellowship (JCF). After this provisional period, three ultrasounds were proctored for each application. Correct performance and interpretation then warranted hospital privileges. During the study period, three emergency physicians had hospital privileges to make clinical decisions based on ED ultrasound.

The 36 residents had performed 0–1000 ultrasounds prior to the 37-month study. During the study period, each performed 200–1200 ED ultrasounds of all applications; 15–200 were of the RUQ. The RUQ application was second only to trauma as the most common application of bedside ultrasound. Residents received 12 hours of didactic ultrasound training per year, in addition to six hours during EM-1 orientation. Faculty attended approximately half of these sessions.

Until July 2004, all ultrasounds were done with a B+K Hawk 2102 ultrasound machine (Copenhagen, Denmark). After that, a Sonosite Titan (Bothell, WA) was also available. Both machines had 3.5–5 MHz abdominal probes and video recorders. Physicians recorded each ultrasound and wrote the bedside interpretation on standard data sheets.

We reviewed emergency department RUQ ultrasounds from July 1, 2001 to August 10, 2004 for gallstones, and examined computer records for radiology ultrasound within one month of the index scan. We did not address other diagnostic ultrasound criteria for cholecystitis. We excluded cases lacking a valid patient identifier, bedside interpretation, video recording, or adequate images as determined at the bedside, or radiologist.

We calculated sensitivity, specificity and predictive values for ED bedside US compared with radiology US interpretations. These parameters were calculated in the usual manner with confidence intervals by the efficient-score method with continuity correction.18

The Institutional Review Board approved the study protocol and documentation of consent was waived.

RESULTS

ED physicians documented 2,321 gallbladder ultrasounds during the study period. Six hundred thirty-one (27%) scans were excluded for lack of: patient identifier (n=26, 1%), bedside interpretation (n=114, 5%), video recording (n=272, 12%), and adequate imaging (n=219, 9%). Of the remaining 1,690 ultrasounds (Figure 1), 575 (34% of 1690 and 25% of 2321) had radiology ultrasound and formed the final study population.

Figure 1

Flow diagram of right upper quadrant emergency department ultrasounds

In the 575 patients who received both bedside and radiology ultrasounds, radiology diagnosed gallstones in 344 of 575 (60%). ED ultrasound was positive for gallstones in 332 (57.7%) patients. ED bedside interpretation was 88% sensitive [95% CI, 84–91%] and 87% specific [82–91%] (Table 1). Positive predictive value (PPV) was 91% [88–94%] and negative predictive value (NPV) was 83% [78–87%].

Table 1

Results of ED ultrasound compared to subsequent radiology ultrasound diagnosis.

	Gallstones present	Gallstones absent	Total
ED ultrasound postive	303	29	332
ED ultrasound negative	41	202	243
Total	344	231	575

Accuracy = 87.2%

Six percent (144/2321) of bedside ultrasound examinations had inadequate images for emergency physician interpretation, compared to 1.4% (8/575) of ultrasound technologist-acquired examinations which were not interpretable by radiologists.

DISCUSSION

Cholecystosonography is the initial imaging modality for the diagnosis of cholelithiasis.15,19–24 Ultrasound is non-invasive, relatively inexpensive, requires no radiation, and is very accurate in skilled hands. As gallstones are present in 90–95% of acute cholecystitis and association with a sonographic Murphy’s sign yields a PPV of 92%, bedside ultrasound may rapidly drive early diagnosis and disposition. This is important, as emergency physician ability to identify other markers of acute cholecystitis (i.e. pericholecystic fluid and thickened gallbladder wall) is much more limited.

We demonstrate that emergency physicians in this training program are accurate in ultrasound diagnosis of gallstones. The sensitivity and specificity of 88% and 87%, compared to radiology ultrasound, is consistent with the four prior studies in the EM literature as noted in the Table 2.3,25–27

Table 2

Prior studies from the emergency medicine literature addressing sensitivity and specificity of ED ultrasound.

Author	Year	Design	Recruitment	Inclusion Group	Sample Size	Criterion Reference	Sensitivity	Specificity
Schlager (3)	1994	Prospective	Convenience	ED abdominal pain	65	Radiology ultrasound (only if ED US abnormal)	86%	97%
Kendall (27)	2001	Prospective	Convenience	ED abdominal pain with radiology ultrasound	109	Radiology ultrasound	96%	88%
Rosen (22)	2001	Prospective	Convenience	ED abdominal pain	116	Radiology ultrasound	91%	78%
Durston (25)	2001	Retrospective	Convenience	Known biliary disease	418	Pathology or Clinical or Radiology ultrasound	89%	98%
Scruggs (Current)	2006	Retrospective	Convenience	ED abdominal pain	575	Radiology ultrasound	88%	87%

The present study documents similar sensitivity as previous efforts. The specificity of studies, including this one, is high when ED ultrasound is performed on all patients with abdominal pain, but lower when considering only select patients who also had radiology ultrasound. This verification bias has been described by Shea et al.19 regarding biliary ultrasound.

In our experience, use of radiology ultrasound is more likely when the bedside ultrasound finds abnormalities. Thus the specificity is underestimated. Descriptions of ultrasonographer experience, quality of machine, and level of practitioner (attending vs. resident) are variable in previous studies and may contribute to the differences in accuracy.

The sample size of 575 verified by radiology ultrasound almost doubles previously published experience with this comparison. The radiology literature demonstrates a sensitivity of 81–99% for radiology ultrasound for the diagnosis of cholelithiasis,21,28–33 though the majority of the studies demonstrating sensitivities at the lower end of the range were performed in the early years of ultrasound. A rigorous meta-analysis of the radiology literature in 1994 by Shea et al.19 for the diagnosis of cholelithiasis found a sensitivity of 97% and specificity of 95% (unadjusted for verification bias). The adjusted sensitivity was 84% and specificity 99% (adjustment by method of Greenes and Begg – ref 29 in Shea). Nevertheless, a variety of textbooks quoted in the same manuscript report sensitivities > 95%, which appears overstated.

Our study demonstrates a 6% rate of inadequate imaging by EPs. This compares to the 18% indeterminate rate reported by Durston and the 5% of cases in the Rosen study in which the gallbladder was not visualized.1,3 The Kendall27 and Schlager3 studies did not comment on their rate of indeterminate scans. Our data suggests that radiology ultrasound is limited by inadequate imaging less frequently, finding only 8/575 (1.4%) to be indeterminate for cholelithiasis.

The increased rate of EP inadequate imaging has several potential explanations. First, EPs typically spend less than 10 minutes on their exams, and likely do not use adjuncts such as patient positioning and water ingestion.27 Second, contracted gallbladders in the ED may distend during the 2–3 hours it takes the ultrasound technologist to respond. Finally, ED ultrasound machines sacrifice image quality for durability, portability, and cost.

We excluded bedside studies with inadequate images from our primary analysis. This mimics real-life practice, as EPs routinely obtain a radiology ultrasound if their own images are inadequate. Ninety-four percent of patients had adequate images in our study, potentially expediting their ED evaluation.

LIMITATIONS

Our study has several limitations. Verification bias is inherent in a retrospective study of a diagnostic test. Three-fourths of patients who had emergency department RUQ ultrasound did not have a confirmatory study by radiology, and it is possible that gallstones were missed completely in those that did not undergo the criterion reference. Conversely, some ED patients undoubtedly had radiology ultrasounds ordered initially, foregoing ED bedside ultrasound.

Second, the clinical and sonographic experience of our physicians varied widely. Paradoxically, some of the most experienced clinicians had limited ultrasound skills whereas many of the junior residents had performed several hundred scans. Finally, our criterion reference lacked pathologic confirmation, a recurrent problem with EM biliary imaging literature.

Given the consistent performance of ED bedside ultrasound, its incorporation in the diagnostic process needs further investigation. The EM and radiology literature lack any likelihood ratios that could be used in a Fagan nomogram to modify the pretest probability for acute cholecystitis.34 The ultimate goal of ED bedside ultrasound is to expedite disposition or consultation of patients with RUQ pain. Although this is commonly done, the process has not been studied quantitatively.

CONCLUSION

We found that emergency physicians are accurate in diagnosing cholelithiasis with bedside ultrasound in the largest sample size to date in emergency medicine ultrasound literature. This study adds to the growing body of evidence supporting the use of ultrasound by emergency physicians for gallbladder pathology.