The acute care diagnostics collaboration: Performance assessment of contrast-enhanced ultrasound compared to abdominal computed tomography and conventional ultrasound in an emergency trauma score bayesian clinical decision scheme.

BACKGROUND: Bayes' theorem describes the probability of an event, based on conditions that might be related to the event.[1] We developed the Bayesian Diagnostic Gains (BDG) method as a simple tool for interpreting diagnostic impact.[234567]. AIM: We aimed to evaluate the clinical diagnostic impact of contrast-enhanced ultrasound (CEUS) compared to traditional abdominal computed tomography (CT) and standard ultrasound (US) in a Bayesian Clinical Decision Scheme.
MATERIALS AND METHODS: Our mathematical method uses Bayesian Diagnostic Gains (BDG) model. For the purposes of our model, the EMTRAS was used as pretest probability and stratified as low risk (0-3 points = 10%), moderate risk (4-6 points = 42%), and high risk (7-12 points = 80%) based on mortality risk. Sensitivity and specificity for US, CT, and CEUS were obtained from pooled data and used to calculate LR- and LR+. Bayesian/Fagan nomogram was used to attain posttest probabilities using baseline probability of an event on the first axis (PRE), with LR on the second axis, and read off the pos-test probability (POST) on the third axis. For the nomogram analysis, the pretest probability (Pre) scoring for the EMTRAS score was obtained using the original EMTRAS data. Posttest probabilities were obtained based on the Bayes/Fagan Nomgram. Relative diagnostic gain (RDG) and absolute diagnostic gain (ADG) were calculated based on the differences deducted from pre- and post-test probabilities. IBM® SPSS® Statistics 20 was used for analysis and modeling. ANOVA was used for association between EMTRAS, CT scan, and CEUS, where P value set at 0.05.
RESULTS: Pooled data for Sensitivity (Se), Specificity (Sp), LR+, and LR- were obtained for US (Se = 45.7%, Sp = 91.8%, LR+ = 5.57, and LR- = 0.59), CEUS (Se 91.4%, Sp 100%, LR+ 91, and LR-0.09), and CT (Se = 94.8%, SP = 98.7%, LR+ = 73, and LR- =0.05). ANOVA analysis for LR+ and LR- showed no significant difference (P < 0.8745 and P < 0.9841). Comparison of CT and CEUS did not yield statistically significant differences for LR+ (P < 0.1).
CONCLUSION: In this Bayesian model, the diagnostic performance of CEUS was found to be similar to traditional abdominal CT. The greatest diagnostic gain was observed in low pretest positive LR groups.

INTRODUCTION

In probability theory and statistics, Bayes' theorem describes the probability of an event, based on conditions that might be related to the event Bayes' theorem then links the degree of belief in a proposition before (pretest probability) and after (posttest probability) accounting for evidence.[1]

With the Bayesian probability interpretation, the theorem expresses how a subjective degree of belief should rationally change to account for evidence: this is Bayesian inference, which is fundamental to Bayesian statistics. The Fagan/Bayesian nomogram is a graphical calculator that is a useful and convenient way to perform calculations without the need to remember the formula integrating pretest probability with diagnostic tests and likelihood ratios (LRs) [Figures 1–3]. The use of the Fagan/Bayes' nomogram has simplified the use of diagnostic test information and is now frequently used by numerous physicians. We developed the Bayesian Diagnostic Gains (BDG) method as a simple clinical tool for interpreting diagnostic impact, this mathematical decision support model has been studies and validated by our ACDC work group.[234567]

Figure 1

Methodology schematic

Figure 3

Bayesian/Fagan nomograms for computed tomography, ultrasound, and contrast-enhanced ultrasound

The Emergency Trauma Score (EMTRAS) was developed to be an easy-to-compute scoring system for the emergency resuscitation and based on a limited number of clinical predictors that are commonly and early available. In 2009, Raum et al.[8] introduced the EMTRAS for early estimation of mortality risk in adult trauma patients. EMTRAS combines four early predictors from the emergency resuscitation room and demonstrated favorable discrimination compared with more complex scores. The early predictors used by EMTRAS are age (year), Glasgow Coma Scale (GCS), base excess (mmol/L), and prothrombin time (PT) (%). For each predictor, a sub score of 0, 1, 2, or 3 points is assigned, based on the actual value of the predictor. EMTRAS is defined as the sum of these sub scores, that is, the lowest (best) EMTRAS is zero and the highest (worst) is 12. The EMTRAS was developed in a large cohort (n = 4808) of trauma patients with an Injury Severity Score[8] of 16 or higher and derived from the German Trauma Registry (http://www.traumaregister.de). The EMTRAS has been internally and externally validated.[89] Therefore, we aimed to evaluate the clinical diagnostic impact of contrast-enhanced ultrasound (CEUS) compared to traditional abdominal computed tomography (CT) and standard ultrasound (US) in a Bayesian Clinical Decision Scheme integrating the EMTRAS.

METHODS

Mathematical model

Sensitivity was defined as the ability of a test to correctly identify those with the disease (true positive rate), whereas test specificity was defined as the ability of the test to correctly identify those without the disease (true negative rate). LRs were used as epidemiological instruments to show how much we should shift our suspicion for a particular test result. The positive LR (LR+) was defined as probability of an individual with the condition having a positive test LR+ = probability of an individual without the condition having a positive test. Similarly, the negative LR (LR−) was defined as probability of an individual with the condition having a negative test LR− = probability of an individual without the condition having a negative test. We defined LR+ and LR− in terms of sensitivity and specificity:

LR+ = sensitivity / (1-specificity) = (a/(a+c)) / (b/(b+d))

LR- = (1-sensitivity) / specificity = (c/(a+c)) / (d/(b+d))

Specificity

Bayes' theorem was used to convert the results from CT, US, and CEUS testing into the probability of the event. Bayes' math describes the analysis as a relation of Pr (AǀX), the chance that an event A happened given the indicator X, and Pr (XǀA), the chance the indicator X happened given that event A occurred. Our mathematical method uses Bayes' nomogram [Figures 1–3]. In statistics, a nomogram is an arrangement of two linear or logarithmic scales such that an intersecting straight line enables an intermediate values or values on a third scale to be read off using a straight edge on the nomogram, line up the baseline probability of an event on the first axis, with LR on the second axis, and read off the posttest probability on the third axis [Figure 1].

Population

Stratification of the population was made using point scores attributed by applying the EMTRAS which is comprised four parameters: patient age, GCS, base excess, and PT. For the purposes of our model, the EMTRAS was used as pretest probability and stratified as low risk (0–3 points = 10%), moderate risk (4–6 points = 42%), and high risk (7–12 points = 80%) based on mortality risk. Sensitivity and specificity for US, CT, and CEUS were obtained from pooled data and used to calculate LR− and LR+ [Figure 1].[10111213]

Outcomes

Bayesian nomogram was used to attain posttest probabilities using baseline probability of an event on the first axis (PRE), with LR on the second axis, and read off the posttest probability (POST) on the third axis. For the nomogram analysis, the pretest probability (Pre) scoring for the EMTRAS score was obtained using the original EMTRAS data.

Posttest probabilities were obtained after inserting EMTRAS score as pretest probability and LRs into Bayesian nomogram [Tables 1–4]. Posterior probability is the mathematical sum of the diagnostic value of the EMTRAS plus CEUS or CT scan.

Table 1

Diagnostic gain results for ultrasound

Table 4

Stuart-Maxwell test results for absolute diagnostic gains and relative diagnostic gains in “rule ins (+) and rule outs (−)”

We developed a simple method for interpreting diagnostic impact, where relative diagnostic gain (RDG) and absolute diagnostic gain (ADG) were calculated based on the differences deducted from pre- and posttest probabilities (ADG = post-test – pre-test) and (RDG = 100 × post-test – pre-test/Pre-test).[234567] Figure 2 reflects an example of the mathematical model for creating the case distribution and calculations. IBM® SPSS® Statistics 20 was (IBM Corp. IBM SPSS Statistics for Windows, Version 20.0. Armonk, NY, USA) used for analysis and modeling. ANOVA was used to evaluate strength of association between EMTRAS, CT scan, and CEUS, and Stuart–Maxwell test was used for symmetry and marginal homogeneity testing, where P value set at 0.05. IBM® SPSS® Statistics 20 was used for analysis and modeling. The mathematical nature of this study makes it Institutional Review Board exempt.

Figure 2

Analysis of variance between computed tomography and contrast-enhanced ultrasound

RESULTS

Pooled data for Sensitivity (Se), Specificity (Sp), LR+, and LR− were obtained [Table 5] for US (Se = 45.7%, Sp = 91.8%, LR+ = 5.57, and LR− = 0.59), CEUS (Se 91.4%, Sp 100%, LR+ 91, and LR− 0.09), and CT (Se = 94.8%, SP = 98.7%, LR+ = 73, and LR− = 0.05).[12]

Table 5

Sensitivity, specificity, and likelihood ratios

US LR+ model results showed low-risk posttest probability of 38%, RDG of 28%, and ADG of 280%, moderate-risk posttest of 80%, RDG of 38%, and ADG of 90.5%, whereas high-risk posttest of 96%, RDG of 16%, and ADG of 20% [Table 1].

Contrast-enhanced US model results for LR + yielded low-risk posttest probability of 91%, ADG of 81.0%, and RDG of 810.0%, moderate-risk posttest probability of 99.0%, ADG of 57.0%, and RDG of 135.7%, whereas high-risk posttest probability of 100.0%, RDG of 20.0%, and RDG of 25.0% [Table 2].

Table 2

Diagnostic gain results for computed tomography for positive and negative likelihood rations

CT LR+ results were low-risk posttest of 89%, RDG of 79%, and ADG of 790%, moderate-risk posttest of 98%, RDG of 56%, and ADG of 133.3%, whereas high-risk scores, posttest of 100%, RDG of 20%, and ADG of 25% [Table 3].

Table 3

Diagnostic gains analysis for contrast-enhanced ultrasound

ANOVA analysis for LR+ showed no significant difference (P < 0.8745), with standard error of difference of 485.39 [Figure 1]. ANOVA analysis for LR− showed no significant difference (P < 0.9841), with standard error of difference of 248.123. Comparison of CT and CEUS did not yield statistically significant differences for LR+ (P < 0.1) [Figure 2].

Stuart–Maxwell test comparing ADG for LR+ P yielded a P < 0.0001 and ADG for LR− a P < 0.04, when comparing RDG for LR+ a P < 0.0001 0001 and RDG for LR− a P < 0.0001 [Table 4].

DISCUSSION

This study using Bayesian statistical model demonstrated the comparable diagnostic quality of CEUS and CT scan in risk-stratified patients using EMTRAS score. More specifically, for rule out of disease, CT and CEUS were statistically significantly superior than traditional US in all three risk populations, proving that clinically, CEUS has equal diagnostic accuracy to discard disease in all patient populations. Lowest diagnostic gain was observed using traditional US, where ADG ranged from 12.5% to 40%, lowest being high pretest probability population. Our results showed greatest ADG (81.0%) for low pretest probability EMTRAS using CEUS.

Previously published literature also provides evidence of the superiority of CT and CEUS over simple abdominal US.[891011121314] More specifically, Catalano et al.[14] evaluated concordance of results among US and CEUS in patients with blunt abdominal trauma, showing an increase in sensitivity, specificity, and accuracy when using CEUS. In our moderate-risk patient population, however, CT scan demonstrated a superior RDG in ruling out disease.

These results are important in that they suggest that moderate-risk individuals represent the most beneficial population in being imaged to rule out disease severity using EMTRAS and CT scan. Likewise, Pinto et al.[15] reported that CEUS can provide a more reliable evaluation of solid organ injuries and vascular-related complications in a more timely manner yet concluded that it does not replace CT imaging. Our results showcase promising potential applications for austere and resource-limited environments.

Limitations of this study include the mathematical nature of this model as well as the use of pooled meta-analysis data for diagnostic performance indicators. Other limitations include those related to severity of injury speculations based on the use EMTRAS score. Furthermore, future studies should take into consideration the cost of a CEUS or CT test when choosing appropriateness of test, as by choosing the correct test for the right patient, one could save an important amount of dollars per patient. A more comprehensive value-based and cost-effective analysis will be performed in a future study. Other further implications in potential future studies that we recommend involve the implications of contrast-enhanced risks of CEUS versus the radiation-related risks posed by CT scan, which would be particularly beneficial in the pediatric population.

CONCLUSION

In a Bayesian Clinical Decision Scheme incorporating EMTRAS for risk stratification, the diagnostic performance of CEUS was found to be similar to traditional abdominal CT. The greatest diagnostic gain was observed in low pretest positive LR groups, and further validation of this model is needed as well as cost-benefit analysis.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.