The Effects of a Systemwide Diagnostic Stewardship Change on West Nile Virus Disease Ordering Practices.

We report that removing the clinically insensitive West Nile virus CSF nucleic acid amplification test (NAAT) from the electronic health record (EHR) test. This diagnostic stewardship intervention decreased costs and may have improved diagnostic yield.

West Nile virus (WNV) is a flavivirus that exists in a transmission cycle between mosquitos and birds [1]. It acts as a zoonotic infection when infected Culex spp mosquitos transmit it to humans who are considered dead-end hosts [2]. Since its emergence in North America in 1999, it has remained endemic and caused thousands of cases each year in the United States [3, 4].

Although the precise incubation period for clinically apparent infections is unknown, in immunocompetent individuals it is thought to be between 2 and 14 days [1, 5]. Approximately three fourths of infections are likely clinically inapparent, whereas approximately 25% will develop WNV fever and <1% develop WNV neuroinvasive disease (WNV-ND) [1]. In contrast, as many as 1 in 50 patients older than 65 develop WNV-ND [6]. Full recovery is common among patients with WNV fever, but WNV-ND is often complicated by severe neurologic sequelae. Patients often experience prolonged recovery of neurological function or even death [7, 8].

Diagnosis of WNV-ND is based on appropriate laboratory testing in the right clinical scenario. The immunoglobulin M antibody capture enzyme-linked immunosorbent assay (MAC-ELISA) is thought to be positive in more than 90% of patients within 8 days of symptom onset [9]. The test is available through commercial laboratories. A positive MAC-ELISA in the cerebrospinal fluid (CSF) is highly suggestive of a WNV infection, but due to cross-reactivity with other arboviruses, it should be confirmed with plaque-reduction neutralization testing (PRNT), or detection of WNV nucleic acid via a nucleic acid amplification test (NAAT). Although NAAT is analytically highly sensitive (detection at 10–100 copies/mL), the short duration of viremia and often low amounts of virus in the CSF results in a low clinical sensitivity (4%–57%) [9, 10]. In addition, NAAT is more expensive than the MAC-ELISA. Therefore, the Centers for Disease Control and Prevention (CDC) recommends testing for WNV using the MAC-ELISA rather than NAAT [11].

In an effort to increase the use of the MAC-ELISA and to decrease unnecessary polymerase chain reaction testing, CSF NAAT was removed from the test menu of the electronic health record (EHR) of a 5-hospital health system. Subsequently, we analyzed data from the EHR to determine the effectiveness of this intervention in reducing the use of the NAAT, decreasing costs, and detecting cases of probable WNV-ND. We hypothesized that removing the NAAT from the EHR ordering menu would decrease the amount of NAATs ordered and increase the use of the MAC-ELISA.

METHODS

In April 2018, the CSF WNV NAAT was removed from the test menu within the EHR of a health system comprising 2 academic hospitals in Baltimore, Maryland and 3 community hospitals (Maryland and Washington D.C.). The NAAT ordering remained possible via a paper order. House officers were provided brief education about this change. We then reviewed WNV testing done on CSF samples obtained from patients at those hospitals from July 2016 through December 2018. The primary objectives were to compare the number of MAC-ELISA and NAAT WNV tests ordered before and after the change to the ordering protocol. The secondary objectives were to determine whether this change led to any cost savings or changes in the detection rate of WNV-ND. The monthly, seasonal, and yearly number of positive test results, total test results, and total costs were determined from July 2017 to April 2018 compared with May 2018 to January 2019. A paired t test was performed to evaluate differences in total testing, total positive, and total costs during nonwinter months before and after the intervention. Positive test results were clinically adjudicated independently by 2 infectious diseases physicians.

RESULTS

Both the WNV CSF MAC-ELISA and the NAATs were available to order at all hospitals in the health system during the study period. The cost incurred by the hospital for the NAAT was $150 per test during the study period and an average of $17 for the MAC-ELISA.

An average of 12.6 MAC-ELISA tests were performed per month (95% confidence interval [CI], 10.3–14.9) before the intervention. This increased to an average of 41 MAC-ELISA tests/month (95% CI, 34.4–47.7) in the postintervention period, which was statistically significant (P < .001). In contrast, there was an average of 46.2 NAATs/month (95% CI, 39.6–52.9) before the intervention, which decreased to 0 NAATs/month afterwards (P < .001) (Figure 1A). In addition, the average number of WNV tests (MAC-ELISA + NAAT) performed decreased from 58.8 tests/month (95% CI, 51.0–66.6) to 41.0 tests/month (95% CI, 34.4–47.6) after the ordering intervention (P = .007). Comparing just the nonwinter months, the average number of NAATs ordered per month decreased from 49.7 tests/month (95% CI, 41.3–58.0) to 0 tests/month after the intervention. In contrast, the average number of MAC-ELISA tests ordered per month increased from 14.3 (95% CI, 12.0–16.7) to 44.0 (95% CI, 39.1–48.9) (Figure 1A).

Figure 1.

West Nile virus (WNV) cerebrospinal fluid (CSF) testing and costs from July 2016 to January 2019. (A) Total nucleic acid amplification tests ([NAATs] blue bars, left axis), total immunoglobulin M antibody capture enzyme-linked immunosorbent assay (MAC-ELISA) tests (gray bars, left axis), positive NAATs (orange bars, right axis), and positive immunoglobulin (Ig)M tests (purple bars, right axis) for each indicated month (x-axis) are shown. Dissemination of information to house officers began March 2018. A red arrow indicates the time of removal of WNV NAAT from the order test menu (May 2018). (B) The NAAT costs (orange bars) and IgM test costs (blue bars) in US dollars (USD) for each month are shown. A red arrow indicates the time of removal of WNV NAAT from the order test menu (intervention).

Because of the difference in cost, the intervention resulted in a 93.5% decrease in WNV-ND test spending from an average of $7199.76 per month to $471.00 per month (P < .001) (Figure 1B). In addition, preceding the intervention, 0.23% of all WNV CSF tests were positive (NAAT+MAC-ELISA), whereas 2.44% of WNV CSF tests were positive after the intervention (P = .03) (Figure 1A). No positive NAATs were reported during the study period. In contrast, there were 3 positive MAC-ELISA tests before the intervention and 9 positive results after the intervention (all during nonwinter months). Of these, 8 were determined to be true positives and 1 was considered not clinically consistent with WNV-ND.

DISCUSSION

A significant amount of healthcare dollars are wasted each year on inappropriate ordering of laboratory tests. Recently, diagnostic stewardship interventions have effectively used the EHR to reduce unnecessary testing for gastrointestinal infections and rheumatologic disorders [12, 13]. In this study, we demonstrate a simple solution to the problem of improper ordering of WNV NAAT by removing it as an option from the EHR test menu and providing brief education to house officers.

By engaging the appropriate departments (ie, medicine, emergency medicine, neurology, and pathology), we were able to successfully make this change. Instrumental in making the change was communicating the CDC guidelines to stakeholders, and also demonstrating to them that we historically had a 0% positivity rate with the NAAT. Although this was a positive change for clinical practice reasons, it also resulted in dramatically decreased costs.

Furthermore, the intervention was associated with an increase in the number of positive WNV CSF tests. This may be due to the increased sensitivity of the MAC-ELISA compared with the NAAT. However, the CDC reported 11 cases of WNV-ND in Maryland in 2018, but only 6 and 5 in 2016 and 2017, respectively [14]. Therefore, the increased detection could also be explained by the increased incidence during the year of the intervention. It is interesting to note that no positive NAATs were found during the study period, further supporting the poor utility of this as the primary test for WNV-ND.

A limitation of this study is that it was designed as a quality improvement study, and we were not able to analyze relevant patient-level clinical data including how many patients also had serum testing for WNV or the time between symptom onset and testing in those that had negative testing. Therefore, we do not know whether clinicians became more discriminating in their ordering after our brief meetings with house staff. Although other flaviviruses may cross-react with the WNV MAC-ELISA [1], the false-positive rate was low in this study (0.11). However, serological testing may be negative in patients who present very early (<3 days) after onset of symptoms or who are immunosuppressed [15]. It is unclear how many patients with negative testing would have fit these criteria.

This study also suggests that significant knowledge gaps exist regarding WNV disease. Before the ordering change, NAAT was performed 3.7 times more often than MAC-ELISA. The NAAT may have been erroneously regarded as a more sensitive test extrapolating from other disease processes or from confusion between analytical and clinical sensitivity. In addition, although WNV-ND is extremely rare during winter [16], a significant number of NAATs were ordered in winter, suggesting that providers do not appropriately judge prior probabilities in their decision to order NAAT.

It is important to note that NAAT remained available to order, but the process required filling out paperwork. During the follow-up study period, there were no paper NAAT orders placed potentially due to the perceived high time cost. The data presented here cannot directly address either the indications for the tests or the motivations for the providers, but systematic “nudges” to improve diagnostic stewardship should be further researched.

CONCLUSIONS

In conclusion, elimination of electronic ordering is an effective way of decreasing inappropriate WNV NAAT ordering and decreasing associated costs, and it may lead to improved diagnosis of WNV-ND. In reducing low-yield testing, evidence-based selection of EHR test menu options is an effective strategy to improve diagnostic accuracy of relatively uncommon or rare diseases.