Criteria to Screen Molecular Tests for the Diagnosis of Herpes Simplex Virus in the Central Nervous System have no Propensity to Harm.

OBJECTIVES: Investigators have ruled out herpes simplex virus (HSV) infection without the detection of herpes simplex deoxyribonucleic acid in cerebrospinal fluid (CSF) (i.e., HSV polymerase chain reaction [PCR]) by laboratory (normal CSF white blood cell count and protein) and clinical criteria (age ≥2 years, no history of human immunodeficiency virus or solid-organ transplant). Compared to HSV PCR of all samples, the algorithm saves money in test costs and may decrease exposure to acyclovir by illustrating the low probability that the patient has HSV. Concern exists that algorithm use may cause harm through alteration of empiric acyclovir treatment in patients with true HSV central nervous system infection.
METHODS: All Department of Veterans Affair's patients with a positive HSV PCR of the CSF between 2000 and 2013 were identified and their medical records reviewed to determine the extent and possible impact of omitted HSV PCR testing by the algorithm.
RESULTS: Of 6357 total results, 101 patients had a positive CSF HSV PCR in the study period. Among the positive CSF HSV PCR results, the algorithm excluded 7 (7%) from PCR testing. Record review indicated these seven patients not tested by the algorithm with a positive CSF HSV PCR were considered by their attending physician not to have active HSV.
CONCLUSION: The algorithm to screen HSV tests had no propensity to harm.

Introduction

Herpes simplex virus (HSV) is the most common cause of sporadic, life-threatening encephalitis in the Western world.[1] Encephalitis-associated hospitalizations are estimated to cost $2 billion with herpes simplex as the most frequently identified etiology.[2] Among laboratory tests, the cost of HSV polymerase chain reaction (PCR) ranks in the 90th percentile.[3] Typical costs for the payer range between $220 and $450 2011 U.S. dollars. The current emphasis on high quality, value-driven health care, motivates a fair yet critical evaluation of this proposal to alter the laboratory evaluation of HSV in the central nervous system (CNS).

Multiple authors have proposed and validated specific criteria as an alternative to routine HSV deoxyribonucleic acid detection (i.e., PCR).[4567] These criteria defer HSV PCR testing when specific clinical and laboratory criteria indicate a very low probability of HSV infection. The most studied criteria, the criteria adopted from previous authors[45] by Hanson et al.,[6] have suggested HSV PCR of cerebrospinal fluid (CSF) can be safely deferred when the CSF white blood cell (WBC) count and protein levels are within the normal range (≤5 cells/mm3 and ≤50 mg/dl, respectively) in a person 2 years of age or older without human immunodeficiency virus (HIV) or transplant (”the algorithm”) [Figure 1].

Figure 1

The algorithm proposed by Hanson et al. to rule out herpes simplex virus infection of the central nervous system. Patients who meet this criteria would not receive a test for the detection of herpes simplex virus DNA (i.e., PCR) in cerebrospinal fluid. *The original paper listed “transplant,” which we interpreted as solid-organ transplant to include patients with nonimmunosuppressive transplants (e.g., cornea). DNA: deoxyribonucleic acid, PCR: Polymerase chain reaction

Arguments exist against the implementation of the algorithm for the diagnostic evaluation of patients suspected of a HSV CNS infection. For Example, a misclassification could delay the administration of acyclovir, a therapy proven effective for herpes simplex encephalitis.[8] In this way, anecdotal case reports suggest the algorithm may cause harm.[910] Other studies have suggested that patients with HSV encephalitis have lower elevations in CSF white cells and protein compared to HSV meningitis, thus making the criteria inaccurate and potentially harmful.[5] The current single-institution validation studies may also have a sample size too small to detect a rare exceptional case.[4567]

To evaluate the potential for the algorithm to cause harm, we performed a retrospective case series study over a 13-year period in the largest integrated medical system in the United States, the Veterans Affairs (VA) Healthcare System. Specifically, we retrospectively applied the algorithm to each patient who had a positive CSF HSV PCR testing to evaluate the following questions (a) would algorithm use have prevented or delayed the empiric administration of acyclovir?[9] (b) would miscommunication of HIV status or solid-organ transplant history have led to an inappropriate application of the algorithm?[6] and (c) would patients with encephalitis, who may have lower CSF white cells and protein levels compared to patients with meningitis, have led to a failure to perform an HSV PCR test due to use of the algorithm?[5]

Methods

Patients

The study cohort was developed by identifying all patients cared for in the VA Healthcare System with a qualifying HSV PCR test performed on CSF from January 1, 2000, to June 30, 2013. HSV tests had one of these LOINC test identifiers: 16952-4, 16960-7, 32141-4, 34655-1, or 5013-8. HSV tests qualified with a positive or negative result (e.g., not “canceled”). Patients with a qualified HSV PCR were included in the study cohort if they also had a WBC count and protein level from CSF within 2 days of HSV sample collection. If multiple CSF WBC or protein results were available, the results used for the analysis were those that had the collection time nearest the time of CSF HSV PCR specimen collection.

For each patient in the cohort, we determined if the algorithm would have allowed HSV PCR testing. The algorithm deferred patients from HSV PCR testing if they had a normal CSF WBC count (≤5 cells/mm3) and protein (≤50 mg/dl). The algorithm automatically tested patients with known HIV, solid-organ recipients, or <2 years of age.[6] We assessed HIV and solid-organ transplant history through medical record review.

Data obtained from the VA Corporate Data Warehouse (CDW) included the patient's age and all laboratory results. Laboratory results were retrieved by LOINC code, which are assigned in the CDW by dedicated personnel. The VA Healthcare System is the largest integrated health-care system in the United States with 8.92 million Veterans enrolled for care in FY2013. The CDW contains a subset of data from the electronic health records of all VA healthcare facilities, which includes billions of laboratory results.

Assessment of harm

The algorithm was considered to have possibly changed empiric acyclovir management if the following scenario occurred. First, the patient had to test positive for HSV PCR when the algorithm would have led to rejection of the test. Second, the patient had to receive acyclovir from the time of the HSV test order until the HSV PCR result returned. This scenario was adopted from the Expert Panel of the Infectious Disease Society of America which states, “Acyclovir treatment should be initiated in all patients with suspected (herpes simplex) encephalitis, pending results of diagnostic studies.”[11]

The algorithm requires clinical information about the patient's age, HIV status, and history of solid-organ transplant. To determine the risk for harm due to misidentification of persons with HIV or who have had a solid-organ transplant, we assessed the agreement between the admission and discharge notes for these conditions.[12]

Finally, it has been previously suggested that patients with HSV encephalitis have lower CSF WBC counts and protein levels compared to those with HSV meningitis.[5] We compared these parameters within our study cohort after reviewing the medical record of all patients with a positive HSV PCR and excluding those patients with a diagnosis other than meningitis or encephalitis. As a second comparison, we compared the difference in WBCs and protein between eligible and ineligible (age <2 years, HIV, or solid-organ transplant) patients. The Wilcoxon rank-sum test was used for all comparisons (R Statistical Language, 2014). Unlike the t-test, the Wilcoxon rank-sum test does not assume the mean difference between the two populations follows a normal distribution.

This study was approved by the Institutional Review Board of the VA Connecticut Healthcare System.

Results

Between January 1, 2000, and June 30, 2013, 6357 HSV PCR tests with accompanying CSF WBC count and protein level from 5977 unique patients were identified from the VA CDW. The total number of HSV PCR tests excludes tests without a positive or negative result, duplicates, and those without a corresponding CSF leukocyte and protein count [Figure 2]. The patients were cared for at 85 facilities within the VA healthcare system and across the United States. The population had a male predominance (93%) and an average age of 59.7 (standard deviation 14.9, range 18–97) years. Of the total tests, 1.6% (101/6357) tests from 94 unique individuals had a positive result.

Figure 2

Herpes simplex virus in cerebrospinal fluid test selection process. HSV CSF tests were retrieved by LOINC code, a universally accepted code to identify laboratory tests (5013-8, 16952-4, 16960-7, 34655-1, and 32141-4) (n = 14,986). Tests without a result of positive or negative were removed (n = 13,135). Some laboratories report two results per HSV test order, one for HSV-1 and another for HSV-2. We counted these only once (n = 8602). Samples without the CSF white blood cell and protein tests within 2 days of the HSV PCR test were removed (n = 6357). HSV: Herpes simplex virus, CSF: Cerebrospinal fluid, PCR: Polymerase chain reaction

Management of acyclovir

We reviewed the medical care delivered for the 101 episodes of positive HSV PCR results to assess if the application of the algorithm would have changed acyclovir management. Of the 101 positive results, 90 would be tested by the algorithm due to either elevated WBCs or protein level. Of the remaining 11 positive HSV tests, four specimens were from patients either HIV-positive or who had a solid-organ transplant and therefore would be automatically tested by the algorithm. The remaining seven episodes would not have been tested for HSV per the algorithm due to normal CSF WBC and protein. None of these seven patients received acyclovir immediately before or after the HSV PCR test result. Thus, none of the cases by our definition would have changed acyclovir management with the implementation of the HSV screening algorithm. Table 1 contains a case history for these seven patients.

Table 1

Patients deferred from herpes simplex virus polymerase chain reaction by the algorithm with a positive herpes simplex virus polymerase chain reaction result

Documentation of human immunodeficiency virus and solid-organ transplant

Of the 94 unique patients with a positive HSV result, relevant clinical notes were available in the medical records for 87 patients. From these patients, we found 11 with HIV (13%) and 1 patient with a solid-organ transplant (1%). All of the 11 patients with HIV had received a diagnosis before testing positive for HSV. For all 87 patients, the admission and discharge notes contained equivalent documentation of HIV and solid-organ transplant. Thus, no discrepancies that could have led to algorithm misuse were identified.

Cerebrospinal fluid white blood cell count and protein level in herpes simplex virus encephalitis

In review of the 101 positive tests, the clinician recorded a primary diagnosis of meningitis in 53 patients, encephalitis in 21 patients, a more compelling alternative diagnosis (i.e., pneumonia, surgical complication, sagittal sinus thrombosis, and brain metastasis) was identified in 13 patients, and the pertinent medical record notes were unavailable for review in 14 patients. No difference in WBC count (280 vs. 165 cells/mm3, P = 0.055) or protein levels (280 vs. 130 mg/dl, P > 0.10) were found between patients with meningitis compared to patients with encephalitis. Leukocyte counts in patients with HIV or solid-organ transplant demonstrated a statistically significant reduction when compared to the absence of either condition (67 vs. 224 cells/mm3, P = 0.018). However, the difference between protein levels did not reach statistical significance (84 vs. 130 mg/dl, P > 0.058) [Table 2].

Table 2

Comparison of cerebrospinal fluid leukocyte and protein levels between patient groups who tested positive for herpes simplex virus in cerebrospinal fluid

Discussion

At the present time, few clinical laboratories employ a screening algorithm to determine whether or not to perform CSF HSV PCR testing. This may be at least in part due to a limited level of evidence to support the safety of this approach. In this assessment, we found that in a large cohort of Veteran patients, the use of the algorithm would likely not have led providers to have altered their administration of acyclovir and thus use of the algorithm would not have caused harm. The algorithm correctly identified all patients with a positive HSV PCR whose providers prescribed acyclovir from the time of sample collection to the time of an available test result; the treatment approach recommended by current guidelines.[11] The algorithm suggested rejecting PCR for seven patients with a positive HSV PCR. It is notable that none of these seven patients were subsequently considered to have HSV disease as evidenced by their providers not administering acyclovir for a period of treatment. Although it is possible that the decision to withhold acyclovir may represent an error in judgment, the clinicians appeared to have a satisfactory rationale to explain their decisions, and no patient was found with late sequelae concerning for untreated HSV CNS disease by our medical record review. These patients had a normal CSF WBC count, normal CSF protein and often a negative magnetic resonance image (MRI), a compelling alternative diagnosis, and/or clinical recovery [Table 1].

In our experience, the perceived clinical need for a HSV PCR test can dramatically change in the first 24 h of patient care. Some proportion of the prototypical patients tested by HSV PCR due to altered mental status of unknown etiology will become lucid, discharged from the hospital, or have an alternative diagnosis. After 24 h, the progression of the initial presentation, the CSF profile, or an MRI result may reduce the influence of the HSV PCR result on the administration of acyclovir. Despite this, HSV PCR is rarely canceled. By identifying tests with a low probability of altering patient care decisions, the screening criteria fulfill an unmet need.

The algorithm, a combination of clinical and laboratory information, may outperform the current gold standard of HSV PCR alone for the diagnosis of HSV CNS infection. The algorithm and HSV PCR have equivalent specificity by definition when HSV PCR is the gold standard (it cannot return a positive result, except with a positive HSV PCR). The algorithm also has a lower sensitivity than HSV PCR (in a large enough sample, the algorithm will eventually defer a positive HSV PCR. In our large sample, this occurred 7/6357 or 0.1%). However, if the gold standard was instead the physician's treatment of active HSV CNS infection with acyclovir, these seven cases were misclassified by HSV PCR and classified correctly by the algorithm. In this scenario, both the algorithm and HSV PCR have equivalent sensitivity, but the algorithm has better specificity.

Second, if the algorithm is used with the same inclusion criteria as were applied in its development, the user requires knowledge of the patients’ HIV status and solid-organ transplantation history. In our cohort of VA patients, we found this information to be readily available and internally consistent in the medical record through a process of manual chart review. The implementation of a similar process by a clinical laboratory will likely represent a challenge. Manual chart review can be time-consuming, costly, and prone to error.[6] A decision support system in the electronic health record based on the clinician's determination of eligibility may represent a comparatively low-cost, high fidelity solution.

Third, we compared WBC counts and protein levels from CSF between eligible and ineligible patients as well as patients with herpes simplex encephalitis and meningitis. A comparison of encephalitis to meningitis found no statistical difference in WBC counts or protein levels. These laboratory tests may not provide useful information to differentiate HSV PCR positive meningitis and encephalitis. These findings may differ from a previous report due to (1) differences in the clinicians’ classification of encephalitis or meningitis, (2) the previous report had a smaller sample size, (3) population demographics, or (4) differences in the statistical test (unpaired t-test vs. Wilcoxon rank-sum test).[5] In contrast, a statistical difference among eligible and ineligible patients did exist for WBCs. This supports the rationale of the screening criteria to test all patients with HIV and solid-organ transplantation because they have an immunosuppressed state and reduced CSF inflammatory markers compared to nonimmunocompromised patients.

Limitations

The demographics of patients receiving care in the VA Healthcare System differ from most health-care systems. It is predominately male with an older average age. Males generally have an equivalent rate of HSV encephalitis compared to females.[5813] Older age is also a risk factor for hospitalization from encephalitis as well as a poor prognostic marker.[28] Therefore, the cohort in this study has an equivalent to increased risk of harm from herpes simplex encephalitis compared to a general population. The results of this study may not be generalizable to pediatrics because our cohort consisted of adults. The study also relied on the LOINC codes assigned to laboratory tests within the CDW database, which may or may not include all relevant tests.

The algorithm may require further refinement of the patient comorbidities that test for HSV PCR irrespective of CSF WBC count and protein. Currently, patients with HIV or solid-organ transplant receive automatic testing. A previous version of the algorithm included “transplant” patients, which included stem cell, but not corneal or skin, transplant recipients.[6] A future version of the algorithm may allow exclusion of HSV PCR in patients with well-controlled HIV (CD4+ >200 cells/µl). Similarly, additional investigations will be required to determine the safety of the algorithm in other immune compromised conditions (i.e., chemotherapy, primary or secondary immune deficiency). The current definition of the algorithm worked well in our cohort [Table 2].

Future studies may investigate effective ways to implement the algorithm through a standardized set of orders or decision support tools. A reasonable set of orders may contain the ability to request acyclovir, HSV PCR, WBCs, and protein levels in the CSF. It would also collect HIV and transplantation history (i.e., heart, liver, lung, kidney, or stem cell) to automatically perform HSV PCR on these immunosuppressed patients. An explanation of the rationale used by a decision support system could reference the supporting literature.[456] Contact information for the laboratory would also facilitate communication with the clinician to circumvent the algorithm for an individual patient, especially for high suspicion of HSV encephalitis or atypical immunosuppression. The availability of standardized laboratory test data across 85 facilities in a centralized data warehouse made this study possible. The addition of HIV status, transplantation history, and physician overrides of the algorithm to the data warehouse could reduce the cost of manual chart review in future studies.

Conclusion

The algorithm promulgated by Hanson et al.[6] to selectively employ HSV PCR was found to be safe in this large cohort of Veteran patients. It likely has additional advantages as well. It can reliably exclude the diagnosis of herpes simplex infection faster than HSV PCR and in this way reduce the exposure to acyclovir, a nephrotoxin, in patients without HSV CNS infection and prompt the clinician to consider an alternative diagnosis. In situations where acyclovir has a limited supply, it could reserve the drug for patients most likely to benefit. It also saves in HSV PCR costs. Therefore, the algorithm proposed by Hanson et al. may improve patient care while decreasing costs.[6]

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.