A 29-MRNA HOST RESPONSE WHOLE-BLOOD SIGNATURE IMPROVES PREDICTION OF 28-DAY MORTALITY AND 7-DAY INTENSIVE CARE UNIT CARE IN ADULTS PRESENTING TO THE EMERGENCY DEPARTMENT WITH SUSPECTED ACUTE INFECTION AND/OR SEPSIS.

ABSTRACT: Background: Risk stratification of emergency department patients with suspected acute infections and/or suspected sepsis remains challenging. We prospectively validated a 29-messenger RNA host response classifier for predicting severity in these patients.
Methods: We enrolled adults presenting with suspected acute infections and at least one vital sign abnormality to six emergency departments in Greece. Twenty-nine target host RNAs were quantified on NanoString nCounter and analyzed with the Inflammatix Severity 2 (IMX-SEV-2) classifier to determine risk scores as low, moderate, and high severity. Performance of IMX-SEV-2 for prediction of 28-day mortality was compared with that of lactate, procalcitonin, and quick sequential organ failure assessment (qSOFA).
Results: A total of 397 individuals were enrolled; 38 individuals (9.6%) died within 28 days. Inflammatix Severity 2 classifier predicted 28-day mortality with an area under the receiver operator characteristics curve of 0.82 (95% confidence interval [CI], 0.74-0.90) compared with lactate, 0.66 (95% CI, 0.54-0.77); procalcitonin, 0.67 (95% CI, 0.57-0.78); and qSOFA, 0.81 (95% CI, 0.72-0.89). Combining qSOFA with IMX-SEV-2 improved prognostic accuracy from 0.81 to 0.89 (95% CI, 0.82-0.96). The high-severity (rule-in) interpretation band of IMX-SEV-2 demonstrated 96.9% specificity for predicting 28-day mortality, whereas the low-severity (rule-out) band had a sensitivity of 78.9%. Similarly, IMX-SEV-2 alone accurately predicted the need for day-7 intensive care unit care and further boosted overall accuracy when combined with qSOFA. Conclusions: Inflammatix Severity 2 classifier predicted 28-day mortality and 7-day intensive care unit care with high accuracy and boosted the accuracy of clinical scores when used in combination.

INTRODUCTION

Sepsis accounts for significant medical burden worldwide, with 49 million incident cases in 2017 and almost 20% of global reported deaths thought to be attributable to this syndrome (1). Rapid and accurate prediction of sepsis severity remains a critical unmet need with current diagnostic tools (2). Clinicians are tasked with the important role of risk-stratifying individuals to guide clinical management. Most cases of sepsis are admitted to hospitals through emergency departments (EDs), where the need for rapid decision making for optimum clinical outcomes and efficient resource utilization is paramount (3–5). Management decisions, most recently updated in the Surviving Sepsis Campaign (SSC) 2021 international guidelines for management of sepsis and septic shock (6), involve early identification of patients at risk.

Several groups have explored the utility of clinical scoring systems (7–9), laboratory biomarkers (10–13), electronic alert systems (1,5,14,15), and machine learning algorithms (16–19) to improve stratification for sepsis. Despite general advances in this area, current performance and turnaround time of these tools are largely insufficient to support use in routine clinical practice. Of interest, the SSC 2021 guidelines recommend against using quick sequential organ failure assessment (qSOFA) compared with other clinical scores as a single screening tool for sepsis or septic shock (6). The Center of Medicare Services in the United States has mandated the so-called sepsis bundle to improve clinical outcomes in patients with sepsis, which currently also includes measurement of lactate concentrations. However, the accuracy of lactate as a predictor of severity remains unclear (6,8–11).

In this study, we evaluated the use of a host response signature for predicting sepsis severity, Inflammatix Severity 2 (IMX-SEV-2) classifier, which is a part of the underdevelopment TriVerity test for acute infections and sepsis (Inflammatix, Inc., Burlingame, CA). This test quantifies the expression of 29 host messenger RNAs (mRNAs) from whole blood, as this was derived from transcriptomic studies (20–23). Proprietary machine learning classifiers were used to process gene expression level data (24) and to generate three separate, prespecified scores for the likelihoods of (1) bacterial infection, (2) viral infection, and (3) severe outcome. The performance of the algorithms for determining the likelihood of bacterial and viral infection (IMX-BVN-2) has been previously described elsewhere (25) and in the same Greek ED patient cohort (26). The current study therefore focused on the performance of the severity risk score. The original IMX-SEV severity score was developed using transcriptomic datasets from more than 600 patients with community-acquired sepsis and showed a summary area under the receiver operating characteristics (AUROC) of 0.89 (95% confidence interval [CI], 0.56–0.99) for prediction of 30-day mortality in the validation set (21). To further improve performance, we developed a machine learning classifier based on the original score, hereby referred to as IMX-SEV-2. In this study, we validated the 28-day mortality prognostic performance of the IMX-SEV-2 severity score in adults presenting with a clinical syndrome consistent with acute infection and/or sepsis to six EDs in Greece.

PATIENTS AND METHODS

Study design and setting

PROMPT is a prospective, noninterventional, multicenter cohort study performed to assess the diagnostic and prognostic test performance of the heparin binding protein assay in patients admitted to the ED with sepsis and suspected infection. The results of the performance of the heparin binding protein test on 371 patients have recently been published (27). Sampling for gene expression analysis was done in parallel.

Patients were recruited from six ED sites in Greece participating in the Hellenic Sepsis Study Group (NCT 03295825, clinicaltrials.gov) between October 2017 and September 2018. The study was approved by the Ethics Committees of each participating hospital. Study participants were screened for eligibility if admitted to the hospital from the ED. Inclusion criteria were age 18 years or more, written informed consent, and suspicion of infection. Suspicion of infection was based on meeting at least one of the following vital sign abnormalities: temperature of <36°C or >38°C, heart rate of >90 beats per minute, respiratory rate of >20/minute, or self-reported fever or chills. Individuals where a laboratory error occurred or with incomplete clinical data were excluded.

Full demographic and clinical data for each subject were extracted from the electronic medical record and transferred into an electronic data base (Medrio). Data included demographics, clinical (including medical comorbidities), imaging, and laboratory data as well as clinical severity scores (qSOFA and sequential organ failure assessment [SOFA]) and 28-day mortality; mortality was ascertained from review of electronic medical records or, where necessary, by follow-up telephone calls.

Whole blood (2.5 mL) was collected from each enrolled participant into PAXgene Blood RNA tubes (PreAnalytics, Hombrechtikon, Switzerland) and stored at −80°C. Collected PAXgene Blood RNA specimens were shipped on dry ice to Inflammatix (Burlingame, CA). RNA extraction from PAXgene Blood RNA samples was performed using a standardized protocol in batched mode on the QiaCube, as previously described (24). Laboratory tests were performed at the discretion of the treating clinician. Testing included complete blood cell count and differential, biochemistry panel, blood gas, blood and urine cultures, and viral respiratory testing. C-reactive protein and procalcitonin (PCT) testing was performed with high-sensitivity nephelometric assays and KRYPTOR assays, respectively. Lactate was ordered in patients for whom blood gas measurement was considered clinically necessary.

The TriVerity test (Inflammatix Inc.) is based on the detection of 29 target mRNAs derived in multicohort analyses (21,22,28); the identity and biological function of the 29 RNAs has recently been reported (23). In the current study, we applied the IMX-SEV-2 severity score algorithm. The accuracy of a previous version of the classifier, Inflammatix Severity 1, has been described in an intensive care unit (ICU) cohort (28); subsequent improvements lead to the generation of IMX-SEV-2 (29). RNA targets were analyzed using the NanoString nCounter SPRINT Profiler from 150 ng of isolated RNA. The expression of four housekeeping genes (CDIPT, KPNA6, RREB1, YWHAB) was also counted to normalize mRNA counts across samples (24). Laboratory personnel were blinded to clinical outcomes, and the IMX-SEV-2 classifier was directly applied to the NanoString data, blinded to clinical outcomes.

Statistical analysis

Statistical analysis was performed using RStudio software v1.3.1093. Continuous variables are presented with the median and interquartile range. Nominal variables are presented as frequencies. The primary outcome was the prognostic performance of the IMX-SEV-2 severity score expressed as interpretation bands (high severity, moderate severity, low severity). Test performance metrics include the AUROC with associated 95% CI per DeLong method as well as likelihood ratios, sensitivity, and specificity for individual bands (30,31). Lactate cutoff values were defined according to the US Food and Drug Administration–approved measurement values, the SSC guidelines, and the National Institute for Health and Care Excellence recommendations on the risk assessment of individuals with sepsis. Quick SOFA and SOFA scores were used to incorporate clinical judgment in the assessment of biomarker prognostic test performance and to better approximate real-life use of the biomarker data. Procalcitonin was not included as a head-to-head interpretation band comparison with IMX-SEV-2 because of lack of established cutoffs for mortality prediction. Logistic regression modeling was used to assess the accuracy of combining the qSOFA and SOFA scores with the IMX-SEV-2 signature and the lactate result. Subgroup analyses were performed for visual representation of the data, but limited statistical assessment was performed due to the small sample size of the subgroups.

RESULTS

Characteristics of study subjects

We enrolled a total of 400 individuals; two were excluded because of missing clinical information, and one patient was excluded because of a laboratory error during RNA extraction, leaving a total of 397 individuals for analysis (Supplementary Fig. 1, http://links.lww.com/SHK/B492). Patient characteristics have already been presented (27). Briefly, 16.9% patients were immunocompromised, and the median qSOFA score on ED admission was 1 (0–1). A total of 38 individuals (9.6%) died within 28 days from any cause, of whom 37 died in-hospital (1 admitted to ICU) and 1 after hospital discharge.

Prognostic performance of the IMX-SEV-2 severity score

Median IMX-SEV-2 score differed significantly between 28-day survivors and nonsurvivors (Fig. 1). Inflammatix Severity 2 classifier predicted 28-day mortality with an AUROC of 0.82 (95% CI, 0.74–0.90) (Supplementary Fig. 2A, http://links.lww.com/SHK/B492); lactate, PCT, qSOFA score, and SOFA score had AUROCs for prediction of 28-day mortality of 0.66, 0.67, 0.81, and 0.91 respectively (Supplementary Fig. 2B, http://links.lww.com/SHK/B492).

Fig. 1

Inflammatix Severity 1 demonstrates the ability to accurately predict 28-day mortality in patients with sepsis. The distribution of the IMX-SEV-2 severity scores is presented, stratified by the clinical outcome of 28-day mortality (death) versus survival (nondeath). The red dashed horizontal lines indicate the threshold values, which stratify the severity score into three result interpretation bands: high risk, moderate risk, and low severity. The full horizontal lines within each box plot represent the median IMX-SEV-2 score, and all data points are included.

Predetermined cutoffs stratify IMX-SEV-2 scores into three result interpretation bands for clinical actionability

To provide results in a clinically actionable format, IMX-SEV-2 distributes absolute scores into interpretation bands: low, moderate, and high severity. Inflammatix Severity 2 classifier performance was therefore assessed by result interpretation band and compared with lactate. Whereas the overall 28-day mortality of the cohort was 9.6%, the mortality stratified for IMX-SEV-2 bands was 3% (low severity), 18% (medium severity), and 48% (high severity) (Fig. 2A). For ruling out 28-day mortality, the IMX-SEV-2 score low-severity (rule-out) band accounted for 67% of patients and demonstrated a sensitivity of 78.9% (likelihood ratio, 0.3). Four of eight patients who died but had low-severity IMX-SEV-2 scores died of causes other than infections or sepsis (one myocardial infarction, one heart failure, two cancer-related deaths). Among 11 patients with high-severity IMX-SEV-2 scores who survived, we found severe conditions: one received vasopressors, three received blood or plasma transfusions, and an additional three had cancer and chemotherapy. In comparison, established lactate cutoffs of <2, 2 to 4, and >4 mmol/L corresponded with mortality rates of 12%, 16%, and 56%, respectively, and a sensitivity of 41.4% for the <2 mmol/L (rule-out) cutoff (Fig. 2B).

Fig. 2

Prognostic performance of the IMX-SEV-2 severity interpretation bands and lactate concentrations for prediction of 28-day mortality. Test performance is shown as patients in band as well as sensitivity, specificity, and likelihood ratio for each interpretation or concentration band. A, The IMX-SEV-2 severity score stratified by predetermined cutoffs into interpretation bands for the overall cohort and (B) lactate stratified by blood concentration. LR, likelihood ratio.

Composite risk prediction results combining IMX-SEV-2 scores with clinical scores

We then investigated the accuracy of composite risk prediction when combining qSOFA with IMX-SEV-2 interpretation bands or lactate. The combination of high qSOFA and high IMX-SEV-2 scores (rule-in scenario) resulted in a specificity of 99% (likelihood ratio, 21.74) for the prediction of 28-day mortality, higher than the specificities of IMX-SEV-2 or qSOFA when used alone (Fig. 3A); reversely, the combination of low qSOFA and low IMX-SEV-2 scores (rule out scenario) resulted in a sensitivity of 92% (likelihood ratio, 0.11). Importantly, 63% of patients were found in the low qSOFA combined with low IMX-SEV-3 group. The combination of other interpretation band results spread between likelihood ratios of 7.32 and 1.15. The combination of qSOFA with lactate improved specificity of the rule in scenario to 99% (likelihood ratio, 14.4); however, the sensitivity of the rule-out scenario was only 76% (likelihood ratio, 0.36) (Fig. 3B).

Fig. 3

Composite risk prediction accuracies combining IMX-SEV-2 interpretation bands with clinical scores or lactate for predicting 28-day mortality. Accuracy of risk prediction using readouts of patients in band, likelihood ratio, sensitivity, and specificity when combining dichotomous qSOFA scores (low ≤2 vs. high ≥2) with IMX-SEV-2 severity interpretation bands (low, moderate, high) (A) or lactate concentrations (high, ≥4; moderate, 2–4; and low, ≤2 mmol/L (B). Graphical depiction of risk prediction for 28-day mortality comparing pretest probability with sequential integration of dichotomous qSOFA scores and IMX-SEV-2 interpretation bands (C) or lactate concentrations (D). Area under the receiver operating characteristics for qSOFA combined with IMX-SEV-2 or biomarkers lactate or PCT using logistic regression to predict 28-day mortality (E). Area under the receiver operating characteristics for SOFA combined with IMX-SEV-2 or biomarkers lactate or PCT using logistic regression to predict 28-day mortality (F). *The IMX-SEV-2 performance subgroup analysis was restricted to individuals for whom the comparator biomarker data were available. NA, not applicable.

We also investigated the stratification of risk prediction using stepwise integration of qSOFA and either IMX-SEV-2 or lactate (Fig. 3C and D). Applying the combination of qSOFA and IMX-SEV-2 classifier, the pretest probability for mortality is 9.7% (Fig. 3C). Quick SOFA stratified patients into low (4.4%) and high risk (45.1%). However, combining qSOFA with IMX-SEV-2 (low, moderate, and high severity) yielded more granular information: first, patients with low qSOFA and low IMX-SEV-2 exhibited slightly decreased mortality compared with all patients with low qSOFA (1.2% vs. 4.4%), whereas moderate and high IMX-SEV-2 implicated a stepwise increase of mortality risk above the risk defined by only low qSOFA (27.3% and 11.0%, respectively). Patients with high qSOFA had mortality of 45.1%. In combination with low IMX-SEV-2, this risk was decreased to 31.1%; patients with both high qSOFA and IMX-SEV-2 had mortality of 70.0%. In contrast, the combination of lactate with qSOFA does not lead to similar reclassification except for patients with both high qSOFA high lactate whose mortality is 71.4%. These changes are reflected in the increase of the respective AUCs for the prediction of 28-day mortality (Figs. 3E and 3F).

Because a state of immunosuppression may impact the accuracy of host response tests, we determined IMX-SEV-2 results by patient immune status. After removing 67 patients with immunosuppression, there was a nonsignificant increase of AUROC for IMX-SEV-2 from 0.82 (95% CI, 0.74–0.90) to 0.87 (95% CI, 0.80–0.95, P = 0.36 per DeLong test).

Prognostic performance of IMX-SEV-2 for near-term outcome of 7-day ICU care

Inflammatix Severity 2 classifier also accurately predicted the need for ICU care the first 7 days with an AUROC of 0.85 (95% CI, 0.79–0.92) (Fig. 4 and Supplementary Fig. 3, http://links.lww.com/SHK/B492) compared with 0.68 for lactate (95% CI, 0.58–0.79).

Fig. 4

Area under the receiver operating characteristic curve of IMX-SEV-2 for prediction of ICU care within 7 days compared with lactate, PCT, day 0 qSOFA, and day 0 SOFA. *The IMX-SEV-2 performance subgroup analysis was restricted to individuals for whom the comparator biomarker data were available. NA, not applicable.

The sequential integration of qSOFA plus IMX-SEV-2 resulted in a boost of overall accuracy to 99% specificity (qSOFA ≥22 plus high-severity IMX-SEV-2) for ruling in the need for ICU care the first 7 days and a boost to 96% sensitivity (qSOFA <2 plus low-severity IMX-SEV-2) for ruling out the need for ICU care the first 7 days (Fig. 5).

Fig. 5

Accuracy of composite risk prediction using sequential integration of qSOFA plus IMX-SEV-2 interpretation bands (A) or qSOFA and lactate concentrations (B) for prediction of ICU care within 7 days.

DISCUSSION

In this study of 397 individuals with suspected acute infection and/or sepsis from 6 EDs in Greece, the 29-mRNA host biomarker severity score IMX-SEV-2 predicted 28-day mortality with good accuracy and improved the accuracy of qSOFA when used in combination. Similarly, IMX-SEV-2 alone accurately predicted the need for ICU care by day 7 and further boosted overall accuracy when combined with qSOFA. We show here that adding an “immune signature” to a clinical score markedly improves the accuracy of predicting short-term and longer-term outcomes in ED patients.

In the SSC guidelines, blood lactate measurements are recommended for risk prediction, but lactate is considered neither sensitive nor specific enough to rule in or rule out sepsis on its own (6). Thus, the accuracy of host response markers such as the IMX-SEV-2 classifier may improve medical decision making in the ED, including optimizing resource utilization. Although IMX-SEV-2 shows similar performance to qSOFA, the SSC guidelines recommend that a positive qSOFA should not be used as a single screening tool for suspected sepsis (6). Indeed, we note here that the combination of IMX-SEV-2 with qSOFA increased the prognostic performance of qSOFA for 28-day mortality. Importantly, although the combination of IMX-SEV-2 with biomarkers such as lactate or PCT also increased accuracy, the combined accuracy did not reach the accuracy of the combination of qSOFA and IMX-SEV-2.

The SOFA score demonstrated the highest single-score accuracy in our study with an AUROC of 0.91. However, the SOFA score, although part of the most recent definition of sepsis, is not useful in the management of ED patients given the delay to obtain required laboratory values. In addition, sequential data points are needed for highly accurate prognostic performance (7). Of importance, immunosuppression (17% of the study population) did not significantly impact performance of IMX-SEV-2; ongoing and future studies are focusing on this question.

The IMX-SEV-2 classifier also has the advantage of stratifying patients into highly actionable severity interpretation bands based on preset cutoffs rather than providing a single cutoff. Only 3% of patients for whom the IMX-SEV-2 classifier showed a low-severity result died resulting in a sensitivity of 78.9% for “rule-out” decisions; 67.0% of all patients fell into this band. Of interest, death in half of the patients with low-severity scores was due to their underlying disease (e.g., cancer) rather than of an acute infection or sepsis-related event, an observation we previously reported in an ICU cohort (23). In comparison, almost 50% of the patients in the high-severity band died, resulting in a specificity of 96.9% for clinically actionable “rule-in decisions.”

Inflammatix Severity 2 classifier is being developed as part of the TriVerity test with a 30-minute turnaround time (32) for highly accurate and immediately actionable results. The need for improved sepsis mortality prediction has been voiced both for clinical trials research and for clinical practice and has stimulated research on biomarker development (33). Although biomarkers have shown varying levels of success for this purpose, efforts have been hindered by the need for repeat testing, longer turnaround times than required for rapid clinical decision making, and lack of reproducibility or availability of testing. Rapid lactate measurements are almost universally available but typically require sequential measurements, and the utility of lactate as an independent prognostic indicator for mortality in sepsis has not been clearly established (11,34). Procalcitonin is currently Food and Drug Administration approved as an aid for 28-day all-cause mortality prediction based on the difference between two measurements performed on day 0 or 1 and day 4 after a clinical diagnosis of sepsis (35). However, PCT shows limited clinical utility when used as a standalone test for prognosis (13,36), and the SSC guidelines do not recommend its use for either risk prediction or to guide initiation of antibiotic therapy (6). A rapid gene expression–based severity score may therefore provide an important advantage given its use of a single measurement at the time of patient presentation, resulting in more rapidly actionable results.

This study benefited from several strengths, including its prospective and multisite design and large sample size with a well-characterized cohort representative of European ED cohorts. Furthermore, it presents data that support the promising performance of a single-measurement assay that can be adapted for point-of-care use. However, the following limitations should be acknowledged: (a) lactate and PCT measurements were not available for all study participants. Nonetheless, we included an assessment of IMX-SEV-2 limited to the subgroup of individuals with available results, and (b) missing data points limited definitive assessment of test performance for different biomarkers and among subgroups. Studies that incorporate multiple data points per individual in early sepsis (monitoring) are under way and will help characterize the kinetics of IMX-SEV-2 and expand understanding of its prognostic utility.

In summary, IMX-SEV-2 measured from a single blood draw at ED admission accurately predicted 28-day mortality and the need of ICU care the first 7 days while it increased the accuracy of clinical scores, including qSOFA and SOFA. In combination with clinical judgment, IMX-SEV-2 may provide a highly accurate and actionable tool to guide clinical management for improved patient outcomes and optimal utilization of hospital resources.