Performance of Diagnostic Model for Differentiating Between COVID-19 and Influenza: A 2-Center Retrospective Study.

BACKGROUND COVID-19 and influenza share many similarities, such as mode of transmission and clinical symptoms. Failure to distinguish the 2 diseases may increase the risk of transmission. A fast and convenient differential diagnosis between COVID-19 and influenza has significant clinical value, especially for low- and middle-income countries with a shortage of nucleic acid detection kits. We aimed to establish a diagnostic model to differentiate COVID-19 and influenza based on clinical data. MATERIAL AND METHODS A total of 493 patients were enrolled in the study, including 282 with COVID-19 and 211 with influenza. All data were collected and reviewed retrospectively. The clinical and laboratory characteristics of all patients were analyzed and compared. We then randomly divided all patients into development sets and validation sets to establish a diagnostic model using multivariate logistic regression analysis. Finally, we validated the diagnostic model using the validation set. RESULTS We preliminarily established a diagnostic model for differentiating COVID-19 from influenza that consisted of 5 variables: age, dry cough, fever, white cell count, and D-dimer. The model showed good performance for differential diagnosis. CONCLUSIONS This initial model including clinical features and laboratory indices effectively differentiated COVID-19 from influenza. Patients with a high score were at a high risk of having COVID-19, while patients with a low score were at a high risk of having influenza. This model could help clinicians quickly identify and isolate cases in the absence of nucleic acid tests, especially during the cocirculation of COVID-19 and influenza. Owing to the study's retrospective nature, further prospective study is needed to validate the accuracy of the model.

Background

The COVID-19 pandemic is on-going [1]. The strong infectivity and the unknown characteristics of this disease impose a tremendous burden on health systems worldwide [2]. The availability of SARS-CoV-2 vaccine in several countries brings hope for ending the pandemic [3]. The key for the control and prevention of COVID-19 will be transferred from high-income countries to low- and middle-income countries (LMICs) due to global inequity in COVID-19 vaccination. For LMICs, reducing the number of new cases of COVID-19 is crucial for the prevention and control of the pandemic. Affordable, effective, and fast diagnosis is still lacking [4].

When COVID-19 emerged, influenza was still widespread [5]. Influenza is the most common respiratory tract disease in the autumn and winter [6], and it can be transmitted via the respiratory route and through direct contact [7]. The majority of patients present with mild-to-moderate respiratory symptoms, but some may progress to severe complications [8]. COVID-19 and influenza share many similarities including transmission and clinical manifestations such as fever, dry cough, sore throat, headache, myalgia, and weakness [9,10], and these similarities may easily lead to misdiagnosis of the 2 diseases. For example, some patients who had influenza-like symptoms had positive test results for SARS-CoV-2 in a study at the Los Angeles Medical Center [11]. Failure to distinguish patients with COVID-19 from those with influenza may significantly increase the risk of cross-infection and promote COVID-19 transmission. In addition, treatments for COVID-19 and influenza are very different, and misdiagnosis could result in treatment delay and unnecessary wastage of medical resources. Rapid differential diagnosis between COVID-19 and influenza remains a great challenge and is important for the next phase of outbreak control.

Nucleic acid detection is the diagnostic criterion standard for COVID-19 and influenza infection [12,13]. However, it requires expensive reagents, such as RNA extraction kits, and quantitative polymerase chain reaction (PCR) machines [13]. The costs of nucleic acid detection and rapid molecular assays are unaffordable for LMICs. Rapid molecular assays (nucleic acid amplification tests) and rapid influenza diagnostic tests are priorities for influenza diagnosis, which may be insufficient for those countries and areas [14,15]. Meanwhile, the COVID-19 pandemic causes economic devastation and worsens poverty [16]. Thus, the pandemic trend in poorer countries is unclear and unpredictable due to the shortage of nucleic acid detection kits and critical medical equipment [17-19]. A complementary method for these preferred diagnostic approaches is needed, and our diagnostic model based on clinical data can be used as a rapid and inexpensive method.

Differential diagnostic models based on clinical features and routine laboratory testing have important clinical significance for pneumonia classification. Sampedro et al [20] created a model based on electronic health records that could predict COVID-19 severity and demonstrated a fair to good fit. Formica et al [21] used 8 complete blood count predictors and established a scoring system for COVID-19 with a discriminatory power of 88% based on the area under the receiver-operating characteristic (ROC) curve (AUC). This system waives the need for swab tests for asymptomatic patients with a low score. A model combining clinical features and inflammatory markers has proven to be useful for predicting the bacterial etiology of community-acquired pneumonia, with a specificity of 96.6% [22,23]. Several biomarkers such as monocyte count, percentage of basophils, and time from symptom onset to hospital admission were significantly different between COVID-19 and influenza, which could help the differential diagnosis [24-29]. These investigations were helpful in the differential diagnosis and elucidated the differing pathogenesis of the 2 diseases. However, some studies were single-center studies with relatively small sample sizes or were based on single-dimension data such as biochemical indicators or clinical features [26,27]. Also, patients involved in another 2 studies were recruited from the Emergency Department, leading to selection bias [24,25]. Larger sample sizes, multidimensional data, and more representative patients are needed for a definitive conclusion.

The purpose of the current study was to build a clinical diagnostic model that can differentiate COVID-19 and influenza. This model was based on multidimensional clinical data from 2 hospitals, and it was intended to serve as a fast-screening method to ease the diagnostic difficulties due to insufficient medical supplies and help physicians quickly determine different isolation measures.

Material and Methods

Ethics

The study was performed according to the principles of the Declaration of Helsinki and was approved by the First Affiliated Hospital to Army Medical University Ethics Committee (Approval No. KY2020036).

Study Design and Patients

This study was designed to build a diagnostic model for differentiating between COVID-19 and influenza. The model was based on epidemiology and the clinical features of hospitalized patients with COVID-19 and influenza. All patients came from 2 hospitals under supervision of China National Health Commission (NHC), including Huangshi Central Hospital, Affiliated Hospital of Hubei Polytechnic University (from January 2019 to March 2020), and the First Affiliated Hospital of Army Medical University (from January 2019 to March 2020). These hospitals are large university teaching hospitals in the southwest region of China for the population of that area. A total of 493 eligible patients were enrolled in the study, including 282 COVID-19 patients and 211 influenza patients.

Selection Criteria

Inclusion criteria for the study were adult patients diagnosed with COVID-19 or influenza. COVID-19 was diagnosed according to the Diagnosis and Treatment Protocol for COVID-19 by the NHC of the People’s Republic of China (trial seventh edition) [30], and the diagnosis of influenza was based on Diagnosis and Treatment Protocol for Influenza from the NHC of China [31]. All the enrolled patients with COVID-19 had positive SARS-CoV-2 PCR results, negative PCR results for influenza, and negative test results for other respiratory viral infections. Influenza patients had positive PCR results for the influenza virus or positive influenza antigen detection, as well as negative results for other respiratory viral infections. Exclusion criteria included patients younger than 18 years of age, pregnant or lactating women, patients with other respiratory infections such as parainfluenza and respiratory syncytial virus, and patients with incomplete clinical data.

Data Collection

Medical information including demographic data, clinical features, and laboratory test results were collected from medical records and reviewed respectively. Respiratory specimens from patients were collected to identify the SARS-CoV-2 infection [32]. Respiratory specimens or blood samples were collected to identify influenza infection using PCR tests or influenza antigen detection [33]. All symptoms and laboratory data were collected simultaneously with pathogen or antibody tests on the first day of hospital admission unless indicated otherwise.

Statistical Analysis

The COVID-19 and influenza groups were each randomized into development and validation sets respectively. The development sets were used to establish a diagnostic model, and the validation sets were used to evaluate it. Based on the development set data, all the statistically significant variables (P<0.05) were regarded as candidates for the diagnostic model. Multivariate logistic regression analysis was applied to screen the possible predictive markers of the model, the risk score equation was built based on the β-coefficients of the regression analysis, and the score of each subject was then calculated. ROC analysis was employed to evaluate the model and to determine the optimal cutoff value for distinguishing COVID-19 from influenza. For model validation, we also calculated the sensitivity, specificity, positive predictive value, negative predictive values, and accuracy of the predicted probabilities in the validation sets. Statistical analyses were performed with SPSS 25.0 (SPSS, Chicago, IL, USA)

Results

Demographic and Clinical Features

As shown in Table 1, a total of 493 eligible patients were enrolled, including 282 (57%) COVID-19 patients and 211 (43%) influenza patients. Numerous differences were identified between the COVID-19 group and the influenza group. Patients with COVID-19 were significantly older (mean age, 49.6±14.4 years vs 42.7±16.4 years; P<0.001). A history of hypertension (COVID-19: 9.9% vs influenza: 22.0%, P<0.001), diabetes mellitus (COVID-19: 7.1% vs influenza: 17.5%, P<0.001), and cardiovascular disease (COVID-19: 8.2% vs influenza: 13.7%, P=0.045) were significantly different between the 2 diseases. In addition, the COVID-19 patients were more prone than the influenza patients to present fever (209 [74.1%] vs 101 [47.9%]; P<0.001), dry cough (193 [68.4%] vs 64 [30.3%]; P<0.001), fatigue (86 [30.5%] vs 10 [4.7%]; P<0.001), myalgia (24 [8.5%] vs 2 [0.9%]; P<0.001), and dyspnea (19 [6.7%] vs 5 [2.4%]; P=0.033). However, the patients with influenza more often experienced faster pulses (87±12 beats/min vs 96±17 beats/min; P<0.001). The mean time from illness onset to hospital admission in COVID-19 patients was shorter than the median time in influenza patients (11.9±5.3 days vs 15.2±4.4 days; P=0.024)

Table 1

Demographic features of patients with COVID-19 and influenza.

Variables	COVID-19 (n=282)	Influenza (n=211)	P value
Age, y	49.6±14.4	42.7±16.4	<0.001*
Sex, Male: Female	134: 148	120: 91	0.045*
Comorbidity, n (%)
Hypertension	28 (9.9)	46 (22.0)	<0.001*
Diabetes mellitus	20 (7.1)	37 (17.5)	<0.001*
Cardiovascular disease	23 (8.2)	29 (13.7)	0.045*
Liver disease	5 (1.8)	2 (0.9)	0.475
Cancer	4 (1.4)	1 (0.5%)	0.391
Signs and symptoms, n (%)
Fever	209 (74.1)	101 (47.9)	<0.001*
Dry cough	193 (68.4)	64 (30.3)	<0.001*
Fatigue	86 (30.5)	10 (4.7)	<0.001*
Headache	22 (7.8)	8 (3.8)	0.086
Myalgia	24 (8.5)	2 (0.9)	<0.001*
Dyspnea	19 (6.7)	5 (2.4)	0.033*
Nausea	5 (1.8)	7 (3.3)	0.377
Diarrhea	16 (5.7)	5 (2.4)	0.113
Pulse median, beats/min	87±12	96±17	<0.001*
Time from illness onset to hospital admission, d	11.9±5.3	15.2±4.4	0.024*

P<0.05.

Laboratory Features

As with the epidemiological characteristics, many differences in laboratory tests were also found between patients with COVID-19 and those with influenza (Table 2). In terms of routine blood test results, decreased levels of white blood cells (WBCs) ([5.2±2.2]×109/L vs [7.6±3.9]×109/L; P<0.001) and neutrophils ([3.5±2.1]×109/L vs [5.6±3.7]×109/L; P<0.001) were identified in the COVID-19 patients. In addition, lower levels of prothrombin time (11.8±2.6 s vs 12.5±2.4 s; P=0.020) and D-dimer (0.4±0.9 mg/L vs 1.8±1.2 mg/L; P<0.001) were found in the COVID-19 group compared with the influenza group. Moreover, decreased concentrations of alanine aminotransferase (AST) (P=0.005), aspartate aminotransferase (ALT) (P=0.001), bilirubin (P<0.001), C-reactive protein (P<0.001), and creatinine (P=0.005) were confirmed in COVID-19 patients; the influenza patients showed opposite trends. However, the influenza group presented lower concentrations of potassium (P<0.001) and sodium (P=0.003), and the activated partial thromboplastin time (aPTT) was also reduced (P<0.001). No obvious differences were found regarding other indicators.

Table 2

Initial laboratory indices of patients with COVID-19 and influenza.

Variables	COVID-19 (n=282)	Influenza (n=211)	P value
White blood cell count, ×109/L	5.2±2.2	7.6±3.9	<0.001*
Eosinophil cell count, ×109/L	0.1±0.3	0.1±0.8	0.031*
Neutrophil count, ×109/L	3.5±2.1	5.6±3.7	<0.001*
Lymphocyte count, ×109/L	1.3±2.6	1.3±0.8	0.066
Hemoglobin, g/L	133.9±24.3	131.8±22.0	0.410
Platelet count, ×109/L	187.1±67.7	195.7±85.8	0.098
Prothrombin time, s	11.8±2.6	12.5±2.4	0.020*
Activated partial thromboplastin times, s	36.4±6.5	30.5±6.5	<0.001*
D-dimer, mg/L	0.4±0.9	1.8±1.2	<0.001*
Albumin, g/L	40.0±6.2	37.0±7.3	<0.001*
Alanine aminotransferase, U/L	31.2±28.4	76.5±166.9	0.001*
Aspartate aminotransferase, U/L	33.5±30.5	68.3±117.6	0.005*
Bilirubin, μmol/L	11.2±8.5	26.4±63.7	<0.001*
Potassium, mmol/L	4.2±0.5	4.0±0.6	<0.001*
Sodium, mmol/L	139.9±3.7	138.9±4.4	0.003*
Creatinine, μmol/L	70.1±31.7	87.5±80.7	0.005*
C-reactive protein, mg/L	22.3±23.2	49.9±62.4	<0.001*
Creatine kinase, U/L	129.6±183.4	220.3±369.0	0.176
Lactate dehydrogenase, U/L	257.2±111.8	376.2±391.3	0.097

P<0.05.

Diagnostic Model for Distinguishing Between COVID-19 and Influenza

To establish a diagnostic model, we randomly assigned COVID-19 patients (n=282) to a development set (n=226) and a validation set (n=56). Influenza patients (n=211) were also split into a development set (n=168) and a validation set (n=43). The mean age (46.6±15.6 years vs 46.8±15.8 years, P=0.88) and sex ratio (men: women, 206: 188 vs 48: 41, P=0.57) were not significantly different between the development and validation sets. We then used the development sets of the COVID-19 group and influenza group to develop a diagnostic model and used the validation sets of the 2 groups to validate the model.

All the statistically significant variables (P<0.05) were regarded as candidates for the diagnostic model. Using univariate logistic regression analysis, 5 variables, including age, fever, dry cough, D-dimer, and WBC count, had high diagnostic values and were chosen as diagnostic model markers. An equation based on the risk scores was established according to the regression coefficients, and the predictive possibility of having COVID-19 was estimated. The score of each study subject was calculated, with a higher score indicating a greater possibility of COVID-19 and a lower score indicating a greater possibility of influenza.

Codes used for the equations:

“Present or positive=1” or “absent or negative=0”

Age=year of admission–year of birth

Dry cough, age, and fever were distinguishing factors of COVID-19, whereas D-dimer and WBC count were distinguishing factors of influenza. ROC analysis indicated the AUC of the model was 0.914 (95% confidence interval [CI], 0.882–0.948) (Figure 1A). When the cutoff value was 0.36, the sensitivity, specificity, and accuracy were 70.8%, 89.4%, and 83.9%, respectively (Table 3).

Figure 1

Receiver-operating characteristic (ROC) curve showing the predictive ability of the diagnostic model in (A) the development set and (B) the validation set. AUC – area under the ROC curve.

Table 3

Sensitivity, specificity, accuracy, and predictive values of the 5-marker COVID-19 risk score by data set*.

Data set	Sensitivity	Specificity	Accuracy	Predictive values
				Positive	Negative
Development set	70.8%	89.4%	83.9%	87.9	73.9
Validation set	73.3%	87.5%	83.0%	87.5	73.3

Cutoff point for predictable diagnosis of COVID-19 was >0.36.

Validation of the Diagnostic Model

We further validated the diagnostic model based on the validation data set. As shown in Figure 1, ROC analysis showed the AUC of the model for the validation set was 0.901 (95% CI=0.819–0.971), and there were no striking differences in AUC between the development set (0.914, 95% CI=0.882–0.948) and validation set (0.901, 95% CI=0.819–0.971; P=0.112). Based on the ROC analysis, the diagnostic model had good distinction power in the 2 data sets (Figure 1). When the cutoff value was 0.36, the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of the validation set were 0.733, 0.875, 0.875, 0.733, and 0.830, respectively (Table 3).

Discussion

In this study, we developed a COVID-19 predictive model based on multidimensional medical data, including epidemiological history, clinical symptoms, and laboratory indices from 2 hospitals. We then verified the accuracy of the model using a validation set. Five markers that were critical for differentiating the 2 diseases were identified, including age, dry cough, fever, WBC count, and D-dimer. We also found other indicators that may have a certain significance for differential diagnosis such as AST, aPTT, fatigue, headache, creatinine, and dyspnea. These abnormalities revealed that the damage of SARS-CoV-2 infection may involve cellular immune response, coagulation disorder, and organ injury. It is also believed that a cytokine storm may play an important role [34].

Numerous methods have been developed for COVID-19 diagnosis. Among these, whole-genome sequencing was reported to have high specificity and sensitivity, but the high cost and time-consuming nature limited clinical application [12]. Single-cell sequencing was also used to distinguish between the 2 diseases. Liu et al [35] reported the single-cell transcriptional landscape of peripheral blood mononuclear cells in COVID-19 patients and influenza A virus patients; it showed distinct immune response pathways in the 2 diseases, but this method currently is limited to scientific research. The utility of clinical signs, symptoms, and serum biomarkers could discriminate bacterial from viral pneumonia with a sensitivity of 75% and a specificity of 84% [36]. Although this approach was accurate, it was not applied in community hospitals owing to the need for large computational resources. By being fast and simple, our model is suitable for community hospitals and LMICs with shortages of pathogenic detection kits and critical medical equipment, especially during the pandemic.

Our study built this predictive model for differentiating COIVID-19 and influenza. In a previous study, Zhang et al [27] compared peripheral blood parameters of COVID-19 patients and influenza patients and found some differences, such as older age and reduced WBC counts for COVID-19 patients, which were consistent with our study. One study by Sieber et al [24] also contrasted the 2 categories of patients from the Emergency Department and showed that the most important difference was a longer latency from symptom onset to hospital admission for COVID-19 patients. This was different from our study, and this difference could be attributable to the studies having different study populations. COVID-19 prevention and control strategies varied between countries. The strict quarantine measures and screening policies in China ensured that COVID-19 patients could get the earliest identification and medical treatments during the pandemic, which made the time from symptom onset to hospital admission shorter. Alternatively, some differences that exist among the current studies may be due to the use of different sample sizes [25,26]. Future studies with larger sample sizes could be done to validate the results of this study. Although the previous studies presented the epidemiological characteristics, clinical features, and laboratory indicators for COVID-19 and influenza, they did not build a comprehensive predictive model from those indices. The noncomprehensiveness of these indicators resulted in low diagnostic performance, and results could not be applied in clinical practice. Jung et al [37] utilized a similar computational approach to develop a diagnostic model to effectively differentiate Crohn disease from intestinal tuberculosis [37]. We hope our model established by the use of this method offers a reliable and practical approach to distinguish COVID-19 and influenza.

We found that age was an important factor in distinguishing patients with COVID-19 from those with influenza. COVID-19 patients were older than influenza patients. The influenza A virus predominantly affects children and young adults more than older adults, and although the death rates are highest in the >65 years group, overall, more deaths are reported in younger people [38-40]. Influenza B viruses also tend to preferentially affect children and young adults, and significant mortality has been confirmed, particularly in the pediatric population [41,42]. The possible reasons for this pattern are that older patients may have a cross-reactive antibody that confers protection to the current influenza strain [38]. According to the recent largest sample study of COVID-19, most patients were 30 to 79 years old (87%) and 3% were 80 or older [43]. A joint investigation report on COVID-19 released by the World Health Organization on February 29, 2020, revealed that the median age of infected patients was 51 years, and 78% of patients were between 30 and 69 years [44]. The data demonstrated that chronic disease, a weak immune system, and poor overall health may increase the susceptibility to SARS-CoV-2 infection.

Fever was one of the markers in the predictive model. Patients with COVID-19 were more prone to have fever compared with those with influenza, which was consistent with a previous study [25]. According to Zhong et al [45], about half of the patients had fever on initial presentation and fever developed in 87.9% of patients following hospitalization. We speculated that the majority of COVID-19 patients were older and had chronic comorbidities and weakened immunity, the virus replicated rapidly in vivo and activated the immune system, and then fever occurred. Previous findings support our hypothesis [46,47]. In the early stage of COVID-19, a normal or decreased total WBC count was a characteristic of viral infection [48]. Compared with influenza, the WBC count being lower with COVID-19 may be due to immune system damage and cytokine storm.

Our study has some limitations. First, studies with expanded samples and prospective clinical trials are required for further validation and higher accuracy. Second, the patients in this study came from the southwest region of China, and the results need to be interpreted with caution. Third, all patients enrolled in this study were inpatients rather than outpatients, which may lead to selection bias. Our model would be more applicable to inpatients.

Conclusions

We built a diagnostic model for distinguishing COVID-19 from influenza. The markers for distinguishing COVID-19 from influenza were age, fever, dry cough, WBC count, and D-dimer. This model will assist clinicians in distinguishing the 2 diseases and accelerate diagnosis and quarantine, and it can even serve as a fast-screening method in community hospitals and LMICs with insufficient testing supplies. All findings should be interpreted with caution, and this model should be validated in larger prospective cohort studies.