Definition and retrospective application of a clinical scoring system for COVID-19 triage at presentation.

BACKGROUND: A simple scoring system for triage of suspected patients with COVID-19 is lacking.
METHODS: A multi-disciplinary team developed a screening score taking into account epidemiology history, clinical feature, radiographic feature, and routine blood test. At fever clinics, the screening score was used to identify the patients with moderate to high probability of COVID-19 among all the suspected patients. The patients with moderate to high probability of COVID-19 were allocated to a single room in an isolation ward with level-3 protection. And those with low probability were allocated to a single room in a general ward with level-2 protection. At the isolation ward, the screening score was used to identify the confirmed and probable cases after two consecutive real-time reverse transcription polymerase chain reaction (RT-PCR) tests. The data in the People's Hospital of Changshou District were used for internal validation and those in the People's Hospital of Yubei District for external validation.
RESULTS: We enrolled 76 and 40 patients for internal and external validation, respectively. In the internal validation cohort, the area under the curve of receiver operating characteristics (AUC) was 0.96 [95% confidence interval (CI): 0.89-0.99] for the diagnosis of moderate to high probability of cases among all the suspected patients. Using 60 as cut-off value, the sensitivity and specificity were 88% and 93%, respectively. In the isolation ward, the AUC was 0.94 (95% CI: 0.83-0.99) for the diagnosis of confirmed and probable cases. Using 90 as cut-off value, the sensitivity and specificity were 78% and 100%, respectively. These results were confirmed in the validation cohort.
CONCLUSION: The scoring system provides a reference on COVID-19 triage in fever clinics to reduce misdiagnosis and consumption of protective supplies.The reviews of this paper are available via the supplemental material section.

Introduction

At the end of December 2019, a novel coronavirus-infected disease (COVID-19) was identified in Wuhan, China. Person-to-person transmission has been demonstrated at the early stage.[1,2] With the spread of the epidemic, the disease has also been found in other parts of China and other countries. The WHO declared the outbreak of COVID-19 a global pandemic on 11 March 2020. As of 15 March 2020, 153,517 cases of COVID-19 had been confirmed globally and 5735 deaths had been reported.[3] Thus, COVID-19 heavily endangered public health and has brought severe economic burden.

The main clinical features of COVID-19 were fever, cough and shortness of breath.[4-6] However, diarrhea as an initial symptom has been reported in some cases.[7,8] In addition, special radiographic features on chest computed tomography (CT) have been reported.[9-11] No chest CT abnormality was reported only in 17.9% of non-severe cases and 2.9% of severe cases.[6] The real-time RT-PCR assay was the standard method to diagnose COVID-19. However, false negative has been reported in some cases.[12,13] As COVID-19 is an infectious disease, how to improve the diagnosis of COVID-19 is important.

As COVID-19 is a new infectious disease, physicians engaged in the fever clinic have no experience on how to diagnose this disease. Whether to allocate the suspected patients who require hospitalization to a general ward or to an isolation ward is a difficult decision. The protective level, supplies of equipment and requirement of staff in an isolation ward were much higher than in a general ward. If patients with low probability of COVID-19 were allocated to isolation rooms, it would increase the shortage of medical resources. If a patient with high probability was allocated to a general ward, it would increase the risk of nosocomial transmission. Therefore, we aimed to develop a screening score to help the physicians to allocate the suspected patients at fever clinics, and, further, to use this screening score to identify confirmed or probable cases in isolation wards.

Methods

Patients

This was a retrospective observational study performed in two hospitals of Chongqing, China (the People’s Hospital of Changshou District and the People’s Hospital of Yubei District). The suspected patients with COVID-19 between 18 January and 29 February in the two hospitals were enrolled. According to the management protocol made by the Health Commission of Chongqing, the suspected patients at fever clinics with low probability were allocated to a single room in a general ward with level-2 protection. The suspected patients with moderate to high probability were allocated to a single room in an isolation ward with level-3 protection. Level-2 protection included disposable hat, disposable gown, disposable gloves and N95 respirator. Level-3 protection included disposal coverall with hood, surgical gloves, eye shield or face shield, shoe covers and N95 respirator (for invasive procedures, such as intubation, powered, air-purifying respirator can be used).

Two consecutive RT-PCR assays were given for the suspected patients who needed hospitalization. Nasopharyngeal and oropharyngeal swabs was obtained at the fever clinic for the first test of SARS-CoV-2. The second test was performed on the next day of hospitalization. If both tests were positive, the patient was confirmed. If only one test was positive, the patient was classified as a probable case. In the general ward, COVID-19 was excluded and the patient was treated as an ordinary case after two negative RT-PCR tests. In the isolation ward, however, if the two tests were negative, a team (at least three professional physicians) discussed whether the patient could be excluded. If the patient was still probable, he or she was treated as a confirmed case in a single room in an isolation ward (multiple tests can be performed at the attending physician’s discretion). Finally, all the confirmed cases had centralized treatment in isolation wards (most of them in multi-person rooms). The study protocol was approved by our ethical committee (the People’s Hospital of Changshou District, No. 2020.3.1). As this is a retrospective observational study, informed consent was waived.

Definition of probability of COVID-19

The stratification of the suspected patients was based on the epidemiological history and clinical features made by our National Health Commission[14] (a detailed description is given in the Supplemental Material online). Low probability of COVID-19 was defined as no epidemiological history but combination of one or two clinical features. Moderate to high probability of COVID-19 was defined as presence of any one item of epidemiological history and combination of any two clinical features, or no epidemiological history but combination of all clinical features. Probable COVID-19 was defined as negative RT-PCR tests (consecutive two times) but combination of epidemiological history and typical chest CT findings.

Development of the screening score

Based on a multi-disciplinary team (the experts in epidemiological, respiratory, infectious, and radiological departments) and current literature about COVID-19, we developed a screening score for COVID-19 triage. The screening score took into account epidemiology history, clinical features, radiographic features, and routine blood tests (Table 1). The highest value was 40 in epidemiology history, 10 in clinical features, 80 in radiographic features, and 20 in routine blood tests. Therefore, the highest summary value was 150.

Table 1.

Screening score for patient triage.

Epidemiology history (14 days before the onset of illness)	Value	Remarks
(1) Travelers from Wuhan	40	Total value ⩽40 points
(2) Travelers from Hubei Province (except Wuhan)	30
(3) Travelers from other parts of China (outside Hubei Province)	20
(4) Has been to hospital or clinic prior to the onset of illness	20
(5) Has been to public places of the local city	5
(6) Has taken public transport in the city	5
(7) Has had close contact with confirmed/suspected cases without effective protection	40
(8) Has had close contact with people back from Wuhan without effective protection	30
(9) Two or more family members “catch a cold”	30
(10) Has confirmed/suspected cases in working area/living community	20
(11) Attend group activities (e.g. dining or meeting)	10
(12) Working during outbreaks	10
Clinical features	Value	Remarks
(1) Fever	5	Total value ⩽10 points
(2) Cough/expectoration/dyspnea	5
(3) Conjunctivitis	2.5
(4) Nausea/vomiting/diarrhea	2.5
(5) Fatigue/poor appetite/muscle soreness/joint pain/headache/sore throat/runny nose	2.5
Radiographic features	Value	Remarks
(1) NCP typical	80	Total value ⩽80 points
(2) NCP probable	60
(3) NCP possible	40
(4) NCP do not exclude	20
(5) Other chest CT imaging change/normal CT	10
Routine blood test	Value	Remarks
(1) Normal leukocyte count (4–10 × 109/L)	5	Total value ⩽20 points
(2) Low leukocyte count (<4 × 109/L)	10
(3) Normal lymphocyte count (0.8–4 × 109/L)	5
(4) Low lymphocyte count (<0.8 × 109/L)	20
(5) Low platelets count (<100 × 109/L)	10

CT, computer tomography; NCP, novel coronavirus pneumonia.

The weight in each variable was assigned as follows. First, 86.2% of confirmed patients with COVID-19 showed typical chest CT findings, which showed highest specificity among all the tests.[4-6,9-11] Therefore, radiographic features were assigned highest weights. Second, COVID-19 has the feature of person-to-person transmission. Among the confirmed patients, 72.3% of cases reported epidemiological history about Wuhan and other high-risk area.[6] So the weight in epidemiological history was the second-highest. The specificity was low in clinical features and routine blood tests as it was not so easy to distinguish COVID-19 from common pneumonia, common cold, mycoplasma chlamydia infection, and other virus infection. As the routine blood test is prior to clinical features to distinguish bacterial pneumonia and virus infection, the weight in routine blood test was higher than that in clinical features.[15] After our multi-disciplinary team discussion and combination of current literature, the score was assigned as 80, 40, 20, and 10 in radiographic feature, epidemiological history, routine blood tests, and clinical feature, respectively.

At the early stage, the epidemiological history was that the patients or their families had recently visited Wuhan or had contact with confirmed cases.[5,6] However, when the community transmission spreading occurred, some patients were infected due to taking public vehicles or traveling to public places; the definite evidence could not be determined in some cases.[16] So we referenced the World Health Organization (WHO) recommendations and the frequency of high-risk factors; we enrolled 12 items of epidemiological history and assigned a different score in each item.[5,6,16-19]

Among the patients with COVID-19, the most common clinical features were fever, cough and shortness of breath, and the uncommon features were runny nose, headache, fatigue, conjunctivitis and diarrhea.[6] Based on these characteristics, we listed five items of clinical features and scores were assigned in each item.

Chest CT is superior to X-ray to identify COVID-19 patients at the early stage[9-11,20] so the chest CT was recommended by our health commission. The score of the chest CT was graded by the radiologist. Two radiologists with more than 10 years of experience independently reviewed the chest images and resolved discrepancies by consensus. The radiographic features were classified as five types and score was assigned based on the weights and our experience (Table 1). As no CT abnormality was found in 17.9% of COVID-19 patients, a score of10 was assigned in this group. Detailed classification of radiographic features in chest CT is given in the Supplemental Material.

The score of routine blood tests was assigned as follows. Low lymphocyte count was present in 83.2% of confirmed cases.[6] Low leukocyte count was found in one-third of confirmed cases as well as low platelets count. However, normal leukocyte count or lymphocyte count did not rule out COVID-19. So the score was assigned based on these features and discussion among our multi-disciplinary team.

Validation of the screening score

As the screening score was developed based on the current literature and discussion among the multi-disciplinary team in the People’s Hospital of Changshou District, we used the data in this hospital to internally validate this score. In addition, we used the data in the People’s Hospital of Yubei District to externally validate this score to test the generalizability.

Statistical analysis

Categorical variables were reported as numbers and percentages, and the differences between groups were analyzed using the Chi-square test or Fisher’s exact test. Continuous variables were reported as mean value and standard deviation or median value and interquartile range as appropriate. The differences between groups were analyzed with the use of unpaired Student’s t test or the Mann–Whitney U test. The area under the receiver operating characteristic curves (AUC) was used to test the distinguishing power. The optimal cut-off value was chosen by the maximum Youden index.[21] A p value less than 0.05 was considered as significant difference.

Results

We enrolled 76 patients in the internal validation cohort and 40 cases in the external validation cohort (Table 2). The screening score among the confirmed or probable patients was the highest. The score in the patients who were excluded in the isolation ward was lower than that in confirmed or probable cases. It was lowest among the patients who were excluded in the general ward.

Table 2.

Demographics and screening scores in internal and external validation cohorts.

	Internal validationn = 76	External validationn = 40	p
Age, years	54 ± 18	44 ± 18	<0.01
Female/male	37/39	14/26	0.17
Underlying disease
Hypertension	15 (20%)	3 (8%)	0.11
Diabetes mellitus	9 (12%)	2 (5%)	0.33
Chronic respiratory disease	15 (20%)	3 (8%)	0.11
Chronic cardiac disease	5 (7%)	0 (0%)	0.16
Others	14 (18%)	7 (18%)	>0.99
Days from onset to admission	4 (2–8)	4 (1–9)	0.64
Screening scores
Excluded cases in general ward[#]	46 ± 10	49 ± 6	0.25
Excluded cases in isolation ward[#]	68 ± 11	74 ± 9	0.05
Probable cases in isolation ward[#]	120 ± 9	99 ± 9	<0.01
Confirmed cases in isolation ward[#]	107 ± 25	105 ± 23	0.86

The number of cases in each group was calculated after two consecutive real-time reverse transcription polymerase chain reaction (RT-PCR) tests.

The patient triage is illustrated in Figures 1 and 2. After two consecutive RT-PCR tests, 49 cases were excluded, 20 were confirmed, and seven were probable in the internal validation cohort. However, the seven probable patients received multiple tests at the attending physician’s discretion. Finally, four out of seven were confirmed (one was positive at the third RT-PCR test obtained sample from nasopharyngeal swab, one was positive at the fifth RT-PCR test obtained sample from anal swab, and the other two were positive both in special IgG and IgM). In the external validation cohort, no further test was performed in the probable cases after two consecutive RT-PCR tests.

Figure 1.

The flow chart of patient allocation in the internal validation cohort.

RT-PCR, real-time reverse transcription polymerase chain reaction

Figure 2.

The flow chart of patient allocation in the external validation cohort.

RT-PCR, real-time reverse transcription polymerase chain reaction

Among all the suspected patients, the AUC was 0.96 [95% confidence interval (CI): 0.89–0.99] to distinguish the moderate to high probability of COVID-19 in the internal validation cohort (Table 3). Using 60 as cut-off value, the sensitivity and specificity were 88% and 93%, respectively. Similar results were confirmed in the external validation cohort.

Table 3.

Identification of patients with moderate to high probability of COVID-19 among all the suspected patients.

	AUC (95% CI)	Cut-off value	SE	SP	PPV	NPV
Internal validation	0.96 (0.89–0.99)	>60	88%	93%	96%	81%
External validation	1.00 (0.91–1.00)	>60	100%	100%	100%	100%

AUC, area under the curve of receiver operating characteristics; CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value; SE, sensitivity; SP, specificity.

Among the patients in an isolation ward, the AUC was 0.94 (95% CI: 0.83–0.99) to distinguish the confirmed and probable cases in the internal validation cohort (Table 4). Using 90 as cut-off value, the sensitivity and specificity were 78% and 100%, respectively. Similar results were confirmed in the external validation cohort.

Table 4.

Identification of confirmed and probable cases in the isolation ward.

	AUC (95% CI)	Cut-off value	SE	SP	PPV	NPV
Internal validation	0.94 (0.83–0.99)	>90	78%	100%	100%	79%
External validation	0.93 (0.77–0.99)	>90	82%	100%	100%	89%

AUC, area under the curve of receiver operating characteristics; CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value; SE, sensitivity; SP, specificity.

Among all the suspected cases, all the patients with screening score ⩽60 were excluded after two consecutive RT-PCR tests (Figure 3). Among those with screening score between 61 and 90, most of the patients were excluded after two consecutive RT-PCR tests (75% in the internal validation cohort and 89% in the external validation cohort). And among the cases with screening score >90, all the patients were confirmed or probable cases. In the internal validation cohort, seven cases were probable after two consecutive RT-PCR tests. However, 57% (4/7) of probable cases were confirmed after multiple tests.

Figure 3.

The screening efficiency in the internal and external validation cohorts.

RT-PCR, real-time reverse transcription polymerase chain reaction

Discussion

The current study developed a scoring system on COVID-19 triage at fever clinics and isolation wards. It had good predictive power to distinguish the moderate to high probability of COVID-19 at fever clinics. The predictive power also reached ‘good’ to identify the probable and confirmed cases in an isolation ward.

COVID-19 is a person-to-person transmission disease.[1,2] It has spread worldwide. In the middle of March 2020, newly confirmed cases were more than 10,000 per day. The fast increasing cases during so short a period largely overloaded the fever clinics. However, suspected cases must be treated seriously. If a patient with COVID-19 was misdiagnosed and allocated to a general ward without any protection, the clinical workers and other patients in the general ward would face high risk of nosocomial infection. However, if a patient without COVID-19 was allocated to an isolation room, it would largely increase the burden of the health care workers and the consumption of protective supplies. The screening score we developed can help the clinical staff to rapidly allocate the suspected patients. Both in internal and external validation cohorts, all the patients with screening score ⩽60 were eventually excluded. However, to prevent misdiagnosis, these patients were allocated to single rooms in general wards with level-2 protection. If two consecutive RT-PCT tests were negative, these patients were treated as ordinary cases. So, it can balance the misdiagnosis and increased work load and protective supplies consumption.

Current studies have reported that two consecutive negative RT-PCR tests did not totally rule out diagnosis of COVID-19.[9,12,13] In our hospital, we treated the probable cases as confirmed patients. In both internal and external validation cohorts, the patients with screening score >90 were confirmed or probable cases. In the internal validation cohort, there were seven probable cases after two consecutive RT-PCR tests. These patients performed multiple tests at the attending physician’s discretion, and eventually four of seven were confirmed. However, the probable cases in the external validation cohort only performed two consecutive RT-PCR tests based on the recommendation of our National Health Commission. Maybe, among the probable patients, there are some atypical cases in the external validation cohort. Therefore, treatment of the probable cases as confirmed ones was reasonable. And using the screening score of 90 as a cut-off value can help the clinical staff to maximally identify the probable and the confirmed cases.

In the patients with screening score between 61 and 90, most of the cases were excluded after two consecutive RT-PCR tests. However, still 11–25% of cases were confirmed in our study. How to rule out these patients with two negative RT-PCR tests is important. A multi-disciplinary team was needed to discuss these patients and decide where to go. For the probable cases, multiple tests, obtaining samples from multiple sites, and multiple methods (RT-PCR assay, antibody assay, viral gene sequence, etc.) were encouraged.

The scoring system in our study was developed based on the patients admitted to fever clinics. It was not suitable for critical patients who required admission to an intensive care unit for intensive care (e.g. severe acute respiratory infection, acute respiratory distress syndrome and sepsis). However, the proportion of critical cases was only 5%.[22] Therefore, during the epidemic of COVID-19, this scoring system can be used in most of the suspected patients.

Our study may be limited by the methodology. First, the scoring system was developed based on the current literature and discussion among a multi-disciplinary team. The data were retrospectively collected to validate this score. Therefore, prospective validation was encouraged to confirm the effect of this scoring system. Second, the epidemiological history in the screening score was based on the status of China. Now COVID-19 has spread worldwide. The epidemiological history must be revised based on the actual conditions in different countries and areas. Finally, the radiologists were of varying experience in the real world. Strict training on CT reading is required before using this scoring system. In addition, artificial intelligence has been used to read CT to assist COVID-19 diagnosis.[23] Therefore, the rating of the CT score can be improved in the centers which can provide artificial intelligence.

Conclusion

The screening score took into account epidemiology history, clinical features, radiographic features, and routine blood tests to allocate suspected patients with COVID-19. It provides a reference on COVID-19 triage to reduce misdiagnosis and consumption of protective supplies.

Click here for additional data file.

Supplemental material, Author_Response for Definition and retrospective application of a clinical scoring system for COVID-19 triage at presentation by Jun Duan, Mei Liang, Yongpu Li, Dan Wu, Ying Chen, Shui Gao, Ping Jia, Mei Yang, Wei Xia, Xiaolan Wu, Quan Li, Fulin Zuo, Yahong Zhang, Yongfang He, Jianghua Nie, Wenxiu Zhou, Xueqin Fu, Xiaobin Peng, Zhoujun Ma, Xiaofeng Fu, Lingwei Zeng, Wenyi You, Yuan Fang, Lingmei Zhu and Ping Liu in Therapeutic Advances in Respiratory Disease

Click here for additional data file.

Supplemental material, Reviewer_1_v.1 for Definition and retrospective application of a clinical scoring system for COVID-19 triage at presentation by Jun Duan, Mei Liang, Yongpu Li, Dan Wu, Ying Chen, Shui Gao, Ping Jia, Mei Yang, Wei Xia, Xiaolan Wu, Quan Li, Fulin Zuo, Yahong Zhang, Yongfang He, Jianghua Nie, Wenxiu Zhou, Xueqin Fu, Xiaobin Peng, Zhoujun Ma, Xiaofeng Fu, Lingwei Zeng, Wenyi You, Yuan Fang, Lingmei Zhu and Ping Liu in Therapeutic Advances in Respiratory Disease

Click here for additional data file.

Supplemental material, Reviewer_2_v.1 for Definition and retrospective application of a clinical scoring system for COVID-19 triage at presentation by Jun Duan, Mei Liang, Yongpu Li, Dan Wu, Ying Chen, Shui Gao, Ping Jia, Mei Yang, Wei Xia, Xiaolan Wu, Quan Li, Fulin Zuo, Yahong Zhang, Yongfang He, Jianghua Nie, Wenxiu Zhou, Xueqin Fu, Xiaobin Peng, Zhoujun Ma, Xiaofeng Fu, Lingwei Zeng, Wenyi You, Yuan Fang, Lingmei Zhu and Ping Liu in Therapeutic Advances in Respiratory Disease

Click here for additional data file.

Supplemental material, Supplementary_materials for Definition and retrospective application of a clinical scoring system for COVID-19 triage at presentation by Jun Duan, Mei Liang, Yongpu Li, Dan Wu, Ying Chen, Shui Gao, Ping Jia, Mei Yang, Wei Xia, Xiaolan Wu, Quan Li, Fulin Zuo, Yahong Zhang, Yongfang He, Jianghua Nie, Wenxiu Zhou, Xueqin Fu, Xiaobin Peng, Zhoujun Ma, Xiaofeng Fu, Lingwei Zeng, Wenyi You, Yuan Fang, Lingmei Zhu and Ping Liu in Therapeutic Advances in Respiratory Disease