Changes in diagnostic usefulness of the JRS scoring system in COVID-19 pneumonia by SARS-CoV-2 vaccination.

INTRODUCTION: The Japanese Respiratory Society (JRS) scoring system is a useful tool for the rapid presumptive diagnosis of atypical pneumonia in non-elderly (aged <60 years) patients. As SARS-CoV-2 vaccination progresses, COVID-19 in elderly people has markedly reduced. We investigated changes in diagnostic usefulness of the JRS scoring system in COVID-19 pneumonia between the Delta variant group (vaccination period) and non-Delta variant group (before the vaccination period).
METHODS: This study was conducted at five institutions and assessed a total of 1121 patients with COVID-19 pneumonia (298 had the Delta variant). During the vaccination period, the Delta variant has spread and replaced the Alfa variant. We evaluated the vaccination period as the Delta variant group.
RESULTS: Among the six parameters of the JRS scoring system, matching rates of two parameters were higher in the Delta variant group than the non-Delta variant group (pre-vaccination period): age <60 years (77.5% vs 42.2%, P < 0.0001) and no or minor comorbid illness (69.1% vs 57.8%, p = 0.0007). The sensitivity of the diagnosis of atypical pneumonia in patients with COVID-19 pneumonia was significantly higher in the Delta variant group compared with the non-Delta variant group (80.2% vs 58.3%, p < 0.0001). When the diagnostic sensitivity was analyzed for different ages, the diagnostic sensitivities for the Delta variant and non-Delta variant groups were 92.6% and 95.5% for non-elderly patients and 39.1% and 32.5% for elderly patients, respectively.
CONCLUSIONS: Our results demonstrated that the JRS scoring system is a useful tool for distinguishing between COVID-19 pneumonia and bacterial pneumonia in the COVID-19 vaccination period, but not before the vaccination period.

Coronavirus disease 2019

Computed tomography

Japanese Respiratory Society

Reverse transcription polymerase chain reaction

Severe acute respiratory syndrome coronavirus 2

White blood cell

Introduction

Pneumonia including aspiration pneumonia is the third leading cause of mortality in Japan. Since 2020, the novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) became the major causative microorganism of pneumonia [1]. Among diagnostic methods, reverse transcription polymerase chain reaction (RT-PCR) assays are thought to be the gold standard for diagnosing coronavirus disease 2019 (COVID-19). However, with oropharyngeal and nasopharyngeal swab specimens, the sensitivity of RT-PCR is not high and depends on the time of collection and the collector [2,3]. In addition, RT-PCR assays are not used for point-of-care testing in daily clinical situations. In contrast to RT-PCR assays, antigen detection assays are rapid, simple diagnostic tests, but their sensitivity is low. Furthermore, some physicians do not carry out RT-PCR or antigen detection tests to avoid the droplet infection or airborne infection in the examination room. Thus, rapid, simple, and non-dangerous testing for the diagnosing COVID-19 is important.

The term ‘atypical pneumonia’ was first applied to viral pneumonia, which was clinically and radiologically distinct from bacterial pneumonia. The Japanese Respiratory Society (JRS) pneumonia guidelines proposed a differential diagnosis between atypical pneumonia and bacterial pneumonia using a rapid and simple scoring system [4]. The JRS extracted six parameters from patients with Mycoplasma pneumoniae pneumonia using multiple regression analysis [4]. Our previous study demonstrated that the JRS scoring system is a useful tool in non-elderly patients (<60 years old), but not useful in elderly patients ( ≥ 60 years old) because M. pneumoniae infections occur predominantly in school-aged children and younger adults [5]. COVID-19 pneumonia occurred mainly in elderly people ( ≥ 60 years old) in the pre-vaccination period in Japan.

In April 2021, vaccination against SARS-CoV-2 was started in elderly people, and infection has subsequently shifted from elderly people to younger age groups [6]. With younger age groups being affected by pneumonia, the JRS scoring system is predicted to be a useful tool for the presumptive diagnosis of COVID-19 pneumonia. In this study, we investigated changes in diagnostic usefulness of the JRS scoring system in COVID-19 pneumonia as SARS-CoV-2 vaccination progresses. We compared the diagnostic sensitivity of the JRS scoring system between Delta variant group (vaccination period) and non-Delta variant group (before the vaccination period).

Methods

Study population

The present study was conducted at five institutions (Kansai Medical University Hospital, Kansai Medical University Medical Center, Kansai Medical University Kori Hospital, Kansai Medical University Kuzuha Hospital, and Kansai Medical University Temmabashi General Clinic) between February 2020 and December 2022. We enrolled adult patients consecutively diagnosed with community-onset pneumonia, defined in accordance with the JRS guidelines [4]. The diagnosis was based on clinical signs and symptoms (cough, fever, productive sputum, dyspnea, chest pain, or abnormal breath sounds) and radiographic pulmonary abnormalities that were at least segmental and were not as a result of pre-existing or other known causes. Exclusion criteria included the following: immunosuppressive illness (i.e., HIV positive, neutropenia secondary to chemotherapy, use of >20 mg/day prednisone or other immunosuppressive agents, and history of organ transplant) and active tuberculosis. All cases of pneumonia occurring more than three days after hospitalization were considered nosocomial and were excluded.

COVID-19 was diagnosed using a positive RT-PCR results from sputum or nasopharyngeal swab specimens according to the protocol recommended by the National Institute of Infectious Diseases, Japan. From June 2021, the fifth wave of COVID-19 began with a new lineage of SARS-CoV-2, the Delta variant, which spread rapidly throughout Japan, and there was 100% replacement of previous variants by the Delta variant in July 2021. We evaluated the JRS scoring system in COVID-19 pneumonia and compared it with the Delta variant group (vaccination period) and non-Delta variant group (before the vaccination period). Other microbiological tests, Gram stain, cultures, antigen detection tests, PCR, and serological tests for detection of common bacteria were performed as described previously [5]. Informed consent was obtained from all patients, and the study protocol was approved by the Ethics Committee of Kansai Medical University (approval number 2020319).

Clinical pathway for the management of pneumonia

Pneumonia is divided into two groups, community-onset pneumonia and hospital-onset pneumonia. At first, physicians evaluate the pneumonia severity to determine the treatment location. In the case of community-onset pneumonia, the severity of pneumonia was evaluated using predictive rules via the A-DROP system proposed by the JRS guidelines [4]. The total score for A-DROP of the 6-point scoring system was calculated by adding a point for 1) age over 70 years in men and over 75 years in women, 2) dehydration (blood urea nitrogen ≥21 mg/dL), 3) respiratory failure (SpO2 ≤90% or PaO2 ≤60 mmHg), 4) orientation disturbance, or 5) low blood pressure (systolic blood pressure ≤90 mmHg). Patients were stratified into four severity classes, mild (0 point, outpatient), moderate (1 or 2 point, outpatient or short-term hospitalization), severe (3 point, hospitalization) and extremely severe (4 or 5 point, intensive care unit [ICU]). Next, the JRS guidelines recommend pathogen-oriented treatment as the initial appropriate therapy in cases in which an etiologic diagnosis is established or strongly suspected. Thus, use of rapid diagnostic tests such as sputum Gram staining or rapid antigen detection assays before antibiotic treatment is recommended. When the causative pathogen was not detected using rapid diagnostic tests, next the JRS guidelines propose a differential diagnosis for bacterial pneumonia and atypical pneumonia using a scoring system in the empirical antibacterial selection. Penicillins with or without a beta-lactamase inhibitor, or cephalosporins, are consider appropriate empirical therapy for suspected bacterial pneumonia. If atypical pneumonia is suspected, then the guidelines recommend the use of macrolides or tetracyclines. The basic selection of these antibiotics is acceptable for patients who have no co-morbid diseases or are younger. Therefore, JRS guidelines also added options for the selection of other antibiotics when patients have co-morbid diseases, are more than 65 years old or have used antibiotics recently. In severe pneumonia patients who need ICU admission, the JRS guidelines propose combination therapy with macrolides plus β-lactams.

Scoring system of the JRS guidelines

The JRS guidelines were selected to allow for the easy differentiation of pneumonia without special examinations [4]. The data obtained from three prospective studies of pneumonia (a total of 1880 patients) were analyzed, and guideline members extracted six parameters from frequently observed background factors, clinical symptoms, and laboratory findings of patients with M. pneumoniae pneumonia [4,7]. These parameters were: 1) age <60 years, 2) no or minor comorbid illness, 3) presence of stubborn cough, 4) absence of chest adventitious sounds, 5) no sputum or no identified etiological agent using rapid diagnostic tests (Gram staining, urinary antigen tests, and nasopharyngeal antigen test), and 6) a peripheral white blood cell (WBC) count <10,000/μL. The presence of at least four out of the six parameters pointed to a suspicion of M. pneumoniae pneumonia [4]. Patients with extremely severe pneumonia, an A-DROP score of 4 or 5 point, were excluded from this study.

Statistical analysis

Discrete variables are expressed as counts (percentages) and continuous variables as medians and interquartile ranges. Frequencies were compared using Fisher's exact test. Between-group comparisons of normally distributed data were performed using Student's t-test. Skewed data were compared using the Mann–Whitney U test.

Results

Patient characteristics

During the study period, all pneumonia patients were received RT-PCR for detection of SARS-CoV-2. Total of 1121 patients with COVID-19 pneumonia and 497 patients with COVID-19 PCR-negative pneumonia (non-COVID-19 pneumonia) were recorded. When the RT-PCR was negative, all patients were received urinary antigen tests for detection of Streptococcus pneumoniae and Legionella spp. Blood culture was done in all hospitalized patients. Gram stain and bacterial culture were performed in 60% patients who was able to collect sputum. When the M. pneumoniae pneumonia was suspected, real-time PCR and serological test were performed as described previously [5]. Cases of pneumonia mixed with other microorganisms were excluded from the study.

Of the 1121 COVID-19 pneumonia patients, 298 had the Delta variant. Of the 823 non-Delta variant group, 659 were inpatients including hotel recuperation and 164 were outpatients. Of the 298 Delta variant group, 223 were inpatients including hotel recuperation and 75 were outpatients.

Of the 497 non-COVID-19 pneumonia cases, the microbial etiology was established in 248 cases; S. pneumoniae in 126 cases, Haemophilus influenzae in 48 cases, M pneumoniae in 32, Moraxella catarrhalis in 20, Staphylococcus aureus in 16, and Klebsiella pneumoniae in 6.

The median age of patients in the non-Delta variant group was 65 years old, but patients in the Delta variant group were significantly younger at 50 years old (p < 0.0001) (Table 1 ). As expected, the prevalence of comorbid illness and pneumonia severity were significantly lower in the Delta variant group than the non-Delta variant group. Forty-one patients (13.8%) had been vaccinated (BNT162b2 or mRNA-1273) against SARS-CoV-2, of which 32 patients had received one dose and 9 patients received two doses.

Table 1

Underlying conditions and clinical findings in patients with COVID-19 pneumonia in the Delta variant and non-Delta variant groups at first examination.

Variables	Delta variant	Non-Delta variant	p-value
No. of patients	298	823
Median age (IQR), years	50 (39–58)	65 (48–75)	<0.0001
No. of males/females	167/131	529/294	0.147
No. (%) of patients with comorbid illnesses
Diabetes mellitus	43 (14.4)	165 (20.0)	0.037
Chronic lung disease	28 (9.4)	98 (11.9)	0.284
Chronic heart disease	11 (3.7)	61 (7.4)	0.027
Neoplastic disease	11 (3.7)	44 (5.3)	0.347
Cerebrovascular disease	10 (3.4)	52 (6.3)	0.056
Chronic liver disease	10 (3.4)	24 (2.9)	0.696
Chronic renal disease	5 (1.7)	55 (6.7)	0.001
Autoimmune disease	5 (1.7)	22 (2.7)	0.388
No. (%) of patients with the following clinical signs and symptoms
History of fever (≥37.0 °C)	269 (90.3)	708 (86.0)	0.069
Cough	224 (75.2)	458 (55.7)	<0.0001
Fatigue	122 (40.9)	277 (33.7)	0.0004
Shortness of breath	86 (28.9)	241 (29.3)	0.941
Sore throat	79 (26.5)	165 (20.0)	0.022
Loss of taste	71 (23.8)	120 (14.6)	0.0004
Anosmia	71 (23.8)	103 (12.5)	<0.0001
Sputum production	55 (18.5)	103 (12.5)	0.015
Headache	37 (12.4)	87 (10.6)	0.390
Joint pain	20 (6.7)	42 (5.1)	0.302
Runny nose	11 (3.7)	60 (7.3)	0.027
Nausea or vomiting	11 (3.7)	30 (3.6)	>0.999
Muscle ache	11 (3.7)	21 (2.6)	0.314
Diarrhea	8 (2.7)	78 (9.5)	<0.0001
Chest pain	6 (2.0)	23 (2.8)	0.531
Abdominal pain	6 (2.0)	7 (0.9)	0.120
Laboratory findings, median (IQR)
White blood cell count,/μL	5000 (3825–6475)	5200 (4340–6700)	0.734
No. (%) of patients with each pneumonia severity score*
0	162 (54.4)	254 (30.9)	<0.0001
1	94 (31.5)	262 (31.8)	0.942
2	30 (10.1)	198 (24.1)	<0.0001
3	12 (4.0)	108 (13.1)	<0.0001

Continuous values are presented as medians and interquartile ranges (IQRs) and categorical/binary values as counts and percentages.

Rates of conformity to the six parameters of the JRS scoring system

Fig. 1 shows the rates of conformity to the parameters of the guideline criteria of patients with COVID-19 pneumonia in the Delta variant and non-Delta variant groups at the first examination. Rates of conformity were higher in the Delta variant group than the non-Delta variant group in the following parameters: age <60 years (77.5% vs 42.2%, P < 0.0001) and no or minor comorbid illness (69.1% vs 57.8%, p = 0.0007). Rates of conformity of other parameters were identical in both groups, and high rates of conformity were observed in the following parameters: absence of chest adventitious sounds (Delta variant group, 73.8%; non-Delta variant group, 71.6%); no sputum or no identified etiological agent using rapid diagnostic tests (81.5% and 87.1%); and a peripheral WBC count <10,000/μL (98.7% and 97.6%). Cough is a common symptom in COVID-19 pneumonia, but the prevalence of stubborn or paroxysmal cough was low in both groups (Delta variant group, 13.1%; non-Delta variant group, 10.4%).

Fig. 1

Rates of conformity for the six parameters of the Japanese Respiratory Society scoring system in patients with COVID-19 pneumonia in the Delta variant and non-Delta variant groups.

Sensitivities and specificities of the JRS criteria

The number of conforming parameters and their sensitivity for the diagnosis of atypical pneumonia are shown in Table 2 . The sensitivities of the diagnosis of atypical pneumonia in patients with COVID-19 pneumonia based on four or more parameters were 80.2% in the Delta variant group and 58.3% in the non-Delta variant group, respectively. Using the confirmed 216 bacterial pneumonia and 32 atypical pneumonia (M. pneumoniae), the sensitivity and specificity of the diagnosis of atypical pneumonia based on four or more parameters were 87.5% and 92.1%, respectively.

Table 2

Sensitivities and specificities of the JRS criteria in patients with COVID-19 pneumonia in the Delta variant and non-Delta variant groups.

No. of features	No. of matching Delta variant (sensitivity, %)	No. of matching non-Delta variant (sensitivity, %)	Specificity (%)a
≥1	298 (100)	822 (99.9)	20.4
≥2	292 (98.0)	791 (96.1)	54.6
≥3	282 (94.6)	682 (82.9)	78.2
≥4	239 (80.2)	480 (58.3)	92.1
≥5	109 (36.6)	221 (26.9)	98.1
≥6	8 (2.7)	25 (3.0)	100

Specificity was calculated using the confirmed 216 bacterial pneumonia.

Diagnostic sensitivity in different age groups

Our previous studies demonstrated that the diagnostic sensitivity of the guideline criteria for the diagnosis of atypical pneumonia was significantly lower in the elderly group (aged ≥60 years) than in the non-elderly group (aged <60 years) [5]. Thus, we evaluated the JRS scoring system in different age groups. Table 3 shows the sensitivity for the diagnosis of COVID-19 pneumonia stratified in 10-year age groups. The diagnostic sensitivity of patients in both the Delta variant and non-Delta variant groups was highest in the 20–29-year age group and decreased in order from the youngest to the oldest age group. The diagnostic sensitivity in each age group was identical in both groups. The diagnostic sensitivities for Delta variant and non-Delta variant groups were 92.6% and 95.5% for non-elderly (aged <60 years) patients and 39.1% and 32.5% for elderly (aged ≥60 years) patients, respectively.

Table 3

Sensitivity for the presumptive diagnosis of atypical pneumonia in different age groups among patients with COVID-19 pneumonia in the Delta variant and non-Delta variant groups.

Age group, years	Delta variant positive cases/number (%)	Non-Delta variant positive cases/number (%)	p-value
20–29	31/31 (100)	70/70 (100)
30–39	44/46 (95.7)	69/69 (100)
40–49	61/64 (95.3)	78/79 (98.7)
50–59	76/88 (86.4)	105/119 (96.3)
60–69	11/28 (39.3)	76/167 (45.5)
70–79	9/21 (42.9)	52/202 (25.7)
>80	7/20 (35.0)	30/117 (25.6)
Total	239/298 (80.2)	480/823 (58.3)	<0.0001

The data represent the number of patients, and numbers in parentheses are percentages.

Discussion

One feature of the JRS guidelines is that it tries to differentiate atypical pneumonia, mainly M. pneumoniae pneumonia and bacterial pneumonia, for the selection of antibiotics. Several studies demonstrated high rates of conformity to the six parameters of the JRS scoring system among patients with M. pneumoniae pneumonia [[7], [8], [9], [10]]: parameters: 1) age (83.0%), 2) comorbid illness (87.9%), 3) cough (75.3%), 4) chest auscultation findings (70.3%), 5) rapid diagnostic tests (80.2%), and 6) WBC counts (82.4%) [4]. In the present study, we evaluated whether the JRS scoring system can be used to differentiate COVID-19 pneumonia from bacterial pneumonia. The rates of conformity in parameters 1, 2, and 3 in the COVID-19 pneumonia non-Delta variant group were 42.2%, 57.8%, and 10.4%, respectively, which were significantly lower than M. pneumoniae pneumonia. As COVID-19 vaccination progresses, however, the rates of conformity in parameters 1 and 2 increased significantly in the Delta variant group more than the non-Delta variant group (Fig. 1). For this reason, the diagnostic sensitivity increased significantly in the Delta variant group compared with the non-Delta variant group (80.2% vs 58.3%, p < 0.0001).

Although M. pneumoniae pneumonia is significantly more common in younger patients [4,5,[7], [8], [9], [10]], the median age of patients with COVID-19 pneumonia is higher than that of patients with M. pneumoniae pneumonia but lower than that of patients with bacterial pneumonia. We then evaluated the accuracy and usefulness of the JRS scoring system in different age groups. The diagnostic sensitivity was highest among patients aged 20–29 years and decreased in order from the youngest to the oldest age group. There was a clear difference between elderly (aged ≥60 years) and non-elderly (aged <60 years) patients with COVID-19 pneumonia in both the Delta variant and non-Delta variant groups. The diagnostic sensitivity for COVID-19 pneumonia was 94.3% for non-elderly patients and 33.3% for elderly patients.

When the COVID-19 pneumonia was classified as an atypical pneumonia using the JRS scoring system, physicians need to distinguish COVID-19 pneumonia from M. pneumoniae pneumonia. The former studies indicate that the diagnosis of M. pneumoniae pneumonia would appear reliable when a combination of bronchial wall thickening and tree-in-bud and centrilobular nodules and/or ground-glass opacity (GGO) with lobular distribution are found on CT findings [11,12]. Typical findings on chest CT among patients with COVID-19 pneumonia were peripheral GGOs with or without consolidation or a crazy-paving pattern and multifocal GGO with rounded morphology [[13], [14], [15], [16], [17], [18]]. Bronchial wall thickening and tree-in-bud and centrilobular nodules are rarely observed in COVID-19 pneumonia. Although physicians may differentiate typical COVID-19 pneumonia from typical M. pneumoniae pneumonia using chest CT findings, CT findings changes over time [19,20]. Thus, discrimination by image is controversial.

The basic policy and main purposes of the JRS pneumonia guidelines include; 1) prevention of bacterial resistance and 2) effective and long-term use of medical resources [4]. Thus, the JRS guidelines have been recommended the prediction of causative microorganisms for the selection of appropriate antibiotics. However, JRS scoring system is an auxiliary diagnosis not definitive diagnostic method. To the last, definitive diagnosis of COVID-19 is detection of SARS-CoV-2 using RT-PCR or antigen detection assay.

Our study had several limitations. First, parameter 3 of the JRS scoring system, presence of stubborn cough, is subjective. Thus, individual physicians may differ in their judgments about them. Second, many patients had normal chest auscultatory findings at the first examination, but different adventitious sounds were heard as time progressed. Thus, physicians should be aware that chest auscultatory findings depend on the timing of the examination when the JRS scoring system is used. Third, we excluded patients with severe COVID-19 pneumonia. The JRS scoring system applies only to mild-to-moderate pneumonia [4].

In conclusion, as vaccination against SARS-CoV-2 progresses, infection in elderly people has markedly reduced, and the number of infected people in their 20s to 40s has increased [6]. Our results demonstrated that the JRS scoring system is a useful tool for distinguishing between COVID-19 pneumonia and bacterial pneumonia in the COVID-19 Delta variant group (vaccination period), but not non-Delta variant group (before the vaccination period).

Funding

No funding was received.

Availability of data and materials

The data will not be shared because of participant confidentiality.

Author's contributions

All the authors conceived the study, participated in its design and coordination, and collected and managed the data, including quality control. NM, YN, and SN drafted the manuscript, and all authors contributed substantially to its revision. All the authors read and approved the final manuscript.

Ethical approval and consent to participate

The study protocol was approved by the Ethics Committee at Kansai Medical University and all participating facilities. Informed consent was obtained from all individual participants in the study.

Consent for publication

Not applicable.

Declaration of competing interest

The authors declare that they have no competing interests.